Skip to content

Commit 67315e3

Browse files
Add GSoC 2026 project proposal for Kubeflow SDK OpenTelemetry integration
Signed-off-by: Dhanisha Phadate <dhanisha.6@gmail.com>
1 parent 77a1ec8 commit 67315e3

1 file changed

Lines changed: 62 additions & 0 deletions

File tree

content/en/events/upcoming-events/gsoc-2026.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -298,6 +298,66 @@ Tracking issue: https://github.com/kubeflow/sdk/issues/238
298298
- Understanding of the Kubeflow Ecosystem and basic Kubernetes concepts.
299299
- Engage and contribute to Kubeflow community on Slack and GitHub.
300300

301+
### Project 7 : Integrate Kubeflow SDK with OpenTelemetry
302+
303+
**Components:** [kubeflow/sdk](https://www.github.com/kubeflow/sdk)
304+
305+
**Mentors:** [@kramaranya](https://github.com/kramaranya), [@dhanishaphadate](https://github.com/dhanishaphadate), [@jaiakash](https://www.github.com/jaiakash)
306+
307+
**Contributor:**
308+
309+
**Details:**
310+
311+
The Kubeflow SDK enables users with limited Kubernetes knowledge to interact with the Kubeflow ecosystem using standard Python APIs. As AI/ML workloads become more complex and distributed, observability into pipeline execution, model training, and inference workflows becomes critical.
312+
313+
This project aims to integrate the Kubeflow SDK with OpenTelemetry (OTel) to provide standardized, vendor-neutral telemetry for Kubeflow-based workloads. The integration will enable end-to-end visibility into SDK operations by capturing distributed traces, metrics, and logs across pipeline compilation, submission, execution, and training lifecycles.
314+
315+
The project will also explore leveraging existing OpenTelemetry and Generative AI instrumentation patterns—such as span conventions for model execution, prompt handling, and inference steps—where applicable.
316+
317+
[Kubeflow SDK Documentation](https://sdk.kubeflow.org/en/latest/index.html)
318+
319+
[Opentelemetry for genAI](https://opentelemetry.io/blog/2024/otel-generative-ai/)
320+
321+
[Issue](https://github.com/kubeflow/sdk/issues/164)
322+
323+
**Features Expected:**
324+
325+
- Add OpenTelemetry instrumentation to key Kubeflow SDK components
326+
- Enable distributed tracing for pipeline execution and SDK operations
327+
- Collect and export metrics related to AI/ML workloads
328+
- Provide configurable OTel exporters and sampling options
329+
- Documentation and examples demonstrating observability setup and usage
330+
- cover below SDK clients
331+
332+
```
333+
kubeflow/
334+
├── trainer/ # TrainerClient - distributed training & fine-tuning
335+
├── optimizer/ # OptimizerClient - Katib AutoML & hyperparameter tuning
336+
├── hub/ # ModelRegistryClient - model artifact management
337+
└── common/ # Shared utilities across clients
338+
```
339+
340+
| SDK Client | Component |
341+
|------------|-----------|
342+
| `TrainerClient` | Kubeflow Trainer |
343+
| `OptimizerClient` | Kubeflow Katib |
344+
| `ModelRegistryClient` | Model Registry |
345+
| `PipelinesClient` | Kubeflow Pipelines |
346+
| `SparkClient` | Spark Operator |
347+
348+
349+
**Difficulty:** [intermediate|hard]
350+
351+
**Size:** [350 hours]
352+
353+
**Skills Required/Preferred:**
354+
355+
- Python
356+
- Understanding of the Kubeflow Ecosystem (preferred)
357+
- OpenTelemetry (tracing, metrics, logging)
358+
- Distributed systems and observability concepts
359+
- Kubernetes and CRDs
360+
301361
### Project 10: Dynamic LLM Trainer Framework for Kubeflow
302362

303363
**Components:**
@@ -349,3 +409,5 @@ Tracking issue: [kubeflow/trainer#2839](https://github.com/kubeflow/trainer/issu
349409
* Experience with LLM fine-tuning frameworks (TRL, TorchTune, Unsloth)
350410
* Understanding of distributed training concepts
351411
* Interest in API and framework design
412+
413+
---

0 commit comments

Comments
 (0)