Skip to content
View jaymyaka's full-sized avatar

Organizations

@TheDataEngineX

Block or report jaymyaka

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
jaymyaka/README.md

Jayapal Reddy Myaka

Senior Data + AI Engineer · Building production-grade data/ML/AI platforms from scratch.


What I Build

I architect and ship end-to-end data infrastructure — from raw ingestion through medallion lakehouse, ML training, vector search, LLM agent orchestration, self-hosted Kubernetes, and production API serving.

Not glue code. Actual systems.


TheDataEngineX Platform

My flagship OSS project: a self-hosted Python framework for the Data + ML + AI engineering lifecycle. One dex.yaml defines pipelines, models, agents, and observability — with native integrations for Airflow, MLflow, and LangChain when you need their specific capabilities.

CSV / S3 / API
      ↓  connectors + quality gates
Bronze → Silver → Gold  (DuckDB / Parquet medallion lakehouse)
      ↓  ML training pipeline
Model Registry + Experiment Tracker + Drift Monitor
      ↓  LiteLLM / vLLM routing
AI Agents + RAG (Qdrant) + Langfuse tracing
      ↓  DexEngine (Python lib)
DEX Studio (FastAPI/Jinja2)  ·  CareerDEX
      ↓  K3s · Helm · Terraform · Hetzner
Repo What it does Status
dataenginex · PyPI Core framework: config, CLI, ML registry, LLM routing, AI agents, DuckDB lakehouse — pure Python library Production
dex-studio B2B web UI — pipelines, warehouse, ML experiments, AI agents, SQL console (FastAPI/Jinja2 + HTMX) Alpha
careerdex B2C career AI — job matching, resume analysis, interview prep, application tracking Alpha
infradex IaC: K3s cluster, Helm charts (Authentik, Langfuse, Qdrant, Prometheus, Grafana, ArgoCD) Alpha

By the numbers: 1500+ tests · 80%+ coverage · mypy strict · Python 3.13+ · self-hosted on Hetzner K3s


Stack

Data & ML

Python DuckDB PySpark dbt Kafka

AI / LLM

LiteLLM vLLM Qdrant Langfuse scikit-learn

Platform & Infra

FastAPI HTMX Kubernetes Terraform Prometheus Authentik

Cloud

AWS GCP Hetzner


What's Next

A few things I'm actively working on:

  • Port CareerDEX from Reflex to FastAPI/Jinja2 — align with dex-studio's stack so all components share templates, auth, and deployment
  • MCP server — expose DataEngineX tools (pipeline run, SQL query, ML training) as Model Context Protocol endpoints for AI-assisted development
  • More connectors — Kafka source/sink, Postgres replication, Delta Lake support
  • dex-studio GA — authentication, multi-user workspaces, SSO, RBAC (the last big missing pieces before 1.0)
  • Plugin system — third-party connectors and AI tools via a stable plugin API

Links

Org PyPI Website LinkedIn


Senior Data Engineer · AI/ML Platform Engineer · Open to new opportunities

Pinned Loading

  1. TheDataEngineX/dex-studio TheDataEngineX/dex-studio Public

    dex-studio: Self-hosted web UI for DataEngineX — FastAPI + Jinja2 + HTMX, pipeline monitoring, ML experiments, AI playground, SQL console.

    Python 6

  2. TheDataEngineX/dataenginex TheDataEngineX/dataenginex Public

    DataEngineX — open-source, self-hosted, local-first Data + ML + AI workbench library

    Python 5 1

  3. Speech-Emotion-Recognition Speech-Emotion-Recognition Public

    Speech Emotion Recognition in Audio Using ML Techniques

    Jupyter Notebook 2