The local-first Data + ML + AI workbench. One command, zero microservices, everything in one process.
Jupyter is great for notebooks. Airflow is great for orchestration. Streamlit is great for dashboards. This is all of them in one app — no stitching together half a dozen tools, no Python ↔ HTTP hops, no data leaving your laptop unless you choose to.
docker compose up
open http://localhost:7860After login → head to Getting Started for your first project, first pipeline, and first AI agent.
A single-page web UI that gives you ingestion, pipelines, warehouse, ML, AI agents, RAG, PII guardrails, scheduling, monitoring, and logs — all backed by one Python library running in the same process.
| Domain | What you can do |
|---|---|
| Data | Connect sources (CSV, Postgres, Kafka, Spark, dbt, S3, GCS, …), define DAG pipelines, browse bronz/silver/gold warehouse, run SQL, profile quality, explore lineage |
| ML / AI | Train models (sklearn, XGBoost, PyTorch), track experiments in MLflow, serve predictions, detect drift, run RAG pipelines, chat with agents, view traces |
| SecOps | Scan for PII, configure masking strategies, review audit logs, set alert rules and policies |
| System | View pipeline runs, scheduler status, live log tail (SSE), Prometheus metrics, compaction, components |
Each Studio page maps to a dataenginex library call — no REST endpoints to version, no separate API to deploy.
- Data engineers who want a local-first workspace for pipeline development before shipping to prod
- ML engineers who want to train, track, and serve models without infrastructure overhead
- Solo devs / small teams who want one reproducible environment per project, not a platform team
- Anyone tired of stitching together Jupyter + Airflow + MLflow + Streamlit + Grafana just to get work done
| Tool | DEX Studio does that, plus… |
|---|---|
| Jupyter | Persistent pipelines, scheduling, warehouse, auth, multi-project — all in one app |
| Airflow | Local-first, no DB/redis dependencies, ML/AI, PII guardrails, instant startup |
| Streamlit | Multi-page nav, auth, scheduling, persistent state, no st.* DSL |
| Metabase / Grafana | Read-write pipelines, ML training, agent chat, SQL console, not just dashboards |
| MLflow UI | Full data pipeline + warehouse + agent runtime alongside experiment tracking |
git clone https://github.com/TheDataEngineX/dex-studio && cd dex-studio
docker compose up
# open http://localhost:7860uv sync
uv run poe dev # http://localhost:7860 with hot-reloadexport DEX_CONFIG_PATH=/path/to/dex.yaml && dex-studioFull observability stack (dex-studio + Prometheus + Grafana + Tempo + cAdvisor + PostgreSQL + Redis + Kafka + Elasticsearch):
docker compose up -d
# dex-studio: http://localhost:7860
# Grafana: http://localhost:3000 (admin/admin)
# Prometheus: http://localhost:9090
# Tempo: http://localhost:3200
# cAdvisor: http://localhost:8080
# Kafka UI: http://localhost:9091
# Schema Reg: http://localhost:8081Stack includes:
- dex-studio — Web UI (FastAPI + Jinja2 + HTMX)
- Prometheus — Metrics collection & alerting
- Alertmanager — Alert routing
- Grafana — Dashboards & visualization
- Tempo — Distributed tracing
- cAdvisor — Container metrics (OOM, CPU throttle, memory)
- PostgreSQL — Shared state for dex-studio
- Redis — Session store / rate limiting / Celery broker
- Kafka — Streaming message bus (KRaft mode)
- Schema Registry — Avro/Protobuf schema management
- Kafka UI — Browse topics, partitions, messages
- Elasticsearch — Lexical search (movie-dex example)
- DuckDB embedded — no Postgres / Redis for the base install
- Ollama for LLMs — no API keys required; OpenAI / Anthropic are opt-in
- No microservices — FastAPI imports
dataenginexdirectly; same process, no HTTP hop - Portable — all project data lives in
.dex/next to your config; copy the folder, move machines - Privacy — every outbound call is logged; PII guardrails mask before any external request
- Optional scale-out — swap SQLite → PostgreSQL, add Qdrant, add S3, add Kafka — when you need it
| Component | Technology |
|---|---|
| Server | FastAPI + Uvicorn |
| Templates | Jinja2 (server-rendered HTML) |
| Interactivity | HTMX + Alpine.js (no JS build step) |
| Styling | Custom CSS + Radix UI design tokens |
| Engine | dataenginex>=0.5.0 — direct import, no HTTP |
| Persistence | DuckDB (embedded) + optional PostgreSQL / Qdrant / S3 |
| LLM | Ollama (default) + optional OpenAI / Anthropic / LiteLLM |
| Streaming | Kafka / Redpanda (optional) |
| ML Tracking | MLflow (optional) |
| Build | Hatchling + uv |
| Quality | Ruff + mypy strict + pytest |
Key dependency versions (0.5.2): fastapi>=0.139.2, pydantic>=2.13.4, structlog>=26.1.0, sqlalchemy>=2.0.51, scikit-learn>=1.9.0, pandas>=3.0.3, orjson>=3.11.9, httpx>=0.28.1, opentelemetry-sdk>=1.44.0
| Data pipelines | SQL console | Warehouse lineage |
|---|---|---|
 |
 |
 |
| ML models | Agent playground | PII guardrails |
|---|---|---|
 |
 |
 |
| System status | Live logs | Scheduler |
|---|---|---|
 |
 |
 |
Full observability stack (dex-studio + Prometheus + Grafana + Tempo + cAdvisor):
docker compose up -d
# dex-studio: http://localhost:7860
# Grafana: http://localhost:3000 (admin/admin)
# Prometheus: http://localhost:9090
# Tempo: http://localhost:3200
# cAdvisor: http://localhost:8080Stack includes:
- dex-studio — Web UI (FastAPI + Jinja2 + HTMX)
- Prometheus — Metrics collection & alerting
- Alertmanager — Alert routing
- Grafana — Dashboards & visualization
- Tempo — Distributed traces storage
- cAdvisor — Container metrics (OOM, CPU throttle, memory)
- PostgreSQL — Shared state for multi-replica dex-studio
- Redis — Session store / rate limiting
- Kafka — Streaming message bus (optional)
- Elasticsearch — Lexical search (optional)
uv run poe dc-up # Start dex-studio + monitoring stack
uv run poe dc-down # Stop it
uv run poe dc-logs # Tail logs
uv run poe dc-ps # List servicesDefined in pyproject.toml. The compose file at docker-compose.yml includes everything needed for local development.
uv run poe lint # ruff lint
uv run poe lint-fix # ruff lint + auto-fix
uv run poe typecheck # mypy strict
uv run poe test # pytest
uv run poe check-all # lint + typecheck + test
uv run poe dev # uvicorn dev server (port 7860, hot-reload)Design tokens: src/dex_studio/static/studio.css.
| Repo | Description |
|---|---|
| dataenginex | The Python library — engine, config, all backends |
| dex-studio | This repo — web UI |
| infradex | Kubernetes deployment via ArgoCD (private) |
Pre-1.0. Active development. All core phases delivered through 0.5.x. See CHANGELOG.
Contributions welcome — open an issue or PR. The architecture is small enough to hold in your head (one FastAPI app, ~30 source files).
License: MIT • Python: 3.13+ • Port: 7860