DLAI - Deadlift AI

Real-time deadlift form analysis using computer vision. Detects barbell position, estimates body pose, computes hip hinge angles, counts reps, and provides live biomechanical feedback.

Architecture

Client (React)  <-- WebSocket -->  Server (FastAPI)
                                       |
                                   Pipeline
                                  /    |    \
                            Detector  Pose  Analyzer
                                       |
                                    Tracker

Backend -- Python, FastAPI, OpenCV, Ultralytics YOLO, ONNX Runtime (future). Handles camera capture, model inference, biomechanical analysis, and streams annotated frames + metrics over WebSocket.

Frontend -- React, TypeScript, Vite, Tailwind CSS, Recharts. Displays the annotated video feed and a real-time dashboard with displacement/velocity charts and form feedback.

Mobile path -- The React frontend can be wrapped with Capacitor or rebuilt in React Native. The inference engine can run on-device via ONNX Runtime Mobile or be accessed over the network.

Project Structure

src/dlai/
  __main__.py        Entry point
  config.py          Pydantic configuration
  camera.py          Camera enumeration and capture
  pipeline.py        Frame processing orchestrator
  server.py          FastAPI + WebSocket server
  engine/
    detector.py      YOLO object detection (barbell, weights)
    pose.py          YOLO pose estimation (keypoints)
    analyzer.py      Hip angle, lift phase, form feedback
    tracker.py       Rep counting, displacement, velocity FSM

web/
  src/
    App.tsx          Main layout
    components/      VideoFeed, MetricsPanel, Charts, Controls
    hooks/           useWebSocket
    types/           Shared TypeScript types

models/              YOLO model weights (gitignored)
tests/               Unit tests

Prerequisites

• Python >= 3.11
• Node.js >= 20
• uv (Python package manager)
• pnpm (Node package manager)

Setup

# Install Python dependencies
uv sync

# Install frontend dependencies
cd web && pnpm install && cd ..

# Place your trained barbell detection model at models/best.pt
# The pose model (yolo11n-pose.pt) downloads automatically on first run.

Development

Start the backend and frontend in separate terminals:

# Terminal 1: Python server (port 8000)
uv run python -m dlai

# Terminal 2: React dev server (port 3000, proxies to backend)
cd web && pnpm dev

Open http://localhost:3000 in a browser.

Production

cd web && pnpm build && cd ..
uv run python -m dlai

The server serves the built frontend from web/dist/.

Configuration

All configuration is in src/dlai/config.py via Pydantic models. Key settings:

Setting	Default	Description
camera.index	0	Camera device index
camera.width	1280	Capture width
camera.height	720	Capture height
model.detector_path	models/best.pt	Barbell detection model
model.pose_path	yolo11n-pose.pt	Pose estimation model
analysis.lift_threshold_px	40	Minimum rise to start a rep
analysis.lockout_angle	175.0	Hip angle for lockout phase

How It Works

1. Camera captures frames at configured resolution/FPS.
2. Each frame is processed by two YOLO models in sequence: barbell detection and pose estimation.
3. The analyzer computes the shoulder-hip-knee angle and classifies the lift phase (idle, setup, ascent, lockout).
4. The tracker maintains a finite state machine for rep counting, displacement history, and velocity.
5. The pipeline annotates the frame with skeleton overlay, bounding boxes, and a HUD.
6. Annotated frames (JPEG) and metrics (JSON) are streamed to the client over a single WebSocket connection.

Training a Custom Barbell Detector

See the original notebook workflow: extract frames from deadlift videos, annotate with Labellerr (or any COCO-format tool), convert to YOLO format, and train with ultralytics. Place the resulting best.pt in models/.

License

MIT