MediaPipe Webcam Object Detection App

A production-ready React + TypeScript web application that uses Google MediaPipe for real-time object detection via webcam. Built with Vite and Bun for optimal performance.

Features

• 🎥 Real-time Webcam Detection: Uses native getUserMedia API for webcam access
• 🤖 AI-Powered Object Detection: Leverages MediaPipe's efficient object detection models
• ⚡ On-Device Processing: All AI processing happens locally in the browser
• 🎨 Modern UI: Clean, responsive interface with Tailwind CSS
• 📱 Mobile Friendly: Responsive design that works on all devices
• 🔧 TypeScript: Full type safety and better development experience

Tech Stack

• Frontend: React 19 + TypeScript
• Build Tool: Vite
• Runtime: Bun
• AI Framework: MediaPipe Tasks Vision
• Styling: Tailwind CSS
• Webcam: Native getUserMedia API

Getting Started

Prerequisites

• Bun installed on your system
• Modern web browser with webcam support
• HTTPS connection (required for webcam access in production)

Installation

1. Clone the repository:

git clone <your-repo-url>
cd machine-learning

2. Install dependencies:

bun install

3. Start the development server:

bun run dev

4. Open your browser and navigate to http://localhost:5173

Usage

1. Load the Model: The app automatically loads the MediaPipe object detection model
2. Start Webcam: Click the "Search with camera" button to activate your webcam
3. Detect Objects: Point your camera at objects to see real-time detection results
4. View Results: See detected objects with confidence scores displayed below the video

Project Structure

src/
├── components/
│   ├── ObjectDetectionApp.tsx    # Main app component
│   ├── VideoCanvasDisplay.tsx    # Video and canvas display
│   └── DetectionResults.tsx      # Detection results display
├── hooks/
│   └── useMediaPipeObjectDetection.ts  # Custom hook for MediaPipe logic
├── utils/
│   ├── drawDetections.ts         # Canvas drawing utilities
│   ├── initializeObjectDetector.ts # MediaPipe initialization
│   ├── detectObjectsAndDraw.ts   # Detection and drawing logic
│   └── cleanupMediaPipeResources.ts # Resource cleanup
└── App.tsx                       # Root component

How It Works

1. Model Initialization: The app loads MediaPipe's object detection model from Google's CDN
2. Webcam Access: Uses the getUserMedia API to access the user's webcam
3. Real-time Processing: Continuously processes video frames for object detection
4. Canvas Overlay: Draws detection boxes and labels on an HTML canvas overlaid on the video
5. Results Display: Shows detected objects with confidence scores in a clean list format

MediaPipe Model

The app uses MediaPipe's EfficientDet Lite0 model, which provides:

• Fast inference speed suitable for real-time applications
• Good accuracy for common object detection tasks
• Small model size for quick loading
• Support for 90+ object categories

Browser Compatibility

• Chrome 88+
• Firefox 85+
• Safari 14+
• Edge 88+

Performance Considerations

• Model Loading: Initial model download may take a few seconds
• Frame Rate: Detection runs at browser's refresh rate (typically 60fps)
• Memory Usage: Efficient memory management with proper cleanup
• CPU Usage: Optimized for modern devices with hardware acceleration

Customization

Changing the Model

To use a different MediaPipe model, update the MODEL_PATH in src/App.tsx:

const MODEL_PATH =
  "https://storage.googleapis.com/mediapipe-models/your-model-path";

Adjusting Detection Parameters

Modify the detection parameters in src/hooks/useMediaPipeObjectDetection.ts:

objectDetectorRef.current = await initializeObjectDetector(
  modelPath,
  0.5, // scoreThreshold - higher = more confident detections
  5 // maxResults - maximum number of objects to detect
);

Styling

The app uses Tailwind CSS for styling. Customize the appearance by modifying the component files or extending the Tailwind configuration.

Troubleshooting

Webcam Not Working

• Ensure you're using HTTPS (required for webcam access)
• Check browser permissions for camera access
• Try refreshing the page and granting permissions again

Model Loading Issues

• Check your internet connection
• Verify the model URL is accessible
• Check browser console for error messages

Performance Issues

• Close other browser tabs to free up resources
• Ensure hardware acceleration is enabled
• Try reducing the video resolution in the webcam constraints

Development

Available Scripts

• bun run dev - Start development server
• bun run build - Build for production
• bun run serve - Preview production build
• bun run lint - Run ESLint

Adding New Features

1. Create new components in the components/ directory
2. Add custom hooks in the hooks/ directory
3. Implement utility functions in the utils/ directory
4. Update types and interfaces as needed

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Google MediaPipe for the AI framework
• LogRocket Blog for the implementation guide
• Vite for the build tool
• Bun for the runtime environment