pixlet

pixlet is a gesture-controlled webcam photobooth built with Python, OpenCV, and MediaPipe.

Use your hands to select visual filters, pose for the camera, and take photos without touching your computer. Pixlet performs real-time hand and face tracking to turn simple gestures into camera controls and applies effects directly to the live video stream.

Features

• Real-time hand tracking using MediaPipe
• Real-time face tracking using MediaPipe Face Mesh
• Gesture-controlled filter selection
• Gesture-controlled photo capture
• 7 visual filters
• Photo preview after capture
• Automatic local photo storage
• Live webcam preview

Filters

Pixlet currently supports seven filters (using ASL numbering):

Gesture	Filter
One	Black & White
Two	Pikachu
Three	Pixelated
Four	Chiikawa
Five	Pink
Six	K-Pop Idol
Seven	Fruit

The filters include both traditional image processing and dynamically positioned image overlays. Face-based filters use facial landmarks to determine where and how large an overlay should be rendered.

Filter Effects

1. Black & White

• Converts the camera frame to grayscale.

2. Pikachu

• Detects the user's face.
• Uses forehead landmarks to determine the position and scale of Pikachu ears.
• Adds a Pikachu image overlay to the frame.

3. Pixelated

• Downscales the camera frame to 128 × 64.
• Upscales it back to the original resolution using nearest-neighbor interpolation to create a pixel-art effect.

4. Chiikawa

• Detects the user's forehead.
• Randomly selects one of several Chiikawa characters when the filter changes.
• Dynamically scales and positions the character based on the user's face.

5. Pink

• Adjusts the red channel of the image to create a pink-tinted effect.

K-Pop

• Randomly selects from several K-Pop artist overlays.

Fruit

• Detects the user's face and places a randomly selected fruit/food overlay around the face.

Gesture Controls

Pixlet uses MediaPipe hand landmarks to interpret finger configurations as commands.

Left Hand — Select Filter

The left hand controls the active filter. Finger positions are analyzed to identify different gestures, which map to filter IDs.

Right Hand — Take Photo

The right hand controls photo capture.

To take a photo:

1. Open your right hand.
2. Make a pinch gesture with your thumb and index finger.
3. Hold the gesture briefly.
4. Pixlet starts a 3-second countdown.
5. The processed frame is captured and saved.

How It Works

Pixlet processes every frame through a real-time computer vision pipeline:

Webcam
   │
   ▼
OpenCV Frame Capture
   │
   ├───────────────┐
   ▼               ▼
Hand Tracking    Face Tracking
(MediaPipe)      (MediaPipe)
   │               │
   ▼               ▼
Gesture          Facial
Recognition      Landmarks
   │               │
   └───────┬───────┘
           ▼
      Filter Selection
           │
           ▼
      Filter Pipeline
           │
           ▼
      Live Preview
           │
           ▼
      Gesture Capture
           │
           ▼
     Countdown + Flash
           │
           ▼
       Saved Photo

The main loop continuously captures frames, mirrors the webcam feed, runs hand and face tracking, determines the current filter, applies the filter, and displays the resulting frame.

Tech Stack

• Python
• OpenCV — webcam capture, image processing, rendering, and photo storage
• MediaPipe Hands — real-time hand landmark detection and gesture recognition
• MediaPipe Face Mesh — facial landmark detection for positioning visual effects
• NumPy — image and pixel manipulation

Project Structure

pixlet/
├── main.py              # Main camera loop and application logic
├── Camera.py            # Camera state, countdown, capture, flash, and preview
├── Hand_Tracker.py      # MediaPipe hand tracking
├── Face_Tracker.py      # MediaPipe face tracking
├── Gestures.py          # Hand gesture recognition
├── Filters.py           # Image filters and visual overlays
│
├── filters/
│   ├── pikachu/
│   ├── chiikawa/
│   ├── kpop/
│   └── fruits/
│
├── unedited_filters/    # Original filter assets
└── pixs/                # Captured photos

Getting Started

Requirements

• Python 3
• A webcam
• OpenCV
• MediaPipe
• NumPy

Install the Python dependencies:

pip install opencv-python mediapipe numpy

Run

Clone the repository and run:

python main.py

Pixlet will open the webcam and display the live filtered preview.

Press Esc to exit.

Photo Capture

Captured photos are automatically saved to the pixs/ directory.

Each photo is saved using a timestamp-based filename:

pixs/pixlet_<timestamp>.png

After taking a photo, Pixlet displays a temporary preview thumbnail in the bottom-right corner of the camera feed and produces a short flash effect.

Architecture

Pixlet separates the camera application into several components:

main.py

Coordinates the entire application:

• Captures webcam frames
• Runs hand and face tracking
• Interprets gestures
• Selects filters
• Applies visual effects
• Handles photo capture
• Displays the final frame

Hand_Tracker.py

Wraps MediaPipe Hands and exposes functionality for:

• Detecting up to two hands
• Tracking hand landmarks
• Identifying handedness
• Drawing hand landmarks for debugging

Face_Tracker.py

Uses MediaPipe Face Mesh to track a single face with refined landmarks, providing the coordinates required for face-aware filters.

Gestures.py

Converts MediaPipe hand landmarks into higher-level gestures such as:

• Finger counting
• Pinching
• Thumb positions
• Filter selection gestures

Filters.py

Contains the image-processing pipeline and filter implementations.

It supports both pixel-level transformations, such as grayscale and pixelation, and alpha-blended PNG overlays that are dynamically scaled and positioned using facial landmarks.

Camera.py

Manages camera-specific state and effects, including:

• Capture cooldowns
• Countdown timer
• Photo saving
• Flash animation
• Post-capture preview

Motivation

Pixlet was built to make taking photos feel more like using a physical photobooth: no keyboard or mouse required.

Computer vision handles the interaction layer, allowing users to control the camera entirely through natural hand gestures while face tracking makes the visual effects responsive to the user's position.

Future Improvements

Potential improvements include:

• Adding more filters and gestures
• Multiple face support
• Improved gesture robustness
• Filter previews before selection
• Photo gallery functionality