Offroad AI Segmentation Platform

An end-to-end semantic segmentation pipeline and real-time visualization platform for desert/offroad environments, built for the BigRock Exchange Hackathon.

Features

• Accurate Semantic Segmentation: Maps 11 distinct classes including Trees, Bushes, Ground Clutter, Flowers, Rocks, and Sky.
• Dual Architecture Implementation:
  • LRASPP MobileNetV3: Extremely fast, lightweight model optimized for real-time inference (target < 50ms latency).
  • DINOv2 + ConvNeXt Head: Advanced vision transformer architecture for maximum segmentation accuracy.
• Robust Training Pipeline: End-to-end training loops with data augmentation, learning rate scheduling, class balancing, and performance metric tracking.
• Interactive UI: Next-generation React frontend allowing users to drag-and-drop images to see real-time AI segmentation masks and confidence heatmaps.
• FastAPI Backend: Edge-preserving post-processing pipeline ensuring sharp object boundaries.

Dataset Structure

The platform expects the FalconEditor synthetic dataset:

• Offroad_Segmentation_Training_Dataset/train: Used for model training.
• Offroad_Segmentation_Training_Dataset/val: Used for validation tracking during training.
• Offroad_Segmentation_testImages: Used for benchmark testing on completely unseen environments.

Quick Start & Setup

1. Environment Setup

We recommend using Conda:

cd Offroad_Segmentation_Scripts/ENV_SETUP
setup_env.bat
conda activate EDU

2. Training the Models

Both training scripts use all available data and are heavily optimized:

cd Offroad_Segmentation_Scripts

# For real-time inference (MobileNetV3):
python train_lraspp.py

# For maximum accuracy (DINOv2):
python train_segmentation.py

Training artifacts and metrics are saved to train_stats/.

3. Running Validation & Inference

To evaluate the DINOv2 model on the test dataset:

python test_segmentation.py

Outputs are saved to predictions/ containing raw masks, colored masks, and side-by-side comparison graphics.

4. Launching the Web Platform

Start the AI Backend:

cd backend
python app.py

Start the React Frontend:

cd webapp
npm install
npm run dev

Navigate to http://localhost:5173 to experience the Interactive Demo.