Skip to content

RideneFiras/QosMLOPS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

42 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Throughput Prediction in 5G Networks

Demo Screenshot

FastAPI Dockerized MLflow PostgreSQL Made with Python


This repository includes all core functionalities along with a beta dashboard for testing and development purposes. For a more lightweight and production-ready frontend, check out the enhanced version here:

πŸ“‘ Project Overview

This project predicts network throughput β€” a key indicator of Quality of Service (QoS) in 5G networks β€” using machine learning. It features a full MLOps pipeline with MLflow tracking, Elasticsearch-based evaluation dashboards, SHAP explainability, and GPT-powered QoS insights. The system runs on a fully Dockerized stack using FastAPI and a simple web frontend for real-time use.


This work was developed as part of the Integrated Project at Esprit School of Engineering, under the guidance of professors Rahma Bouraoui, Safa Cherif, and Zaineb Labidi.


✨ Features

Frontend

  • Upload CSV to get QoS predictions
  • GPT-powered explanations with SHAP insights
  • Caching avoids repeated GPT API calls
  • Export results as PDF

Backend

  • FastAPI for predictions, explainability, and GPT summaries
  • Uses SHAP values to explain model decisions
  • GPT-4o via GitHub API creates human-readable reports
  • PostgreSQL stores predictions & insights

MLOps Pipeline

  • Data preprocessing, model training, evaluation
  • MLflow logs parameters, metrics, and artifacts
  • Elasticsearch logs evaluation metrics for monitoring dashboards
  • Simple CLI via Makefile commands

πŸ–ΌοΈ Kibana Evaluation Dashboard:

Evaluation Dashboard


πŸ—‚οΈ Project Structure

β”œβ”€β”€ static/                 # Frontend HTML + JS
β”œβ”€β”€ services/               # SHAP explainability + GPT interpretation logic
β”œβ”€β”€ Dataset/                # Training and test data (CSV)
β”œβ”€β”€ Models/                 # Saved models and processed data
β”œβ”€β”€ assets/                 # Visuals (e.g. Elastic dashboard screenshots)
β”œβ”€β”€ testinput/              # Example input for testing predictions
β”œβ”€β”€ app.py                  # FastAPI server
β”œβ”€β”€ main.py                 # MLOps training & evaluation pipeline
β”œβ”€β”€ model_pipeline.py       # Sklearn training, preprocessing, evaluation
β”œβ”€β”€ csvv.py                 # CSV parsing logic
β”œβ”€β”€ docker-compose.yml      # Orchestrates app + DB + Elasticsearch + MLflow
β”œβ”€β”€ Dockerfile              # Container for FastAPI + logic
β”œβ”€β”€ Makefile                # CLI commands for training, evaluation, etc.
β”œβ”€β”€ requirements.txt        # Python dependencies
β”œβ”€β”€ .env.example            # Example environment variable file
β”œβ”€β”€ .flake8 / .pre-commit/  # Code quality configuration
β”œβ”€β”€ README.md               # Project overview and instructions

βš™οΈ Setup

# Clone repo
git clone https://github.com/RideneFiras/QosMLOPS.git
cd QosMLOPS

# Create and activate a virtual environment (recommended)
python -m venv qos
source qos/bin/activate

# Install Python dependencies
pip install -r requirements.txt

# Train model and prepare everything
make all

# Start FastAPI and web UI
make fastapi

# Or launch entire stack with Docker
docker-compose up --build

🧠 GPT Insight System

After the model predicts throughput:

  • SHAP explains feature impact per prediction
  • GPT-4o turns SHAP values into a readable QoS summary
  • Insights include:
    • Overall QoS rating (e.g. Very Low β†’ Very Good)
    • Top influencing features
    • Suggestions to improve performance
  • Markdown is rendered cleanly in-browser
  • Export as PDF with a single click

πŸ§ͺ Usage Guide

  • Visit http://127.0.0.1:8000
  • Upload your test CSV
  • Click β€œExplain” next to any result
  • Read GPT-powered QoS explanation
  • Save report as PDF if needed

πŸ”‘ Environment Variables

Store these in a .env file (not tracked by git):

  • DATABASE_URL β€” PostgreSQL connection string
  • GITHUB_TOKEN β€” GitHub token for GPT API access
  • IS_DOCKER β€” Set to true inside containers (used for routing)

πŸ”§ Makefile Commands

πŸ”§ Makefile Commands

make all          # Run full pipeline: prepare β†’ train β†’ evaluate β†’ predict
make train        # Train the ML model and log to MLflow
make evaluate     # Evaluate performance and send metrics to MLflow & Elastic
make fastapi      # Start FastAPI locally (with database in background)
make services-up  # Start Elasticsearch, Kibana, PostgreSQL, and MLflow (Docker)
make docker-run   # Run the entire app in Docker (FastAPI + dependencies)
make docker-down  # Stop all running Docker containers
make notebook     # Launch Jupyter Notebook
make check        # Run both code formatting (Black) and linting (Flake8)
make clean        # Remove cache, .pkl files, and logs

βš™οΈ Tech Stack

  • Python & Scikit-learn
  • FastAPI + SHAP + GPT-4o
  • MLflow for model tracking
  • Elasticsearch + Kibana for evaluation dashboards
  • PostgreSQL for storing prediction results
  • Tailwind CSS for styling
  • Docker for deployment

βœ… Notes

This is an end-to-end, real-world MLOps project combining AI explainability, model monitoring, and GPT summarization. Designed with clarity, reliability, and user insights in mind β€” ready for production.