Distributed API Rate Limiter & Traffic Scheduler

A high-performance, production-grade API rate-limiting engine built in C++17 using token bucket and sliding window algorithms for concurrent traffic control. Exposes configurable per-client throttle policies through a RESTful HTTP API built on raw POSIX sockets.

---

Features

Feature	Details
Token Bucket	Burst-tolerant rate limiting with configurable capacity & refill rate
Sliding Window Log	Strict per-window limit with O(k) amortised eviction
Per-client Policies	unordered_map-backed registry with shared_mutex read/write locking
Thread Pool	Fixed-size pool with condition_variable-based work signaling
Traffic Scheduler	Back-pressure queue with configurable capacity limit
REST API	9 HTTP endpoints over raw POSIX BSD sockets
Benchmarking	Multi-threaded traffic simulator with throughput, latency (avg/p99)
Zero Dependencies	Pure C++17 + POSIX — no Boost, no external HTTP libs

---

Project Structure

cpp_project/
├── CMakeLists.txt
├── README.md
├── include/
│   ├── rate_limiter/
│   │   ├── token_bucket.hpp       # Token Bucket algorithm
│   │   ├── sliding_window.hpp     # Sliding Window Log algorithm
│   │   ├── rate_limiter.hpp       # Unified interface
│   │   └── policy_manager.hpp     # Per-client policy registry
│   ├── http/
│   │   ├── http_server.hpp        # POSIX socket HTTP server
│   │   ├── http_request.hpp       # HTTP/1.1 request parser
│   │   └── http_response.hpp      # HTTP response builder
│   ├── scheduler/
│   │   ├── worker_pool.hpp        # Fixed thread pool
│   │   └── traffic_scheduler.hpp  # Back-pressure dispatcher
│   └── benchmark/
│       └── benchmark.hpp          # Traffic simulator
├── src/                           # Implementations
└── tests/                         # Unit + integration tests

---

Build

Requirements: CMake ≥ 3.16, a C++17-capable compiler (GCC 9+ / Clang 10+)

# Configure (Release mode)
cmake -B build -DCMAKE_BUILD_TYPE=Release

# Build everything
cmake --build build --parallel

# Run all tests
cd build && ctest --output-on-failure

---

Run the Server

./build/rate_limiter_server --port 8080 --workers 8

On startup, two demo clients are pre-registered:

• demo_client — Token Bucket, 10 tokens burst, 2 tokens/sec refill
• demo_sliding — Sliding Window, 5 req/sec

---

REST API Reference

POST /policy — Create / Update Policy

# Token Bucket policy
curl -s -X POST http://localhost:8080/policy \
     -H "Content-Type: application/json" \
     -d '{
           "client_id":   "user_42",
           "algorithm":   "token_bucket",
           "max_tokens":  20,
           "refill_rate": 5,
           "description": "Premium user"
         }'

# Sliding Window policy
curl -s -X POST http://localhost:8080/policy \
     -d '{
           "client_id":    "free_user",
           "algorithm":    "sliding_window",
           "max_requests": 10,
           "window_ms":    1000
         }'

GET /policy/{client_id} — Get Policy

curl http://localhost:8080/policy/user_42

DELETE /policy/{client_id} — Remove Policy

curl -X DELETE http://localhost:8080/policy/user_42

GET /policies — List All Clients

curl http://localhost:8080/policies

POST /check/{client_id} — Check Rate Limit

Returns 200 OK if allowed, 429 Too Many Requests if limited.

curl -s -X POST http://localhost:8080/check/demo_client
# {"allowed":true,"client_id":"demo_client","algorithm":"token_bucket","remaining":9,"retry_after_ms":0}

# When limited:
# HTTP 429 {"allowed":false,"retry_after_ms":450}

GET /stats — Global Statistics

curl http://localhost:8080/stats
# {"total_clients":2,"total_requests":42,"total_accepted":37,"total_denied":5,"overall_accept_rate":0.88}

GET /health — Health Check

curl http://localhost:8080/health
# {"status":"healthy","version":"1.0.0","registered_clients":2}

POST /benchmark — Traffic Simulation

curl -s -X POST http://localhost:8080/benchmark \
     -d '{
           "clients":             20,
           "requests_per_client": 100,
           "algorithm":           "token_bucket",
           "max_tokens":          50,
           "refill_rate":         10
         }'
# {
#   "total_requests": 2000,
#   "accepted": 1127,
#   "denied": 873,
#   "accept_rate": 0.5635,
#   "throughput_rps": 112340.2,
#   "duration_ms": 10.03,
#   "avg_latency_us": 4.8,
#   "p99_latency_us": 18.1
# }

---

Data Structures & Algorithms

Component	DSA Used	Complexity
Token Bucket	double counter + steady_clock	O(1)
Sliding Window	std::deque<time_point>	O(k) amortised evict
Policy Registry	std::unordered_map	O(1) avg lookup
Worker Pool	std::queue<Task> + condition_variable	O(1) enqueue
Traffic Scheduler	Bounded queue + atomic counters	O(1)
Benchmark Latency	std::vector + std::sort	O(n log n)

---

Architecture

HTTP Client
     │  TCP Connection
     ▼
┌──────────────┐
│  HttpServer  │  ← POSIX sockets, accept loop
│  (main thr.) │
└──────┬───────┘
       │  submit(Task)
       ▼
┌──────────────────┐
│ TrafficScheduler │  ← back-pressure queue (max 4096)
│  (WorkerPool)    │  ← N worker threads
└──────┬───────────┘
       │  route(request)
       ▼
┌──────────────────┐
│  PolicyManager   │  ← unordered_map + shared_mutex
│  (rate limiters) │
└──────┬───────────┘
       │
  ┌────┴────┐
  ▼         ▼
Token    Sliding
Bucket   Window

---

Running Tests Manually

./build/test_token_bucket
./build/test_sliding_window
./build/test_scheduler

---

Quick Demo Script

# Start server in background
./build/rate_limiter_server &
SERVER_PID=$!
sleep 0.5

# Create policy (5 req burst, 2/sec refill)
curl -s -X POST http://localhost:8080/policy \
     -d '{"client_id":"demo","algorithm":"token_bucket","max_tokens":5,"refill_rate":2}'

# Fire 7 requests — first 5 allowed, rest denied
for i in $(seq 1 7); do
  echo -n "Request $i: "
  curl -s -o /dev/null -w "%{http_code}\n" -X POST http://localhost:8080/check/demo
done

# Run benchmark
curl -s -X POST http://localhost:8080/benchmark \
     -d '{"clients":5,"requests_per_client":50}' | python3 -m json.tool

kill $SERVER_PID