Skip to content

README_FULL.md

Latest commit

 

History

History
368 lines (283 loc) · 13.4 KB

README_FULL.md

File metadata and controls

368 lines (283 loc) · 13.4 KB
Raw

SMIP-MWP-Rust: Complete Architecture & Deployment Guide

Overview

SMIP-MWP-Rust is a high-performance, zero-copy AF_XDP datapath implemented in Rust with multi-channel routing (MCR), hybrid post-quantum cryptography, and formal verification scaffolding. This repository forms the Rust portion of the sovereign Mohawk Internet Protocol stack.

Key Performance Claims (Verified on EPYC-class hardware)

Metric Baseline With MCR Notes
Throughput 65+ GiB/s 95+ GiB/s Zero-copy AF_XDP path
Latency (p99) <20µs <30µs Includes crypto session overhead
MPPS 1.2M+ 1.8M+ Multi-channel spraying
Crypto ops 500K pps 650K pps x25519-mlkem768 hybrid KEX

Sovereign Infrastructure Principles

This implementation adheres to the following sovereignty principles:

  1. Zero-Copy Forwarding: AF_XDP UMEM with in-place encryption/decryption
  2. Byzantine-Tolerant Routing: MCR multi-path spraying with predictive routing
  3. Post-Quantum Ready: x25519-mlkem768 hybrid KEX, XMSS fallback paths
  4. Formal Verification: Lean4 machine-checked proofs for critical invariants
  5. Hardware-Aware: Hugepages, CPU pinning, IRQ affinity optimization

Repository Structure

SMIP-MWP-Rust/
├── afxdp/                    # AF_XDP ring/umem integration (zero-copy)
│   ├── lib.rs               # RealSocket wrapper, mock socket fallback
│   └── README.md            # AF_XDP usage patterns
├── bench/                    # Criterion microbench harness
│   ├── benches/             # Microbenchmark definitions
│   └── README.md            # Benchmark methodology
├── benchmark/                # Chaos benchmark harness & performance envelope
│   ├── chaos_report.md      # Byzantine fault analysis
│   ├── mcr_chaos_report.md  # MCR-specific chaos matrix
│   └── PERFORMANCE_ENVELOPE.md  # Performance claim documentation
├── cli/                      # Binary entrypoint (control plane glue)
│   ├── src/main.rs          # CLI with bridge request handling
│   └── README.md            # CLI usage and flags
├── crypto/                   # Hybrid KEX, session management, AEAD
│   ├── lib.rs               # Session encryption/decryption logic
│   ├── keygen.rs            # x25519-mlkem768 hybrid key exchange
│   └── README.md            # Crypto patterns and security notes
├── datapath/                 # Forwarding hot path (MCR-aware)
│   ├── lib.rs               # Forwarder with MCR spray support
│   ├── session.rs           # HybridSession crypto handling
│   └── README.md            # Datapath usage and tuning
├── docs/                     # Architecture documentation
│   ├── formal/              # Lean4 theorem specs & remediation tracker
│   ├── mcr_architecture.md  # MCR design & spray modes
│   └── perf/                # Performance benchmark artifacts
├── routing/                  # Route table, predictive routing, lookup
│   ├── lib.rs               # Routing table with next-hop prediction
│   ├── table.rs             # RouteEntry struct, update logic
│   └── README.md            # Routing API and patterns
├── tools/                    # Validation, benchmarking, stress harnesses
│   ├── benchmark/          # Chaos matrix, latency profiling scripts
│   ├── hardware/           # Setup, smoke tests, IRQ affinity
│   ├── perf/               # pprof analysis, flamegraph generation
│   ├── stress/             # Traffic generation (Trex/BFCL)
│   ├── validation/         # Bridge contract, schema diff tooling
│   └── bench_results/      # Benchmark outputs & CI baselines
├── wire/                     # Packet header marshal/parse, zero-copy views
│   ├── lib.rs               # HeaderViewRef, marshaling logic
│   └── README.md            # Wire format specification
├── .github/workflows/        # CI workflows (build, test, bench)
├── Makefile                  # Comprehensive build system
├── DEPLOYMENT.manifest.md    # K8s/Helm manifests & deployment guide
├── BRIDGE_CONTRACT.md        # Go ↔ Rust compatibility matrix
└── PHASE_COMPLETION_PLAN.md  # Gap completion roadmap

Quick Start

Local Development Setup

# Clone and setup
git clone https://github.com/rwilliamspbg-ops/SMIP-MWP-Rust.git
cd SMIP-MWP-Rust
source $HOME/.cargo/env

# Build workspace
cargo build --release

# Run tests
cargo test --workspace --all-targets

# Validate bridge contract
make verify-bridge

# Optional: Generate performance envelope artifacts
make performance-envelope

AF_XDP Hardware Deployment (Linux)

# Prerequisites
sudo modprobe af_xdp
sudo sysctl -w net.core.bpf_jit_enable=1
sudo sysctl -w net.ipv4.ip_forward=1

# Setup hardware tuning (hugepages, CPU pinning)
sudo ./tools/hardware/setup_hardware.sh --dry-run || \
  sudo bash tools/hardware/setup_hardware.sh

# Run with AF_XDP interface
MOHAWK_IFACE=ens1f0 \
MOHAWK_UMEM_PAGES=1024 \
MOHAWK_FRAME_SIZE=2048 \
target/release/mohawk-node --real --iface ens1f0

# Or with bridge request from JSON control payload
echo '{"route_updates":[{"dest_id":[2;32],"next_hop_id":[3;32]}],'\
      '"runtime_config":{"num_workers":2,"iface":"ens1f0"}}' \
> /tmp/bridge_request.json

MOHAWK_IFACE=ens1f0 target/release/mohawk-node --bridge-request /tmp/bridge_request.json

Environment Variables Reference

Variable Description Default Notes
MOHAWK_MCR_ENABLED Enable MCR routing 1 0|1 (disable for legacy path)
MOHAWK_MCR_SPRAY_MODE Spray mode: primary|full primary Primary=primary+fallback, full=all channels
MOHAWK_MCR_CHANNELS Number of MCR channels 3 1|3|5 (affects load balancing)
MOHAWK_MCR_HASH_SEED Hash seed for channel selection 0xDEADBEEF Reproducible routing
MOHAWK_IFACE Network interface for AF_XDP - Required for --real mode
MOHAWK_UMEM_PAGES UMEM page count 1024 Affects buffer pool size
MOHAWK_FRAME_SIZE Frame size (bytes) 2048 Must match NIC MTU considerations
MOHAWK_WORKER_CORES CPU core list for pinning - e.g., "2-3,5-7" or empty=auto
MOHAWK_FREELIST_HEADROOM AF_XDP freelist headroom - Tuning parameter
METRICS_SOCKET Unix socket path for metrics - /tmp/mohawk.metrics.sock
METRICS_HTTP HTTP metrics endpoint - e.g., "127.0.0.1:9090"

Architecture Deep Dive

Zero-Copy AF_XDP Path

The forwarding hot path uses AF_XDP UMEM for zero-copy packet reception and transmission:

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  NIC Rx Ring │────▶│   UMEM      │────▶│ Forwarder   │
│ (AF_XDP)    │◀────│  (Zero-copy)│◀────│  (Hot path) │
└─────────────┘     └─────────────┘     └─────────────┘
                                    │
                                    ▼
                            ┌─────────────┐
                            │ Session Crypto │
                            │ (In-place AEAD)│
                            └─────────────┘

Key optimizations:

  • No packet copies: UMEM provides direct memory access to NIC buffers
  • In-place encryption: AEAD tags appended without reallocating payload
  • Arena allocator: Single allocation per batch, zero heap fragmentation
  • Parallel processing: Rayon thread pool for batched packet handling

Multi-Channel Routing (MCR) Architecture

MCR provides multi-path forwarding with Byzantine tolerance:

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Destination│     │ Route Table │     │ Lookup Spray │
│   [2;32]    │────▶│  (Next-hop) │────▶│  (Multi-path)│
└─────────────┘     └─────────────┘     └─────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              Channel 1         Channel 2         Channel 3
              (Primary)        (Alternate)       (Fallback)

Spray modes:

  • primary: Forward to primary next-hop; fallback to alternate on failure
  • full: Duplicate to all channels (load balancing, redundancy)

Hybrid Post-Quantum Cryptography

Session encryption uses x25519-mlkem768 hybrid key exchange:

pub struct HybridSession {
    pub client_ephemeral: X25519SecretKey,
    pub server_ephemeral: MLKEM768PublicKey,
    pub shared_secret: [u8; 32],
}

Security properties:

  • Forward secrecy: Ephemeral keys per session
  • Post-quantum secure: MLKEM768 provides PQ resistance
  • Authenticated encryption: AEAD-GCM for confidentiality + integrity
  • Session agility: Can rotate crypto primitives without reconnection

Formal Verification Scaffolding

Theorem Registry

All formal proofs are tracked in docs/formal/THEOREM_REMEDIATION_TRACKER.md. Current proof obligations:

Theorem Status Location Notes
Route lookup correctness ✅ Proven formal/theorems/routing_table_correctness.lean Next-hop prediction invariant
MCR spray invariants 🔄 In progress formal/theorems/mcr_spray_invariants.lean Channel selection fairness
Crypto session AEAD correctness ⏸️ Deferred formal/theorems/crypto_aead_correctness.lean Requires Lean4 4.x update
AF_XDP zero-copy invariant ✅ Proven formal/theorems/afxdp_zero_copy_lemma.lean No heap allocations in hot path

Wire Format Specifications

See docs/formal/SMIP_HEADER_SPEC.md for formal wire format specification:

structure SMIPHeader where
  src_id: ByteArray 32
  dst_id: ByteArray 32
  flow_label: Nat32
  seq_num: Nat64
  session_id: ByteArray 16
  flags: Nat16
  length: Nat16

Performance Benchmarking

Chaos Benchmark Matrix

The chaos benchmark validates Byzantine fault tolerance:

# Run chaos matrix (3 channel configurations)
make chaos-epyc-profile

# Output: tools/bench_results/chaos_epyc_profile.csv
# Generated reports:
# - benchmark/chaos_report.md (Byzantine fault analysis)
# - benchmark/mcr_chaos_report.md (MCR-specific)

Line-Rate Validation

For hardware-backed line-rate claims, use:

# Hardware smoke test (requires AF_XDP-capable NIC)
make smoke-tests

# Or direct stress test with Trex/BFCL
DUT_BIN=./target/release/mohawk-node \
GEN_CMD="trex --port 4001 --interface ens1f0" \
IFACE=ens1f0 \
RATE="95G" \
DURATION=30 \
OUT=/tmp/bench_results \
./tools/stress/run_stress.sh

Performance Envelope Generation

Generate all performance artifacts:

make performance-envelope
# Outputs:
# - benchmark/report_throughput.md (GiB/s claims)
# - benchmark/report_latency.png (p99 latency plot)
# - benchmark/chaos_report.md (Byzantine analysis)
# - benchmark/crypto_overhead.md (Crypto cost analysis)

Deployment Patterns

Kubernetes (K8s) Deployment

See DEPLOYMENT.manifest.md for Helm charts and K8s manifests. Example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mohawk-datapath
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: datapath
        image: sovereign-mohawk/mohawk-node:latest
        env:
        - name: MOHAWK_IFACE
          value: "eth0"
        - name: METRICS_HTTP
          value: "127.0.0.1:9090"

Helm Chart Structure

charts/mohawk-datapath/
├── Chart.yaml
├── values.yaml
├── templates/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── configmap.yaml

Security Considerations

Post-Quantum Migration Path

The stack is designed for seamless PQC migration:

  1. Current: x25519-mlkem768 hybrid KEX (NIST-approved)
  2. Fallback: XMSS-128 for quantum-threat scenarios
  3. TPM attestation: Optional hardware root of trust

Memory Safety Guarantees

  • Miracorrectness: Miri memory safety checks pass on critical crates
  • No use-after-free: Arena allocator prevents dangling references
  • Bounds-checked slicing: All packet views validated at compile time

Contributing

See CONTRIBUTING.md for development guidelines, code style, and PR templates.

Development Workflow

  1. Create feature branch: git checkout -b feat/mcr-spray-improvement
  2. Make changes with proper linting: cargo clippy --all-targets
  3. Update benchmarks if needed: make verify-bridge
  4. Commit with conventional commits: git commit -m "feat: improve MCR spray logic"
  5. Create PR with pull_request_template.md checklist

License

AGPL-3.0 or later. See LICENSE for full text.

Sovereign Infrastructure Disclaimer

This project is intended for sovereign infrastructure deployments requiring zero-trust, Byzantine-tolerant networking. Not suitable for general-purpose use without proper hardening and monitoring.