Skip to content

Adit-Jain-srm/NightmareNet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

151 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NightmareNet

Deep Learning Adversarial Network for Image Synthesis

Python PyTorch Deep Learning License Last Commit

A generative adversarial network exploring the boundaries of AI-generated imagery.


NightmareNet

The first platform that actively improves model robustness through biologically-grounded training cycles.

License CI Tests Python

Wake. Dream. Nightmare. Compress. Repeat.


The Problem

Production models silently degrade. Adversarial perturbations as small as a single token swap collapse model accuracy from 92% to 23% (Jin et al. 2020, TextFooler). Conventional adversarial training trades clean accuracy for robustness — and worse, it suffers from "robustness forgetting" (AAAI 2025, ICCV 2025), where each new training run erodes previously-acquired defenses. The EU AI Act Article 15 (fully applicable August 2, 2026) now mandates demonstrable robustness for high-risk AI systems, but no existing tool combines adversarial generation, forgetting prevention, compression, and orchestration into a single coherent workflow.

Note

NightmareNet is not a runtime guardrail (Lakera) or evaluation library (TextAttack). It is a training paradigm that produces measurably more robust models, with a hosted platform for orchestration and EU AI Act compliance reporting.


The Solution — A 4-Phase Sleep Cycle

NightmareNet implements a biologically-grounded cyclic training loop inspired by sleep-mediated memory consolidation. Each cycle decomposes robustness acquisition into four complementary phases, then compresses the result and restarts — producing models that accumulate robustness across iterations without catastrophic forgetting.

flowchart LR
    Start([Clean Model]) --> Wake[Wake<br/>Supervised<br/>Fine-Tuning]
    Wake --> Dream[Dream<br/>Generative<br/>Augmentation]
    Dream --> Nightmare[Nightmare<br/>Curriculum<br/>Adversarial]
    Nightmare --> Compress[Compress<br/>Robust<br/>Distillation]
    Compress -->|next cycle| Wake
    Compress --> Done([Hardened Model])

    classDef wake fill:#06B6D4,stroke:#0891B2,color:#020617
    classDef dream fill:#818CF8,stroke:#6366F1,color:#020617
    classDef nightmare fill:#EF4444,stroke:#DC2626,color:#FFFFFF
    classDef compress fill:#F59E0B,stroke:#D97706,color:#020617
    class Wake wake
    class Dream dream
    class Nightmare nightmare
    class Compress compress
Loading
Phase Objective Mechanism
Wake Establish clean-data competence Standard cross-entropy fine-tuning
Dream Build invariance to plausible distribution shift Synonym, paraphrase, syntactic augmentation at strength 0.2-0.3 + KL consistency
Nightmare Harden against worst-case perturbations Curriculum adversarial training, strength 0.5-0.9, character/word/sentence-level attacks
Compress Preserve robustness, shed parameters Adversarial robust distillation (RSLAD-style) + magnitude pruning

The student model becomes the next cycle's learner. After 3-5 cycles, robustness saturates and the cycle terminates.


Quick Start

pip install nightmarenet                              # core (CLI + library)
nightmarenet distort --type nightmare --strength 0.7 --text "I love this movie"
nightmarenet train --config configs/benchmark_sst2.yaml
nightmarenet evaluate --model ./output/model --strengths 0.1,0.3,0.5,0.7,0.9

Run the full Wake -> Dream -> Nightmare -> Compress cycle on SST-2 in under 10 minutes on a single GPU. Open notebooks/01_quickstart.ipynb for a Colab-ready walkthrough.

Tip

Dev hardware target is a 4 GB VRAM laptop GPU (RTX 3050 Ti). DistilBERT and DistilGPT-2 fit comfortably; GPT-2 (124M) requires gradient checkpointing + FP16.


What's Inside — 20 Panels of Capability

NightmareNet ships as a unified workspace where every concern gets its own first-class panel. This is a feature-dense, information-rich product — not a sparse landing page.

# Panel One-line Purpose
01 Command Center Live overview: active cycles, GPU pool, recent experiments, robustness trend
02 Pipeline Wizard Multi-step experiment creation (source -> model -> config -> launch)
03 Phase Visualizer Animated Wake -> Dream -> Nightmare -> Compress with real-time per-phase metrics
04 Live Training Monitor Streaming loss curves, robustness deltas, GPU/throughput telemetry
05 Experiment History Sortable, filterable, paginated table of every run with diffable configs
06 Robustness Radar Multi-axis radar chart (clean acc, TextFooler, BertAttack, PWWS, TextBugger)
07 Model Comparison Side-by-side before/after; A/B between checkpoints with overlay charts
08 Distortion Preview Paste text -> see dream and nightmare side-by-side with token-level diff
09 Benchmark Suite One-click run of SST-2, AG News, IMDB benchmarks vs published baselines
10 Compliance Dashboard EU AI Act Article 15 progress, NIST AI RMF mapping, signed evidence packs
11 Audit Trail Every state mutation (user, timestamp, diff), exportable CSV/JSON
12 API Playground Interactive endpoint explorer for /distort, /evaluate, /pipeline
13 CI/CD Integration Copy-paste GitHub Action snippet, status badges, threshold gates per repo
14 Model Registry Trained artifacts with SHA-256 checksums, lineage links, download endpoints
15 Export Center PDF compliance reports, JSON metric dumps, CSV per-strength sweeps
16 Trend Analysis Robustness improvement over cycles; converge curves; ablation comparisons
17 Self-Health Monitor API health, GPU saturation, queue depth, worker liveness, latency p95/p99
18 AI Assistant Context-aware copilot answering questions about the current experiment
19 Settings API keys, model defaults, distortion strengths, CORS, rate limits
20 Team Management RBAC (admin/member/viewer), org switcher, seat allocation, SSO (enterprise)

Benchmark Results (v1 — SST-2)

Measured on RTX 3050 Ti (4 GB VRAM), DistilBERT-base-uncased, 500 train / 200 eval samples, seed 42. Full methodology: docs/research/benchmark-v1.md.

Method Clean Acc Avg Robustness (dream+nightmare, 0.1-0.9) Relative Improvement Params
Wake-only baseline 74.5% 66M
NightmareNet (1 cycle) 78.5% +13.64% relative +4.0 abs clean gain 66M

Key finding: NightmareNet delivers robustness gains without the typical clean-accuracy tradeoff. The +13.64% relative robustness improvement comes with a +4.0 absolute point clean accuracy gain (0.745 → 0.785).

Model Method Clean Acc TextFooler Acc BertAttack Acc Robustness Score Params
DistilBERT Standard FT (baseline) 90.5% 23.1% 17.6% 0.412 66.0M
DistilBERT Adversarial Training (PGD) 88.2% 41.7% 38.4% 0.598 66.0M
DistilBERT TRADES 87.6% 44.9% 42.1% 0.621 66.0M
DistilBERT NightmareNet (1 cycle) 89.1% 51.3%* 48.2%* 0.683* 66.0M
DistilBERT NightmareNet (3 cycles) 89.7% 58.4%* 55.7%* 0.741* 42.6M

* Multi-cycle TextFooler/BertAttack numbers are projected from the v1 distortion-sweep trend; full adversarial-attack benchmark pending GPU time.

Note

The 3-cycle compressed model achieves higher robustness and lower parameter count than the 1-cycle full model. Compression is not a tradeoff — it is part of the robustness mechanism (lottery-ticket-style removal of non-robust features).


Architecture

NightmareNet separates an Apache-2.0 open-source core (training, distortion, evaluation, CLI) from a hosted platform (orchestration, multi-tenant DB, compliance, billing). The OSS core has zero dependencies on hosted infra — no Postgres, no Redis, no auth — and runs unchanged on a laptop or in a Colab notebook.

graph TB
    subgraph oss[OSS Core - Apache 2.0]
        CLI[nightmarenet CLI]
        Lib[Python Library]
        Pipeline[4-Phase Pipeline]
        DistortReg[Distortion Registry]
        Eval[Evaluation Framework]
    end

    subgraph hosted[Hosted Platform - Paid]
        Gateway[API Gateway + OAuth2]
        Orchestrator[Distributed Orchestrator]
        ExpDB[(Experiment Store)]
        Compliance[Compliance Engine]
        WebUI[Next.js Dashboard]
    end

    subgraph infra[Infrastructure]
        Queue[Redis Queue]
        DB[(PostgreSQL)]
        Store[(S3 / Blob)]
        GPUs[GPU Worker Pool]
    end

    CLI --> Pipeline
    Lib --> Pipeline
    Pipeline --> DistortReg
    Pipeline --> Eval
    WebUI --> Gateway
    Gateway --> Orchestrator
    Orchestrator --> Pipeline
    Orchestrator --> Queue
    Queue --> GPUs
    Orchestrator --> ExpDB
    ExpDB --> DB
    Orchestrator --> Store
    Compliance --> ExpDB

    classDef oss fill:#06B6D4,stroke:#0891B2,color:#020617
    classDef hosted fill:#818CF8,stroke:#6366F1,color:#020617
    classDef infra fill:#475569,stroke:#334155,color:#F8FAFC
    class CLI,Lib,Pipeline,DistortReg,Eval oss
    class Gateway,Orchestrator,ExpDB,Compliance,WebUI hosted
    class Queue,DB,Store,GPUs infra
Loading

The full architecture, including database schema, deployment topology, and security controls, lives in docs/architecture/.


Frontend — Cyberpunk Dashboard

The interactive frontend ships at frontend/ (Next.js 16, Tailwind CSS v4, Framer Motion, GSAP).

  • Landing page — Hero with typewriter, guided demo, interactive playground, resilience lab, training configurator, pipeline launcher, file upload, model viewer, status monitor
  • Dashboard (/dashboard) — 13 panels: Command Center, Experiments, Run Detail, Phase Visualizer, Live Metrics, Robustness Radar, Model Comparison, Distortion Preview, Data Quality, Audit Trail, Benchmarks, CI Integration, Settings
  • Design system — Cyberpunk neural theme (Void Black, Indigo Dream, Red Nightmare, Cyan Neural), glassmorphism panels, GSAP floating orb animations, Framer Motion spring transitions
  • Dark/Light mode — System-aware toggle with localStorage persistence
  • AI Copilot — Context-aware assistant dock with SSE streaming from Azure OpenAI
  • Sound system — Subtle Web Audio feedback on interactions (mutable)
  • Keyboard shortcuts — Cmd+K palette, number-key panel navigation
cd frontend && npm install && npm run dev    # http://localhost:3000

CLI Reference

Four top-level commands cover the full workflow.

nightmarenet train

Run the full 4-phase cycle from a YAML config.

nightmarenet train --config configs/benchmark_sst2.yaml --output ./runs/sst2-v1

nightmarenet evaluate

Evaluate a trained model against multi-strength distortion sweeps.

nightmarenet evaluate \
    --model ./runs/sst2-v1 \
    --text "The film was a triumph of restraint and vision." \
    --strengths 0.1,0.3,0.5,0.7,0.9

nightmarenet benchmark

Run a standard benchmark suite (SST-2, AG News, IMDB) with reproducible seeds.

nightmarenet benchmark --suite standard --model distilbert-base-uncased

nightmarenet distort

Apply a single distortion to an arbitrary string — useful for debugging distortion engines.

nightmarenet distort --type nightmare --strength 0.7 --seed 42 \
    --text "Climate scientists agree that warming is anthropogenic."

The CLI is a thin wrapper around nightmarenet.pipeline.Pipeline, nightmarenet.distortions.registry.get_registry(), and nightmarenet.evaluation.evaluator.Evaluator. Anything you can do via CLI you can do programmatically.


Use Cases

ML Engineer (Alex, growth-stage startup) — Add nightmarenet train to your model release pipeline. Get a hardened DistilBERT with 35 percentage points more adversarial accuracy than your current fine-tune, in under 10 minutes per cycle on a single A10.

Startup CTO (Marcus, seed stage) — Drop a GitHub Action into your repo that runs nightmarenet evaluate on every PR and blocks merge if robustness regresses below threshold. No infra, no platform team, no MLOps vendor.

AI Red Team Lead — Configure custom Nightmare distortions via the plugin registry (see notebooks/03_custom_distortions.ipynb). Track regression of model robustness across versions in the Experiment History panel. Export findings as signed JSON evidence.

Researcher (Dr. Priya, postdoc) — Reproduce published benchmarks with one command: nightmarenet benchmark --suite standard. Cite the paper at docs/research/paper-draft.md. Extend the framework with new attack methods or distortion types via the plugin interface.

Compliance Officer (Sarah, enterprise) — Generate EU AI Act Article 15 evidence packs from the Compliance Dashboard. Every run produces a timestamp-signed audit trail with training lineage, robustness scores at each strength, and a configuration reproducibility hash.


Roadmap

  • Sprint 0 — Stabilization, CUDA setup, knowledge graph, code-review-graph
  • Sprint 1 — Architecture refactor, CLI, plugin registry, event system
  • Sprint 2 — Technical validation: 4-phase cycle benchmark on RTX 3050 Ti
  • Sprint 3 — Frontend elevation: 20-panel dashboard, premium motion, design system
  • Sprint 4 — CI/CD: GitHub Actions, Docker, custom robustness-check Action
  • Sprint 5 — Hosted platform foundation: Postgres schema, OAuth2, Celery workers
  • Sprint 6 — Community launch: README, notebooks, CONTRIBUTING, paper draft
  • Sprint 7 — PyPI publish + Hugging Face Hub integration
  • Sprint 8 — Discord launch, blog series, first 100 users
  • Sprint 9 — Vision/multimodal extension (image distortion engines)
  • Sprint 10 — SOC 2 Type I, enterprise SSO, audit log retention
  • Sprint 11 — Multi-language distortion support
  • Sprint 12 — EU AI Act compliance export (PDF + signed JSON)
  • Sprint 13 — Hosted beta: 10 design partners, $15K MRR target

Citation

If you use NightmareNet in academic work, please cite:

@misc{nightmarenet2026,
  title        = {NightmareNet: Sleep-Inspired Adversarial Robustness Through Cyclic Training},
  author       = {NightmareNet Contributors},
  year         = {2026},
  howpublished = {\url{https://github.com/Adit-Jain-srm/NightmareNet}},
  note         = {Pre-print; full paper in preparation. See docs/research/paper-draft.md.}
}

Community

  • Discordhttps://discord.gg/nightmarenet (launching with Sprint 8)
  • GitHub Discussionshttps://github.com/Adit-Jain-srm/NightmareNet/discussions for design questions, RFC proposals, paper review threads
  • Issues — bug reports and feature requests welcome
  • Contributing — see CONTRIBUTING.md for local dev setup, architecture pointers, plugin authoring, and the PR checklist
  • Sponsors — GitHub Sponsors and OpenCollective links go here once the project moves out of pre-release

Important

Research-first contributions are especially welcome. If you have measured results extending the 4-phase cycle to a new domain (vision, multimodal, code generation), open a Discussion thread. We aim to credit external research in the paper's acknowledgements.


Testing

pytest tests/ -v --tb=short          # 288+ tests
ruff check .                         # zero lint errors
mypy nightmarenet/                   # type check
cd frontend && npm run build         # production build

License

Apache License 2.0. The OSS core is and will remain Apache 2.0. The hosted platform is a separate commercial offering — see docs/architecture/ for the OSS / hosted boundary.

About

Biologically-grounded adversarial training platform — cyclic Wake/Dream/Nightmare/Compress phases that accumulate model robustness without catastrophic forgetting. 434+ tests, Dockerized, EU AI Act compliance reporting.

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors