Skip to content

CLAUDE.md

Latest commit

 

History

History
97 lines (78 loc) · 4.96 KB

CLAUDE.md

File metadata and controls

97 lines (78 loc) · 4.96 KB
Raw

CLAUDE.md — NeuralYul

Project-level instructions. Extends (does not replace) the global ~/.claude/CLAUDE.md.

What this project is

ML-guided Yul pass orchestrator + EVM bytecode superoptimizer for solc. Two optimization boundaries (Yul-MLGO strategist, bytecode RL tactician) plus a correctness gate (differential fuzzing + Z3). Full math spec in docs/Architecture.md and docs/Architecture_update.md; spec→file map in docs/INFRASTRUCTURE.md.

The hard boundary: infra is built, models + data are NOT

This repo is the infrastructure around the AI, not the AI. When asked to "add a model" or "train", respect this seam:

  • Excluded: the GIN backbone body, NT-Xent/EMA/PPO/PRM training loops, the YulCode dataset (350k contracts), and any weights/checkpoints/.onnx.
  • Present: parser, dataset engine, RL env, gas bridge, ring buffer, correctness gate, solc/foundry/daemon glue.
  • The seam is src/neuralyul/models/ — typed ABCs whose methods raise NotAModelError. Drop a real NN in by subclassing; nothing else should change.

If a task would put network bodies, training code, weights, or dataset files in this repo, stop and confirm — that crosses the boundary.

Layout

src/neuralyul/        Python infra package (importable with only networkx)
  config.py           SINGLE SOURCE OF TRUTH for hyperparams + the 36-d feature layout
  pdg/                Yul PDG parser: AST + CFG + DFG, labelled edges (0/1/2)
  data/               NFM/EP/SE augmentations + PyG two-view contrastive dataset
  models/             interfaces ONLY (the boundary)
  env/                24 Yul passes, solc driver, Gymnasium env (reward = gas saved)
  reward/             gas bridge (mock + PyO3) + shared-memory ring buffer client
  correctness/        differential fuzzer + Z3 equivalence (KECCAK as UIF)
  daemon.py           FastAPI superoptimizer API
  cli.py              `neuralyul` entrypoint (parse/passes/gas/fuzz)
executor/             Rust: ring buffer + GasBackend (Mock/revm) + PyO3 + worker bin
solc-plugin/          C++: FeatureExtractor + YulMLRunner (ONNX) + Suite.cpp hook
orchestration/        Rust Foundry wrapper (neuralyul-forge)
src/L1-encoder/       original AST parser + visualizer (predates the package)
tests/                pytest; optional-dep suites self-skip

Dev environment

nix develop provides python312, rust, maturin, solc 0.8.33, z3, graphviz. Python deps go in the project .venv via pip (the flake's shellHook creates/activates it) — this venv is the project-local exception to the global "Nix only" policy. Heavy ML deps are optional extras (.[graph], .[rl], .[verify], .[daemon]); base install needs only networkx.

Build / run

# Python infra (mock gas backend — no native build needed):
neuralyul parse|passes|gas|fuzz src/L1-encoder/sample.yul
pytest                                              # optional suites self-skip

# Rust executor:
cargo build --manifest-path executor/Cargo.toml     # lib + worker (mock)
maturin develop -m executor/Cargo.toml              # PyO3 ext (mock) -> venv
maturin develop -m executor/Cargo.toml --features revm   # PyO3 ext (REAL EVM)

load_default_evaluator() returns the PyO3 backend if the extension is installed, else the pure-Python mock — so everything runs either way.

Non-obvious gotchas (learned the hard way — keep these intact)

  1. solc Yul AST needs outputSelection {"*":{"":["ast"],"*":["*"]}}. Requesting only ["ast"] returns {"errors":[]} with no AST. (pdg/parser.py.)
  2. 36-dim feature layout is a contract. config.NODE_FEATURE_DIM == 36, indices 32/33/34/35 are fixed. The C++ FeatureExtractor and NODE_VOCAB must produce byte-identical vectors to the Python featurizer or the ONNX model sees OOD input. Parity table is in solc-plugin/README.md.
  3. Ring buffer byte layout is duplicated in reward/ringbuffer.py and executor/src/protocol.rs. Change BOTH in lockstep; the magic+version header is checked at attach (0x4E59554C / v1). SPSC discipline: publish head/tail LAST.
  4. PyO3 0.23 requires #[pyclass] to be Sync. GasEvaluator holds a non-Sync boxed backend, so it's #[pyclass(unsendable)] (single-threaded use). Don't drop that attribute.
  5. revm is feature-gated (--features revm) and not built by default/CI. Cargo resolves revm 14.x; the real backend targets the Context::mainnet() API and may need a version bump — if so, only executor/src/evm.rs::revm_backend changes.
  6. The mock gas evaluator is semantics-blind (gas from size; output hash from bytes). It validates the fuzzer's compare logic, not real equivalence — real semantic checks require the revm backend + the Z3 verifier.

Conventions

Global rules apply (immutability, small files, conventional commits, no Co-Authored-By). Frozen dataclasses for config. Keep optional deps lazily imported so the base package stays importable. New hyperparameters go in config.py, never hardcoded at use sites.