Skip to content

feat(l1): add benchmark-grade custom block workloads#71

Open
ilitteri wants to merge 2 commits into
mainfrom
feat/l1-custom-block-workloads
Open

feat(l1): add benchmark-grade custom block workloads#71
ilitteri wants to merge 2 commits into
mainfrom
feat/l1-custom-block-workloads

Conversation

@ilitteri

Copy link
Copy Markdown
Collaborator

Motivation

ethrex-replay custom could only generate blocks of ETH transfers (and --tx erc20-transfer panicked). To benchmark ethrex and the zkVM backends with controlled, reproducible load, we need synthetic blocks that stress execution at different levels — compute, state reads/writes, contract calls, deployments, and realistic DeFi.

What changed

Turns custom block/custom batch into a benchmark-grade workload generator.

Workloads (--tx): eth-transfer, erc20-transfer, uniswap-v2-swap, contract-deploy, keccak, mulmod, ecrecover, sstore-fresh, sload-cold.

Pure blocks invariant: every produced block contains only the requested workload's transactions. Provisioning (token deploys, mints, liquidity) runs at build time on a scratch chain and is folded into genesis — never into a produced block. Contracts we author (the compute/state loop contract, the cold-read prestate) are injected into genesis directly; third-party contracts (ERC20, Uniswap) are provisioned by executing real setup transactions and folding the resulting state, so layouts are correct by construction.

Determinism & sizing: --seed makes generation byte-reproducible; --n-senders distributes transactions across seed-derived, genesis-funded accounts; --gas-target fills a block to a gas budget (alternative to --n-txs); --tx-gas, --deploy-code-size, --prestate-slots tune the loop/deploy/state workloads. Runs persist their cache and can be re-run with --cached. The existing --no-zkvm/--repeat profiling path works on synthetic blocks.

Design notes: the compute/state workloads share one EVM bytecode loop template (EEST JumpLoop shape with a gas-bounded clean exit borrowed from spamoor's gas burner). The uniswap-v2-swap AMM is a minimal Uniswap-V2-style implementation (exact constant-product math + 0.3% fee, real router → pair → token call graph) compiled from committed source (fixtures/contracts/uniswap_v2_min/UniswapV2Min.sol) with solc 0.8.x; tokens reuse the existing TestToken fixture.

Bug fixes: the erc20-transfer panic, and multi-block batch nonce reuse (NonceTooLow) — sender nonces and storage-slot cursors now thread across batch blocks; mempool is drained between blocks.

ethrex bump (required)

Bumps the ethrex dependencies (ziskos 0.15.0 → 0.16.1) and migrates to the new ethrex APIs. This is required: the previously pinned ziskos depended on ZisK patch repositories that have been deleted upstream, so the ZisK guest could not build. The migration threads the new Crypto trait explicitly through the VM/witness layer (code_hash, Code::from_bytecode, tx.sender, get_transactions_with_sender, Evm::new_for_l1/l2, prepare_block, execute_block), splits witness conversion into two steps (into_execution_witness + GuestProgramState::from_witness), drops Holesky, and renames EMPTY_KECCACK_HASH. Adds the ethrex-crypto dependency.

ZisK benchmark

docs/zisk_benchmark.md reports ZisK execution cost for every workload across 1/10/100/1000 txs (run on a 32-core CPU host, ZisK 0.16.1, execution-only — no proving). Headline (ZisK steps per gas, ~130× spread):

Workload steps/gas Workload steps/gas
contract-deploy 0.89 uniswap-v2-swap 5.76
sstore-fresh 1.16 sload-cold 6.06
erc20-transfer 2.60 keccak 11.36
eth-transfer 3.66 mulmod 116.92
ecrecover 5.59

Gas mis-prices zkVM cost in both directions: state growth is gas-expensive but step-cheap, while 256-bit arithmetic (mulmod) is gas-cheap but ~130× step-expensive. Raw data in scripts/zisk_bench_results.csv; the runner is scripts/zisk_bench_matrix.sh.

How to test

cargo run --release -- custom block --tx erc20-transfer --n-txs 100
cargo run --release -- custom block --tx uniswap-v2-swap --n-txs 50
cargo run --release -- custom block --tx mulmod --n-txs 10 --tx-gas 1000000
cargo run --release -- custom batch --n-blocks 3 --tx eth-transfer --n-txs 50 --no-zkvm --repeat 5
cargo run --release -- custom block --tx keccak --gas-target 30000000
cargo test && cargo test --features l2

Notes

  • The ZisK benchmark uses execution (step counts) because the host has no GPU; proving is out of scope here.
  • The real ZisK guest is built via the new zisk-build-elf feature; plain zisk keeps the lightweight compile-only stub so CI is unaffected.
  • Suggest a follow-up to pin the ethrex dependencies to a fixed rev instead of branch = "main" so builds are reproducible (cargo install currently drifts onto incompatible upstream commits).

ilitteri added 2 commits June 12, 2026 18:58
Extend `custom block`/`custom batch` from ETH-transfers-only into a
synthetic workload generator for benchmarking L1 execution and zkVMs.

Workloads (--tx): eth-transfer, erc20-transfer, uniswap-v2-swap,
contract-deploy, keccak, mulmod, ecrecover, sstore-fresh, sload-cold.
Each produced block contains only the requested workload's transactions:
contract provisioning (deploys, mints, liquidity) runs at build time on a
scratch chain and is folded into genesis, never into produced blocks.

Generation is deterministic (--seed), distributed across --n-senders
seed-derived genesis-funded accounts, and supports --gas-target block
filling, --tx-gas/--deploy-code-size/--prestate-slots knobs, cache
persistence with --cached re-run, and the existing --no-zkvm/--repeat
profiling path. Compute and state workloads share one EVM bytecode loop
template (EEST JumpLoop shape with a gas-bounded clean exit); the
Uniswap-V2-style AMM is compiled from committed source with solc 0.8.x.

This also fixes the prior erc20-transfer panic and the multi-block batch
nonce-reuse bug (sender nonces and slot cursors now thread across blocks).

Bumps the ethrex dependencies (ziskos 0.15.0 -> 0.16.1) and adapts to the
new ethrex APIs: the Crypto trait is threaded explicitly through the
VM/witness layer (code_hash, Code::from_bytecode, tx.sender,
get_transactions_with_sender, Evm::new_for_l1/l2, prepare_block,
execute_block), witness conversion is two-step (into_execution_witness +
GuestProgramState::from_witness), Holesky was removed, and
EMPTY_KECCACK_HASH was renamed. The bump is required because the older
ziskos pinned ZisK patch repositories that have been deleted upstream.

See docs/custom_blocks.md.
Benchmark every custom L1 workload in the ZisK zkVM (execution / step
counts, no proving) across 1/10/100/1000 transactions per block, run on a
32-core CPU host with ZisK 0.16.1.

The headline metric is ZisK steps per gas, which spans ~130x across
workloads (contract-deploy 0.89 to mulmod 116.9): gas mis-prices zkVM
cost in both directions, with state growth cheap in steps and 256-bit
arithmetic expensive. Full tables, methodology and the raw CSV are
included.
Copilot AI review requested due to automatic review settings June 12, 2026 21:59
@github-actions github-actions Bot added the L1 label Jun 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades ethrex-replay custom into a deterministic, benchmark-oriented L1 synthetic workload generator (pure-workload blocks with build-time provisioning folded into genesis), while migrating the replay/witness pipeline to the newer ethrex APIs (notably the explicit Crypto plumbing).

Changes:

  • Added a new workloads module implementing multiple benchmark workloads (transfers, ERC20, Uniswap V2-style swaps, deploys, and compute/state loop templates) with deterministic seed-driven generation and batch-safe nonce/state threading.
  • Reworked custom L1 block/batch generation to build blocks in-process, fold setup transactions into genesis, support --gas-target, persist/reload caches, and ensure produced blocks contain only workload transactions.
  • Migrated witness conversion/execution code to new ethrex APIs (decode_witness_headers, into_execution_witness, GuestProgramState::from_witness, explicit NativeCrypto usage), and added end-to-end tests + documentation/benchmark artifacts.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/custom_workloads.rs New E2E tests validating workload purity, determinism, caching, gas targeting, and sender distribution.
src/workloads/mod.rs Core workload enum + deterministic generation context (GenCtx) and EIP-1559 tx builder helper.
src/workloads/eth_transfer.rs Baseline ETH transfer workload implementation.
src/workloads/erc20.rs ERC20 deploy/mint provisioning (folded) + per-block transfer tx generation.
src/workloads/uniswap.rs Uniswap V2-min style provisioning (folded) + per-block router swap tx generation.
src/workloads/loops.rs Loop-template workloads + contract-deploy workload wiring.
src/workloads/bytecode.rs Minimal bytecode “assembler” + runtime/initcode builders for compute/state loop workloads.
src/tx_builder.rs Removed legacy tx builder (superseded by workloads).
src/cli.rs Major custom L1 generation rewrite: sizing (--n-txs/--gas-target), caching, folding setup into genesis, mempool draining, and new workload options.
src/cache.rs Added Cache::write_named to support deterministic custom cache filenames.
src/run.rs Updated witness decode/convert and guest state construction to new ethrex APIs + explicit NativeCrypto.
src/rpc/mod.rs Updated constant rename to EMPTY_KECCAK_HASH.
src/rpc/db.rs Updated sender recovery/code hash/code construction and LEVM calls to pass NativeCrypto.
src/lib.rs Exported cache and introduced public workloads module.
src/fetcher.rs Adapted to updated get_block_number() return type.
Cargo.toml Added ethrex-crypto dependency and tokio dev-dependency features for tests; introduced zisk-build-elf feature.
README.md Updated custom command description and linked docs.
docs/custom_blocks.md New documentation describing workloads, sizing, determinism, caching, and architecture.
docs/zisk_benchmark.md New ZisK execution benchmark report for all workloads.
scripts/zisk_bench_matrix.sh Benchmark runner script to generate the matrix CSV.
scripts/zisk_bench_results.csv Added benchmark result dataset.
fixtures/contracts/uniswap_v2_min/UniswapV2Min.sol Added minimal Uniswap V2-style contracts used for the swap workload.
Comments suppressed due to low confidence (1)

src/run.rs:140

  • run_tx allocates a fresh Arc for NativeCrypto multiple times (for the wrapper DB and again per-transaction when constructing the EVM). This adds avoidable heap allocations in a hot path; creating a single Arc and cloning it is enough.
    let mut wrapped_db = GuestProgramStateWrapper::new(guest_program_state, Arc::new(NativeCrypto));

    #[cfg(feature = "l2")]
    let fee_config = FeeConfig::default();


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/cli.rs
Comment on lines 417 to 420
#[derive(Parser)]
#[command(group(ArgGroup::new("block_size").args(["n_txs", "gas_target"]).multiple(false)))]
pub struct CustomBlockOptions {
#[command(flatten)]
Comment thread src/workloads/erc20.rs

let mut txs = vec![
build_eip1559(
0,
Comment thread src/workloads/erc20.rs
.map_err(|err| eyre::eyre!("failed to encode freeMint calldata: {err}"))?;
for (i, sender) in senders.iter().enumerate() {
// The deployer already consumed nonce 0 with the deploy.
let nonce = if i == 0 { 1 } else { 0 };
Comment thread src/workloads/erc20.rs
.map_err(|err| eyre::eyre!("failed to encode freeMint calldata: {err}"))?;
for (i, sender) in senders.iter().enumerate() {
// The deployer already consumed nonce 0 with the deploy.
let nonce = if i == 0 { 1 } else { 0 };
Comment thread src/workloads/mod.rs
Ok(Self {
chain_id,
seed,
next_nonces: vec![0; senders.len()],
Comment thread src/workloads/mod.rs
self.next_nonces[i] = store
.get_nonce_by_account_address(latest, sender.address())
.await?
.unwrap_or(0);
Comment thread src/workloads/mod.rs
let i = self.next_sender;
self.next_sender = (self.next_sender + 1) % self.senders.len();
let nonce = self.next_nonces[i];
self.next_nonces[i] += 1;
Comment thread src/workloads/uniswap.rs
.map_err(|err| eyre::eyre!("invalid router initcode: {err}"))?;

// Per-sender nonce tracking for the setup transactions.
let mut nonces = vec![0u64; senders.len()];
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants