Draft: stateless validator by envestcc · Pull Request #4809 · iotexproject/iotex-core

envestcc · 2026-03-19T09:50:17Z

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

[] Bug fix (non-breaking change which fixes an issue)
[] New feature (non-breaking change which adds functionality)
[] Code refactor or improvement
[] Breaking change (fix or feature that would cause a new or changed behavior of existing functionality)
[] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[] make test
[] fullsync
[] Other test (please specify)

Test Configuration:

Firmware version:
Hardware:
Toolchain:
SDK:

Checklist:

[] My code follows the style guidelines of this project
[] I have performed a self-review of my code
[] I have commented my code, particularly in hard-to-understand areas
[] I have made corresponding changes to the documentation
[] My changes generate no new warnings
[] I have added tests that prove my fix is effective or that my feature works
[] New and existing unit tests pass locally with my changes
[] Any dependent changes have been merged and published in downstream modules

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

tools/witnessmeasure: new CLI that fetches block witnesses via debug_traceBlockWitnessByNumber and measures: - witness JSON payload size - contracts/entries/proofNodes/proofBytes per block - VerifyContractStorageWitness timing per block and per contract - linear regression model: proof_bytes and verify_ms vs exec_tx count blockchain/witness: export ContractWitness.ToEVMWitness() so external tools can call proof verification without re-implementing the conversion. testnet_samples.csv contains 15 blocks spread across 5 exec-tx-count quintiles (5-1803 txs/block) from the testnet 28M height range for a representative measurement run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Previously verifyContractStorageProof rebuilt the hash→node map on every entry call, making verification O(n²) in the number of witness entries. Now the map is built once in VerifyContractStorageWitness and shared across all entries, reducing complexity to O(n·depth). Also removes the unused in-memory trie construction that served no purpose in the verification path. Benchmark results (Apple M1 Pro): entries=10: 82µs → 24µs (3.4×) entries=50: 1.85ms → 302µs (6.1×) entries=100: 7.53ms → 958µs (7.9×) entries=200: 32.9ms → 3.2ms (10.3×) entries=500: 253ms → 10.7ms (23.6×) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

sonarqubecloud · 2026-03-19T09:51:17Z

Quality Gate failed

Failed conditions
5.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Copilot

Pull request overview

This PR introduces an experimental “stateless validator” workflow by adding contract-storage witness capture/verification, new Web3 debug endpoints to expose witnesses and storage access summaries, and new sync modes (CS sync / standalone sync) to consume blocks plus witness sidecars.

Changes:

Add Merkle Patricia Trie proof export/verification and EVM contract-storage witness generation/verification primitives.
Persist and serve per-block witness payloads (via a dedicated witness DB) and expose new Web3 debug RPC methods for block witness/storage tracing and retrieval.
Add experimental syncing paths (CS sync and standalone sync) and developer tools for validating/measuring witness payloads.

Reviewed changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tools/witnessmeasure/testnet_samples.csv	Adds sample block heights for witness measurement runs.
tools/witnessmeasure/main.go	New CLI tool to fetch witness payloads via RPC and measure size/verify time.
tools/statelessdemo/main.go	New demo verifier that polls blocks and validates witness summaries + proofs.
tools/statelessdemo/main_test.go	Unit tests for statelessdemo validation helpers.
state/factory/workingsetstore_test.go	Adds test coverage for flusher options that skip contract-storage flush.
state/factory/workingsetstore_erigon_simulate.go	Extends Erigon simulate store to retain a base store reference.
state/factory/workingset.go	Adds BaseState/BasePutState/BaseDelState to bypass Erigon dry-run store.
state/factory/statedb.go	Adds skip-contract-storage flush option when stateless validation is enabled.
server/itx/server.go	Enables real p2p agent in standalone mode under experimental flag.
db/trie/trie.go	Introduces ProofTrie interface (GetProof/VerifyProof).
db/trie/mptrie/merklepatriciatrie.go	Implements GetProof and VerifyProof for MPT.
db/trie/mptrie/merklepatriciatrie_test.go	Adds tests for proof generation/verification and tamper detection.
consensus/consensus.go	Validates blocks before committing in consensus callback.
chainservice/stateless_witness.go	Adds RPC client for fetching persisted block witnesses by hash.
chainservice/stateless_witness_test.go	Tests for stateless witness RPC client and parsing.
chainservice/cs_syncer.go	Adds CS syncer to fetch blocks via gRPC and witnesses via Web3/debug RPC, with local caching.
chainservice/cs_syncer_test.go	Tests witness caching behavior and Start() non-blocking behavior.
chainservice/builder.go	Wires experimental stateless validation and CS sync into chainservice builder.
blockchain/witness/store.go	Adds witness persistence store (bolt DB) + parsing into stateless validation context.
blockchain/witness/json.go	Adds JSON witness schema and conversion to EVM witness structures.
blockchain/witness/collector.go	Adds runtime witness collector to capture per-action witnesses during block execution.
blockchain/config.go	Adds experimental config flags for standalone sync, stateless validation RPC, CS sync, witness DB path.
blockchain/blockchain_witness_test.go	Ensures witness store is skipped for CS-sync consumer nodes.
blockchain/blockchain.go	Integrates witness collection/persistence, stateless validation option plumbing, and witness retrieval APIs.
api/web3server_utils.go	Updates tx decoding to use CodeAt(ctx, …) and extends EVM tracer to capture storage access/witnesses.
api/web3server_marshal.go	Adds JSON marshaling for storage access summaries and witness payloads; block-level summarizers.
api/web3server.go	Adds new debug RPC methods for block storage tracing and witness retrieval endpoints.
api/web3server_test.go	Adds extensive tests for new trace/witness JSON outputs and new debug endpoints.
api/mock_apicoreservice.go	Updates mock core service with BlockWitnessByNumber/BlockWitnessByHash methods.
api/coreservice.go	Adds block witness getters; adds no-index fallback for action/receipt lookups; wires tracer callbacks into EVM.
api/coreservice_test.go	Adds tests for no-index action/receipt lookup and trace call witness capture.
action/protocol/execution/evm/context.go	Adds stateless validation context + tracer hooks for storage accesses/witnesses.
action/protocol/execution/evm/storage_access_trace.go	Adds storage access tracer and stable ordering utilities.
action/protocol/execution/evm/storage_witness.go	Adds contract storage witness types and helpers.
action/protocol/execution/evm/storage_witness_test.go	Tests witness building requirements and absence proofs.
action/protocol/execution/evm/storage_witness_validate.go	Adds witness proof verification logic.
action/protocol/execution/evm/storage_witness_validate_test.go	Tests verification success and tamper rejection.
action/protocol/execution/evm/evmstatedbadapter.go	Adds opt-in storage access tracing + witness assembly from cached contracts.
action/protocol/execution/evm/evmstatedbadapter_test.go	Tests storage access tracing and reset behavior.
action/protocol/execution/evm/evmstatedbadapter_erigon.go	Adds witness assembly for Erigon adapter and stateless contract selection.
action/protocol/execution/evm/evm.go	Emits traced accesses/witnesses via tracer context; enables tracing option when tracer is present.
action/protocol/execution/evm/evm_test.go	Tests storage tracing opt-in via tracer context.
action/protocol/execution/evm/tracer.go	Ensures helper context exists for tracing and makes TraceEnd tolerant of missing VM tracer.
action/protocol/execution/evm/contract.go	Adds witness building, pre-state snapshot capture, and stateless contract implementation.
action/protocol/execution/evm/contract_adapter.go	Ensures adapter forwards BuildStorageWitness.
action/protocol/execution/evm/contract_erigon.go	Declares BuildStorageWitness unsupported for pure erigon contract.
action/protocol/execution/evm/contract_stateless_kvstore.go	Adds mirrored KV store to keep digest consistent while using in-memory proof nodes.
action/protocol/execution/evm/contract_stateless_test.go	Comprehensive tests for stateless contract behavior and Erigon integration.
action/protocol/execution/evm/contract_dryrun_hybrid.go	Adds dry-run hybrid contract to separate execution reads from proof assembly reads.
action/protocol/execution/evm/contract_dryrun_hybrid_test.go	Tests dry-run hybrid routing and error propagation.
action/protocol/execution/evm/witness_bench_test.go	Adds benchmarks/measurement for witness size and verification cost.
Makefile	Adds build target for statelessdemo tool and includes it in build-all.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+		_, blk, _, err := core.lookupActionByHashNoIndex(h)
+		if err != nil {
+			return nil, err
+		}
+		receipt := filterReceipts(blk.Receipts, h)
+		if receipt != nil {
+			return receipt, nil
+		}
+		receipts, err := core.dao.GetReceipts(blk.Height())
+		if err != nil {
+			return nil, err
+		}
+		if receipt := filterReceipts(receipts, h); receipt != nil {
+			return receipt, nil
+		}
+		return nil, errors.Wrapf(ErrNotFound, "failed to find receipt for action %x", h)


+func (core *coreService) lookupActionByHashNoIndex(h hash.Hash256) (*action.SealedEnvelope, *block.Block, uint32, error) {
+	for height := core.bc.TipHeight(); height >= 1; height-- {
+		blk, err := core.dao.GetBlockByHeight(height)
+		if err != nil {
+			return nil, nil, 0, errors.Wrap(ErrNotFound, err.Error())
+		}


+				svCtx, err = witnessClient.blockWitnessByHash(context.Background(), blk.HashBlock())
+				if err != nil {
 					return err
 				}
 			}


+	ctx, cancel := context.WithTimeout(context.Background(), maxDuration(s.blocksyncCfg.Interval, 30*time.Second))
+	defer cancel()


+	if err := c.call(ctx, "debug_getBlockWitnessByHash", []any{common.BytesToHash(blockHash[:]).Hex()}, &result); err != nil {
+		if strings.Contains(err.Error(), "not exist in DB") {
+			return append(json.RawMessage(nil), _emptyBlockWitnessResult...), nil
+		}


+func (s *mirroredTrieKVStore) Start(ctx context.Context) error {
+	if err := s.primary.Start(ctx); err != nil {
+		return err
+	}
+	return s.mirror.Start(ctx)
+}


+func (s *Store) PutRaw(blockHash hash.Hash256, height uint64, raw json.RawMessage) error {
+	if err := s.db.Put(byHashNamespace, blockHash[:], raw); err != nil {
+		return err
+	}
+	return s.db.Put(hashByHeightNamespace, byteutil.Uint64ToBytesBigEndian(height), blockHash[:])
+}


sonarqubecloud · 2026-05-15T08:25:08Z

Quality Gate failed

Failed conditions
5.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

envestcc and others added 16 commits March 16, 2026 16:51

Add contract storage access tracing

d23e423

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Expose storage trace summaries

e6b6bcb

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add MPT proof verification

ed670bd

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add storage witness assembly

d527eab

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Expose storage witnesses in debug trace

68569c4

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Capture prestate storage witnesses

a4aca33

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Support dryrun hybrid witnesses

fa155b1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add block witness trace APIs

b7e31b0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove action-index Web3 dependencies

d91e7cf

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add stateless witness plumbing

696ab39

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Harden stateless contract proofs

0710cf9

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Skip stateless contract flush

d88abd3

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add stateless demo verifier

7775ab1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix evm helper

9d0a8eb

envestcc requested a review from Copilot March 19, 2026 09:50

Copilot AI reviewed Mar 19, 2026

View reviewed changes

envestcc closed this May 15, 2026

envestcc reopened this May 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: stateless validator#4809

Draft: stateless validator#4809
envestcc wants to merge 16 commits into
rc235from
testing-statelessvalidator

envestcc commented Mar 19, 2026

Uh oh!

sonarqubecloud Bot commented Mar 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

sonarqubecloud Bot commented May 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		ctx, cancel := context.WithTimeout(context.Background(), maxDuration(s.blocksyncCfg.Interval, 30*time.Second))
		defer cancel()

Conversation

envestcc commented Mar 19, 2026

Description

Type of change

How Has This Been Tested?

Checklist:

Uh oh!

sonarqubecloud Bot commented Mar 19, 2026

Quality Gate failed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

sonarqubecloud Bot commented May 15, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants