Skip to content

feat: Circle v2 strategies + 4 v1-vs-v2 benches (industry-default → Helix discovery)#7

Merged
adrianhihi merged 2 commits into
mainfrom
feat/circle-v2-strategies-bench
May 26, 2026
Merged

feat: Circle v2 strategies + 4 v1-vs-v2 benches (industry-default → Helix discovery)#7
adrianhihi merged 2 commits into
mainfrom
feat/circle-v2-strategies-bench

Conversation

@adrianhihi

Copy link
Copy Markdown
Collaborator

Circle v2 Strategies + V1-vs-V2 Benches — PR #3

Follow-up to merged PR #1 (Circle Foundation) and PR #2 (Experimental Capsules). Adds 3 new v2 strategies + 4 v1-vs-v2 comparison benches. Reveals industry-default approaches have limitations Helix discovers through systematic experimentation.


Headline Results

Bench v1 (industry default) v2 (Helix discovery) Improvement
2.1 Rate Limit (sustained load) serialize_and_backoff — 76% success, 24/100 lost chunk_concurrent — 100% success, 0 lost +24 percentage points reliability
2.2 Param Routing hold_and_notify — 0% auto-fix smart_param_repair — 80% routing to auto-fix 80x more issues routed for repair
2.3 Metadata Trust override_api_decimals — decimals only metadata-trust-on-chain — decimals + symbol + audit-source Detects symbol drift (USDC vs USDC.e) v1 misses entirely
2.4 Param E2E (live) (validated via Bench 2.2) strip_field20/20 real Arc Testnet tx submitted First end-to-end auto-fix validation on real Circle Sandbox

Real data, real Arc Testnet (220 total live transactions), JSON in scripts/circle-bench/results/.


Honest Anomalies Surfaced

PR #3 includes 2 documented findings that PR #4 will address:

  1. v1-serialize-and-backoff-anomaly.md: v1 retries fire concurrently and re-trigger rate limit under sustained load, losing 24% of transfers.

  2. refresh-metadata-reachability.md: v2's refresh_metadata action is currently unreachable on live Circle (Circle's tokenId errors take other paths). PR feat: Circle experimental capsules — decimals & stale_quote (2 of 2) #4 will add numeric-code routing for code 156002refresh_metadata.

This is research-grade honesty: surface findings, document, address in next PR.


Code Delivered

3 standalone v2 strategies (not yet integrated into PCEC engine — Gene Map multi-strategy is PR #4):

  • packages/core/src/strategies/circle-v2/chunk-concurrent.ts (100 lines)
  • packages/core/src/strategies/circle-v2/smart-param-repair.ts (255 lines)
  • packages/core/src/strategies/circle-v2/metadata-trust.ts (207 lines)

4 bench scripts:

5 JSON result files + 2 analysis docs in scripts/circle-bench/results/


Test Coverage

53 new tests added. Full suite: 623/623 passing. No regressions.


Scope Boundaries (intentional)

v2 strategies are validated standalone via bench. They become first-class Gene Map citizens in PR #4 with proper Thompson Sampling between v1/v2 alternatives.


Roadmap (estimated)

See Section 9 of the walkthrough document (circle-walkthrough-v2.md) for full self-improving roadmap.


Verification

git checkout feat/circle-v2-strategies-bench
npm install && npm run build -w packages/core
npm test  # 623/623

# Re-run benches on your own Circle Sandbox account
set -a; source scripts/circle-bench/.env; set +a

# Bench 2.1 — real Arc Testnet load test
BENCH_TRANSFERS=20 BENCH_TRIALS=5 \
  npx tsx scripts/circle-bench/v1-vs-v2-rate-limit.ts

# Bench 2.2 — routing correctness (synthetic shapes)
npx tsx scripts/circle-bench/v1-vs-v2-param-repair.ts

# Bench 2.3 — metadata trust (partial RPC)
npx tsx scripts/circle-bench/v1-vs-v2-metadata-trust.ts

# Bench 2.4 — smart_param_repair real E2E (consumes ~20 testnet USDC for Group A)
npx tsx scripts/circle-bench/v2-smart-param-e2e.ts

JSON results land in scripts/circle-bench/results/ for inspection.

…ity analysis

20 real Arc Testnet tx in Group A (idempotencyKey strip).
20 correct holds in Group C (UUID format violation).
Group B revealed refresh_metadata action is design-intent only —
Circle's actual tokenId error shapes route to hold_and_notify
via uuid_format (Rule 2) or catch-all (no errors[] array).
Documented for PR #4 numeric-code routing.
Companion script to commit fd3ccf7 which captured Bench 2.4 results
and refresh_metadata reachability analysis. Tests the three groups:
- Group A: idempotencyKey deliberate trigger (strip_field path)
- Group B: nil-UUID tokenId trigger (uuid_format violation path)
- Group C: 'not-a-uuid' walletId trigger (hold_and_notify path)

Allows independent reproduction of Bench 2.4 findings.
@adrianhihi adrianhihi merged commit fe3833c into main May 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant