fix(opstack): server receives no mainnet blocks — V4 topic, yamux, identify, OP-spec max message size#801
Conversation
Base mainnet's sequencer publishes signed unsafe heads on the V4 topic (/optimism/<chain_id>/3/blocks) since Isthmus; subscribers listening only on the V3 topic (/2/blocks) receive nothing and the consensus server never serves a head. Matches the symptom of the public operationsolarstorm mainnet feeds (502 / no data) while base-sepolia still works. Keep V3 subscription for chains that still publish on it; the handler verifies the same sequencer-signed commitment envelope on both.
…ge size Two independent defects starved the consensus server of blocks (heads went stale for 20-40 min windows, with only short bursts of delivery): 1. No libp2p identify behaviour. go-libp2p based op-nodes close connections ~300ms after establishment when /ipfs/id/1.0.0 is unsupported (observed: every dial to a Base mainnet peer ended with a remote-initiated close; gossipsub mesh stayed at 0 of 8). Add identify (empty protocol version, agent "helios"), mirroring kona. 2. Default gossipsub max_transmit_size (64 KiB). The first Base block larger than 64 KiB kills the peer's inbound gossipsub stream with "Failed to read from inbound stream: Failed to encode/decode message" and delivery from that peer never resumes. Set the OP spec gossip maximum (10 MiB), same as kona's MAX_GOSSIP_SIZE and op-node's MaxGossipSize. A/B verified against an unpatched instance on Base mainnet: unpatched received 14 blocks in 10 min (all conns dropped in ~300ms); with identify connections persist, GRAFT lands, and blocks arrive at full 2s cadence until a >64 KiB block kills the stream; with both fixes delivery is continuous.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
Problem
The opstack consensus server (
opstack/bin/server.rs) receives no blocks on any mainnet OP-stack chain:/latestservesnull(or a stale head) indefinitely. Four independent defects, found by running the server against Base mainnet withlibp2p_gossipsub=debugand fixing one variable at a time.1. Subscribed to the wrong gossip topic (original scope of this PR)
BlockHandlersubscribes only to the V3 blocks topic:but op-node publishes each block to exactly one topic version, selected by hardfork (op-node
p2p/gossip.go):Post-Isthmus mainnets publish on V4 (
/optimism/<chain_id>/3/blocks) only. Fix: subscribe V4 alongside V3 (the commitment envelope and signature scheme are identical on both).2. mplex-only transport — modern op-nodes can't multiplex with us
The server's transport offers mplex only (libp2p 0.51 era). mplex has been deprecated and dropped across the go-libp2p ecosystem; current Base/OP mainnet peers speak yamux. Measured: 7+ minutes, dozens of dials, zero connections established, zero blocks. Fix: bump libp2p to 0.56 and use yamux (
chore: update libp2p to 0.56.0+ clippy follow-up, cherry-picked from our integration branch).3. No
identifybehaviour — peers close every connection within ~300 msWith yamux in place connections establish, but go-libp2p based op-nodes close them ~100–300 ms later, remote-initiated, 100% of dials:
gossipsub's mesh stays
0 of 8forever; blocks only leak in when a brand-new peer briefly delivers before hanging up — bursts of seconds separated by 20–40 min of silence. The missing piece is theidentifyprotocol (/ipfs/id/1.0.0): kona composesping + gossipsub + identify (+ sync req/resp); this server had onlyping + gossipsub. Fix mirrors kona:identify::Config::new("", public_key).4. 64 KiB default
max_transmit_size— one big block silences a peer foreverWith identify in place the mesh GRAFTs and blocks stream at full 2 s cadence — until the first Base block larger than 64 KiB (the libp2p gossipsub default) arrives:
The peer's inbound gossip stream dies and never recovers, while the connection stays up — silent starvation. (The first V4 message we ever captured was 59,502 bytes — already brushing the limit.) Fix: OP spec gossip maximum, 10 MiB — same as op-node's
MaxGossipSizeand kona'sMAX_GOSSIP_SIZE.Verification (Base mainnet, A/B, one variable at a time)
/latest=null/latestcontinuously advancingThe full fix set has been running in production since 2026-06-04 — 6 k8s instances across 2 regions (initially also 3 bare-metal hosts, since consolidated): every instance converges to full 2 s block cadence and holds.
Possibly related: the public
base.operationsolarstorm.org/op-mainnet.operationsolarstorm.orgconsensus endpoints currently return HTTP 502 whilebase-sepolia.operationsolarstorm.orgworks — consistent with V3-only subscribers behind the mainnet feeds.