Skip to content

fix: harden and simplify hyperswarm negentropy sync#1198

Merged
Bushstar merged 53 commits into
mainfrom
bush/gc-profiler
May 2, 2026
Merged

fix: harden and simplify hyperswarm negentropy sync#1198
Bushstar merged 53 commits into
mainfrom
bush/gc-profiler

Conversation

@Bushstar

@Bushstar Bushstar commented Apr 20, 2026

Copy link
Copy Markdown
Contributor

Summary

This branch improves Hyperswarm sync reliability around negentropy, switches Gatekeeper’s Postgres backend to a row-per-event storage model.

What Changed

  • Gatekeeper Postgres storage now uses a row-based event store as the primary persistence model, with migration support from the older JSONB-array layout.
  • Hyperswarm negentropy was reworked around paged history windows and per-session state:
    • sync now rebuilds the local sync-store from Gatekeeper on startup instead of trying to preserve drifted state
    • the mediator now uses per-session/per-window negentropy engines rather than sharing one mutable live engine across peers
  • Hyperswarm transport now supports framed messaging for upgraded peers:
    • added length-prefixed framing utilities
    • negotiated backward-compatible transport mode per connection
    • kept the initial ping in legacy/raw form so upgraded nodes can still talk to legacy peers during capability exchange
  • Sync persistence and transfer behavior were tightened:
    • avoid re-importing already-known ops
    • avoid re-sending known ops and improve ordering/normalisation of transferred sync records
    • prioritise negentropy before legacy fallback

@Bushstar Bushstar requested a review from Edgeman-SelfID April 20, 2026 11:10
Bushstar added 24 commits April 21, 2026 08:01
Return all exportable events when hashes are omitted.
Update gatekeeper API/client/types, hyperswarm call sites, and tests.
Add optional hashes argument to exportBatch for hash-based event export.
Remove exportEvents method and /events/export route.
Update mediator call sites and tests to use exportBatch(dids, hashes).

@Edgeman-SelfID Edgeman-SelfID left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Bushstar Bushstar merged commit c302224 into main May 2, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants