feat(communities): enables embedding pinned communities in the binary#7562
Merged
Conversation
✅ Companion PR Verified#21349 - feat(communities): up status-go to get pinned communities feature |
Member
Jenkins BuildsClick to see older builds (66)
|
There was a problem hiding this comment.
Pull request overview
This PR adds a “pinned communities” bootstrap path to the protocol Messenger startup, allowing one or more community description payloads to be bundled into the binary (or overridden from a local directory) and imported into the DB on first run so communities are visible before store-node fetches complete.
Changes:
- Embed
*.rawpayload.hexcommunity description payload(s) into the binary and add a loader for embedded vs directory-based sources. - Add an async Messenger startup bootstrap that imports pinned community payloads for new profiles and then triggers non-blocking store-node refreshes.
- Add tests and documentation describing the bootstrap behavior and source precedence.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| protocol/README.md | Documents pinned community bootstrap behavior and source precedence. |
| protocol/pinnedcommunities/loader.go | Implements loading/decoding pinned payloads from embedded FS or a directory. |
| protocol/pinnedcommunities/loader_test.go | Tests directory loading behavior and validates embedded payloads decode as importable community descriptions. |
| protocol/messenger.go | Starts pinned bootstrap asynchronously during Messenger startup. |
| protocol/messenger_pinned_communities.go | Implements bootstrap logic: load payloads, import via community description handler, and trigger store-node refresh. |
| protocol/messenger_pinned_communities_integration_test.go | Integration coverage for import + “first-run only” semantics. |
| protocol/messenger_communities.go | Adds optional env-controlled dumping of community raw payloads to *.rawpayload.hex files. |
| protocol/communities/persistence.go | Adds a DB count query helper for communities table. |
| protocol/communities/manager.go | Exposes Count() and hardens token metadata fetch against nil services/data (plus initializes token supply). |
| pinned-communities/embedded.go | Declares the embedded FS (//go:embed *.rawpayload.hex). |
| pinned-communities/0x02b5bdaf5a25fcfe2ee14c501fab1836b8de57f61621080c3d52073d16de0d98d6.rawpayload.hex | Adds the embedded pinned community payload blob. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
Author
|
I just realized that |
jrainville
force-pushed
the
feat/pin-status-community
branch
2 times, most recently
from
fd69b75 to
3e6ddaf
Compare
Member
Author
|
Good news! I found the root cause of the slow saving. Thankfully, it's not a DB freeze. It's because we fetched and saved the token metadata in the same thread in a blocking (doing network calls). Now, it is done in another thread and is safe. The pinned community import now takes less than a second and is show as soon as you open the portal! See the updated video in the description |
jrainville
force-pushed
the
feat/pin-status-community
branch
2 times, most recently
from
91d1ba5 to
87dd8aa
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #7562 +/- ##
===========================================
- Coverage 62.17% 62.13% -0.05%
===========================================
Files 855 857 +2
Lines 119159 119415 +256
===========================================
+ Hits 74091 74200 +109
- Misses 37495 37639 +144
- Partials 7573 7576 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
|
caybro
approved these changes
Jun 22, 2026
jrainville
force-pushed
the
feat/pin-status-community
branch
5 times, most recently
from
0b820e3 to
20870ba
Compare
Fixes status-im/status-app#20954 This commit adds a production-ready pinned community bootstrap flow so the app can show preloaded communities immediately on first launch, even before network fetches complete. Pinned payloads are now bundled into the binary and used by default, with an optional directory override for local development. The bootstrap is guarded to run only for new profiles, preserving existing data for returning users. During startup, pinned community payloads are imported through the existing community description handling path, then a non-blocking store-node refresh is triggered per imported community so fresher metadata replaces seeded data shortly after launch. The bootstrap runs asynchronously to avoid delaying app start. The import flow now also publishes imported communities back to the app through the standard messenger response signal path, so UI state can update immediately instead of waiting for a separate fetch cycle. Tests were added and updated to validate embedded payload loading, database import behavior, and first-run-only semantics, and documentation was updated to describe source precedence and runtime behavior.
Part of status-im/status-app#20954 I found that saving a big community, like the Status app, for the first time is very slow, averaging 35 seconds. Turns out the reason is because of the community tokens. The community description handling called the token data fetching in the same thread and in a blocking way. Meaning that nothing was returned until the tokens were fetched. It also means that if the token fetching failed, the whole community saving and signaling would fail. I moved the token handling in another thread, so that it doesn't slow the main operation. Then, once the fetching and saving is done, we signal again the new community data
jrainville
force-pushed
the
feat/pin-status-community
branch
from
20870ba to
cc22be0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes status-im/status-app#20954
status-app PR: status-im/status-app#21349
This commit adds a production-ready pinned community bootstrap flow so the app can show preloaded communities immediately on first launch, even before network fetches complete. Pinned payloads are now bundled into the binary and used by default, with an optional directory override for local development. The bootstrap is guarded to run only for new profiles, preserving existing data for returning users.
During startup, pinned community payloads are imported through the existing community description handling path, then a non-blocking store-node refresh is triggered per imported community so fresher metadata replaces seeded data shortly after launch. The bootstrap runs asynchronously to avoid delaying app start.
The import flow now also publishes imported communities back to the app through the standard messenger response signal path, so UI state can update immediately instead of waiting for a separate fetch cycle. Tests were added and updated to validate embedded payload loading, database import behavior, and first-run-only semantics, and documentation was updated to describe source precedence and runtime behavior.
For the demo video below, I disabled the normal community fetch that happens when getting the curated communities. This means that only the pinned community mechanism was in use. This simulates store nodes not working.
pinned-community.webm
As you saw in the video, importing the community is quite slow. It takes 35 seconds for me.
So while this work helps in case of store node issues, the Status community is now so big (5MB), that storing it in the DB takes a very long time.
I'll open a new issue to try and find the pain points and hopefully solve them.