Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ This is an adaptation of STE principles, not a requirement for full controlled-v

- Use American English, such as "behavior," "color," "initialize," and "analyze."
- Write concepts and benefits in full sentences. Use specific claims with evidence rather than vague attribution.
- Support every performance, scale, or cost benefit. Give a measured number with its conditions, such as "raises replication throughput from 2,450 to 4,900 rows per second with 1 KB rows and 100 ms S3 latency." Alternatively, derive the benefit from a mechanism the page describes and state its limits, such as faster deployments because only changed streams are reprocessed. Otherwise, describe what changed instead of calling it "faster," "more efficient," or "more scalable." A source that asserts a benefit without a measurement is not evidence for it.
- Omit promotional language, editorial filler, and statements that minimize difficulty. Avoid "breathtaking," "it's important to note," "simply," "just," "easily," and "obviously."
- Avoid title and description filler such as "Comprehensive" or "Complete," and unnecessary transitions such as "moreover" or "furthermore."
- Explain the practical meaning of "local-first" or "offline-first": apps read and write locally, respond instantly, and remain functional without a network connection.
Expand Down
2 changes: 2 additions & 0 deletions .claude/agents/document-reviewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Review added or modified content unless the user requests a full-page audit. Rea
Apply the canonical standards in three passes and report findings from each:

1. **Accuracy:** claims, platform scope, versions, and consistency with the surrounding page. Before reporting a claim as unverified, check the sources the PR or the user cites, such as the source PR, divergence issue, release notes, or code at the merged commit, and follow the links inside them. If nothing is cited, look up the release the text names. Report a claim as unverified only when no source covers it or a source contradicts it. Give evidence, do not invent problems, and do not approve unverified claims as correct.

Also list every claim that something is faster, more efficient, more scalable, cheaper, or lowers load or cost, including in frontmatter descriptions. Treat each one as a finding unless it passes the performance claim rule in [Language and Tone](../CLAUDE.md#language-and-tone). Report it even when the claim is plausible or a cited source repeats it. Warnings that something is slow or limited are caveats, not benefit claims.
2. **Necessity:** list every sentence that describes what the product prints, displays, logs, or says in an error. Treat each one as a finding to remove unless it passes the restating rule in [Content Strategy](../CLAUDE.md#content-strategy), and report it even when the sentence is accurate. Describing visible output is not a mechanism, consequence, or signal. Also flag internal mechanics, rare exceptions, and repetition. Flag missing context only when readers need it, and do not require every entry to explain a mechanism, consequence, signal, action, and trade-off.
3. **Clarity and format:** plain technical English, clear actors and actions, and suitable examples and components.
4. **Sync Rules containment:** flag any Sync Rules mention, example, or tab outside `sync/rules/` and `snippets/sync-shared/` that does not state a behavior difference. Check that split twins, wrappers, and shared snippets keep their counterpart comment.
Expand Down
6 changes: 2 additions & 4 deletions client-sdks/frameworks/expo-go-support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,11 +116,9 @@ After adding PowerSync to your app:
## Data Persistence

<Warning>
Support for persistence with `@powersync/adapter-sql-js` is deprecated. The reason is that the implementation is inefficient,
doesn't provide consistency guarantees expected from SQLite, and writes are skipped as long as the database is connected.
Support for persistence with `@powersync/adapter-sql-js` is deprecated. The reason is that the implementation is inefficient, doesn't provide consistency guarantees expected from SQLite, and writes are skipped as long as the database is connected.

If you need persisted data across app restarts, use the [recommended setup](/client-sdks/reference/react-native-and-expo#installation)
with a local app build instead of Expo Go.
If you need persisted data across app restarts, use the [recommended setup](/client-sdks/reference/react-native-and-expo#installation) with a local app build instead of Expo Go.
</Warning>

The default version of this adapter uses in-memory persistence, but you can specify your own `persister` option to the open factory. See an example in the package [README](https://www.npmjs.com/package/@powersync/adapter-sql-js).
Expand Down
2 changes: 1 addition & 1 deletion maintenance-ops/self-hosting/deployment-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Set the environment variable `NODE_OPTIONS=--max-old-space-size-percentage=80` t

The API container handles streaming sync connections, as well as any other API calls.

The replication process is run using the docker command `start -r api`, for example `docker run powersync start -r api`.
The API process is run using the docker command `start -r api`, for example `docker run powersync start -r api`.

Each API container is limited to 200 concurrent connections, but we recommend targeting 100 concurrent connections or less per container. This may change as we implement additional performance optimizations.

Expand Down
9 changes: 3 additions & 6 deletions sync/advanced/storage-version-4.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
---
title: "Storage Version 4"
description: "Storage version 4 provides faster sync, incremental reprocessing of Sync Streams changes, and S3 object storage."
description: "Storage version 4 enables incremental reprocessing of Sync Streams changes and S3 object storage."
---

{/* Split page: the Sync Rules (Legacy) version of this page is sync/rules/storage-version-4.mdx. It is a short summary that links here for details, so keep the anchors opt-in, switching-storage-versions, incremental-reprocessing, s3-object-storage, and self-hosted-s3-setup stable. Mention Sync Rules here only to state that incremental reprocessing requires Sync Streams. */}

Storage version 4 is the current format the PowerSync Service uses to store the data it syncs to clients. It is the default for MongoDB bucket storage since PowerSync Service v1.27.0.

Compared to version 2, it provides:
Compared to version 2, it adds:

- Faster sync and faster reprocessing after a deployment.
- [Incremental reprocessing](#incremental-reprocessing): a Sync Streams deployment reprocesses only the streams you added or changed. Clients no longer download all their data again after every deployment.
- [S3 object storage](#s3-object-storage): larger blocks of synced data move from the storage database to S3. This reduces load on the storage database when many clients sync at once or sync large amounts of data.

Expand Down Expand Up @@ -101,7 +100,7 @@ If a deployment takes longer or reprocesses more than you expect, see [Checking

Your instance keeps the data it syncs to clients in its bucket storage database, alongside everything else it needs to run. With S3 object storage, larger blocks of that data move to Amazon S3 or an S3-compatible object store, and the PowerSync Service syncs them to clients directly from there. Smaller blocks, and the metadata that locates each block, stay in MongoDB.

Reading larger blocks from S3 reduces the data MongoDB must read and transfer during sync. When those reads limit performance, offloading them can speed up initial sync and let an instance serve more concurrent clients. The benefit is most noticeable when clients sync large amounts of data or many clients connect at once. The PowerSync Service still handles every client connection, so its CPU and memory capacity also limit concurrency.
Reading larger blocks from S3 means MongoDB reads and transfers less data during sync. This lowers the load on the storage database when clients sync large amounts of data or many clients connect at once.

For self-hosted instances, offloading bucket data to S3 can reduce storage and data transfer costs. Compare the reduction in database costs with the object store's storage, request, and data transfer charges for your workload.

Expand Down Expand Up @@ -154,8 +153,6 @@ PowerSync enables S3 object storage per instance on request. We plan to enable i
</Step>
</Steps>

After enabling S3 object storage, you can raise [`max_concurrent_connections`](/configuration/powersync-service/self-hosted-instances#param-max-concurrent-connections) from its default of 200 per API process. With storage version 4 and S3 object storage, each API process can handle up to 1,000 concurrent client connections. Performance degrades if a large share of those clients run an initial sync at the same time, so scale out the API before a deployment that makes all clients download their data again. More concurrent connections also increase CPU and memory usage.

The [S3 object storage configuration reference](/configuration/powersync-service/self-hosted-instances#param-object-storage) lists all supported settings, including timeouts, request concurrency, and the size threshold below which blocks stay in MongoDB.

Keep the scheduled [compact](/maintenance-ops/compacting-buckets) job running. It removes files that are no longer needed. The `teardown` command deletes PowerSync's files under the prefix before it drops the storage database. The `powersync_object_storage_size_bytes` [metric](/maintenance-ops/self-hosting/monitoring) reports how much object storage PowerSync uses.
5 changes: 1 addition & 4 deletions sync/rules/storage-version-4.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,7 @@ description: "What storage version 4 and S3 object storage mean for instances th

Storage version 4 is the format the PowerSync Service uses to store the data it syncs to clients. It is the default for MongoDB bucket storage since PowerSync Service v1.27.0.

With Sync Rules, storage version 4 provides:

- Faster sync and faster reprocessing after a deployment.
- [S3 object storage](/sync/advanced/storage-version-4#s3-object-storage): larger blocks of synced data move from the storage database to S3. This reduces load on the storage database when many clients sync at once or sync large amounts of data. On PowerSync Cloud, PowerSync enables it per instance on request, and plans to enable it on all instances automatically. Self-hosted instances follow the [S3 setup instructions](/sync/advanced/storage-version-4#self-hosted-s3-setup).
With Sync Rules, storage version 4 enables [S3 object storage](/sync/advanced/storage-version-4#s3-object-storage): larger blocks of synced data move from the storage database to S3. This reduces load on the storage database when many clients sync at once or sync large amounts of data. On PowerSync Cloud, PowerSync enables it per instance on request, and plans to enable it on all instances automatically. Self-hosted instances follow the [S3 setup instructions](/sync/advanced/storage-version-4#self-hosted-s3-setup).

[Incremental reprocessing](/sync/advanced/storage-version-4#incremental-reprocessing) requires Sync Streams. With Sync Rules, every deployment still reprocesses all data, and clients download all their data again. To use incremental reprocessing, [migrate to Sync Streams](/sync/rules/migrate-to-sync-streams).

Expand Down
Loading