Skip to content

fix(storage): keep rel scans on the CSR state they started with - #1116

Open
kory-io wants to merge 2 commits into
LadybugDB:mainfrom
kory-io:fix/csr-scan-across-checkpoint
Open

kory-io wants to merge 2 commits into
LadybugDB:mainfrom
kory-io:fix/csr-scan-across-checkpoint

Conversation

@kory-io

@kory-io kory-io commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Description

A rel scan that is already running when CSRNodeGroup::checkpoint runs kept its cached CSR header, then read the group the checkpoint had replaced. In-memory rels disappeared. On a persistent group, newly inserted rels were missing and deleted rels came back. This is the CSR follow-up to #1094. The node-table refresh does not carry over: CSR offsets move during checkpoint, and CSRNodeGroup::checkpoint does not bump the counter that refresh checks. Fixes #1053.

The scan pins the persistent chunk metadata, version info, column updates, CSR index, and in-memory groups it started with. While a scan holds that pin, checkpoint writes replacement pages out of place and frees the old pages only when the last scan drops the pin. Checkpoint does not wait for the scan.

Release build, ENABLE_RUNTIME_CHECKS off, before the fix:

  • in-memory scan resumed after checkpoint: 2048 of 5999 rels
  • persistent scan with inserts and deletes, resumed after checkpoint and again inside the shadow-page window: 1714 rels missing and 279 deleted rels returned

After the fix those three pass, as does a scan that starts inside the shadow-page window. ScanAcrossCheckpointTest (the #1094 node scans) still passes, and CSRGroupReadsBackAfterFailedCSRHeaderCheckpoint still passes. No on-disk format change.

Types of changes

  • Bug fix
  • New feature
  • Breaking change
  • Documentation Update

Checklist

  • I have changed storage version if on disk format has changed.
  • I have requested a review from a maintainer.
  • I have updated the documentation (if needed).

A checkpoint can rewrite a CSR node group while a scan is still walking
the header, pages, and in-memory lists it started with. Pin that state
on the scan, and keep those pages until the scan finishes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: a rel scan in progress across a CHECKPOINT returns missing and deleted rels

1 participant