Skip to content

refactor: checkpoint API cleanup#5318

Merged
greysonlalonde merged 5 commits into
mainfrom
refactor/checkpoint-api-cleanup
Apr 7, 2026
Merged

refactor: checkpoint API cleanup#5318
greysonlalonde merged 5 commits into
mainfrom
refactor/checkpoint-api-cleanup

Conversation

@greysonlalonde

@greysonlalonde greysonlalonde commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Rename CheckpointConfig.directory to location — the field means a directory for JsonProvider and a database file path for SqliteProvider, so the generic name fits both
  • Add prune method to BaseProvider protocol so each provider owns its own cleanup logic
  • Move pruning out of checkpoint_listener into the providers where it belongs
  • max_checkpoints stays on CheckpointConfig as the single place users configure it; the listener passes it through to provider.prune after each write

Test plan

  • Existing checkpoint tests pass
  • JsonProvider prunes files when max_checkpoints is set
  • SqliteProvider prunes rows when max_checkpoints is set
  • CheckpointConfig(location=..., max_checkpoints=5) works for both providers

Note

Medium Risk
Medium risk because it renames a public CheckpointConfig field and changes pruning responsibilities, which could break existing integrations or custom providers if not updated.

Overview
Checkpointing API cleanup. Renames CheckpointConfig.directory to location across code and docs to reflect that the value may be either a filesystem directory (JsonProvider) or a SQLite DB file path (SqliteProvider).

Provider-owned retention. Adds prune(location, max_keep) to the BaseProvider protocol and removes file-pruning logic from checkpoint_listener, delegating retention to JsonProvider.prune (delete old JSON files) and SqliteProvider.prune (delete old rows), with tests updated accordingly.

Reviewed by Cursor Bugbot for commit c61a40e. Bugbot is set up for automated code reviews on this repo. Configure here.

…ders

- Rename CheckpointConfig.directory to location — the field means a
  directory for JsonProvider and a database file path for SqliteProvider,
  so the generic name fits both
- Add max_checkpoints to BaseProvider protocol
- Add max_checkpoints and pruning to JsonProvider, matching SqliteProvider
- Remove prune logic from checkpoint_listener since providers own cleanup
@github-actions github-actions Bot added the size/M label Apr 7, 2026

@iris-clawd iris-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Checkpoint API Cleanup

Clean refactor — directorylocation is the right abstraction, and moving prune logic into providers is good separation of concerns.

🟡 Prune is now a separate transaction (SqliteProvider)

Previously, _PRUNE ran inside the same sqlite3.connect() context as the insert (same transaction). Now prune() opens a new connection. Two implications:

  1. If the process crashes between checkpoint() and prune(), you accumulate an extra row. Minor — alpha-level acceptable.
  2. More importantly: two separate connections means two separate transactions. Under concurrent writes (e.g., multiple crews checkpointing to the same DB), there's a window where a prune could race with an insert from another thread. WAL mode helps, but the old atomic approach was safer.

If this matters, prune could accept an optional connection/cursor to reuse the write transaction. Not blocking for alpha.

🟡 No aprune on BaseProvider

BaseProvider defines both checkpoint and acheckpoint, but prune is sync-only. The listener always calls it synchronously via _do_checkpoint, so it works today. But if someone calls acheckpointprune in an async context, they'll block the event loop on SQLite I/O. Worth adding aprune to the protocol for symmetry, even if the default impl just calls prune.

🟢 Looks good

  • Breaking changes are fine for alpha (CheckpointConfig.directorylocation, SqliteProvider.__init__ dropping max_checkpoints)
  • JsonProvider prune logic is a clean lift from the old _prune function
  • SqliteProvider prune reuses the existing _PRUNE SQL correctly
  • Docstrings updated consistently across all files

Minor stuff, nothing blocking. 💬 163

Comment thread lib/crewai/src/crewai/state/provider/core.py Dismissed

@iris-clawd iris-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — already reviewed. Notes on separate prune transaction and missing aprune are suggestions for later. Good to land. ✅

@iris-clawd iris-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after new commits. ✅

@greysonlalonde greysonlalonde merged commit 5958a16 into main Apr 7, 2026
50 of 51 checks passed
@greysonlalonde greysonlalonde deleted the refactor/checkpoint-api-cleanup branch April 7, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants