Repository navigation
tracking(storage): task storage visibility, reclamation and scoped cleanup #5825
Copy link
Copy link
Closed
Description
Step 1 of the task lifecycle plan agreed in #5776. It makes local storage visible, reclaims space that deletion already frees logically, and lets users clean up archived tasks by scope. It adds no automatic archiving or deletion; those are later, opt-in steps tracked in #5776. The Host computes and executes; Desktop only presents.
Baseline: main@0323714.
Current state
- Nothing reports how much space Maka uses, or which tasks use it.
runtime.sqlitenever shrinks.- It is WAL with
auto_vacuum=NONE(sqlite-runtime-schema.ts). - Nothing calls
wal_checkpoint, setsjournal_size_limit, or runsVACUUM. - Deleted sessions leave free pages that SQLite reuses but never returns to the OS.
- It is WAL with
context-offload.sqliteis set up for reclamation but never reclaims.- It is created with
auto_vacuum=INCREMENTAL(sqlite-context-offload-schema.ts). - No code runs
incremental_vacuum.
- It is created with
- Sessions have no archive time.
session_metadataandSessionHeaderhave no archive time, andcommitted_atis overwritten on every metadata update, so it can't serve as one. - "Delete all archived" runs in the renderer.
- It calls
session.removeonce per session, serially (session-row-actions.ts). - Its confirmation shows a count, not a preview of what will be removed.
- It calls
- fix(desktop): managed artifact previews outlive deletion and share one global quota #5341 is an in-memory preview lease that outlives deletion, not disk residue. fix(desktop): invalidate artifact previews after deletion #5394 fixes it at the Host retirement hook, and this issue depends on that fix rather than redoing it.
Plan: three PRs, each usable on its own
M1: storage visibility (read-only)
- New Host query
storage.usage.query, query mode, Ready only.- Totals by kind:
database,transcript,artifacts,context_offload,worktrees(count only),memory,usage_historyandother. - The kind set is open for extension, for example the checkpoint store in feat(runtime-host): publish verified local session checkpoints #5605.
- It also reports reclaimable free pages (
freelist_count * page_size).
- Totals by kind:
- Per-task sizes are computed on demand for a bounded list of session IDs (the visible page) and cached in memory.
- There are no triggers or write-path counters, so nothing adds to the write amplification tracking(perf): bound storage and protocol read/write amplification #5038 is bounding.
- Context-offload sizes per session are logical (blobs are deduplicated), and the UI says so.
- Desktop presents the results.
- The Data page gets a "Storage" group with a breakdown and the reclaimable space.
- The Tasks page shows a size on each row.
- Epoch bump.
M2: reclamation
- A bounded
incremental_vacuumlane forcontext-offload.sqlitein the existingHostStorageMaintenanceloop. runtime.sqliteconversion toauto_vacuum=INCREMENTAL. It needs one fullVACUUM, which needs roughly 2x the database size in free disk, so it runs only on explicit user request ("Compact database"). The Host persists the request and runs it at the next start, before Ready, next to migrations, after a free-disk check.- After conversion, bounded incremental vacuum plus a passive checkpoint run in the maintenance loop.
- The Data page reports what was actually freed.
- Epoch bump.
M3: scoped cleanup
- Durable
archived_at.- It is set when a session is archived and cleared when it is restored.
- It is projected into the session catalog.
- Sessions archived before the field exists have
NULL, show "archive time unknown", and are excluded from age filters.
session.remove.previewtakes a list of sessions and reports tasks, child tasks, bytes and worktrees that would be removed.- The Tasks page adds filters for "archived more than N days" and project. The confirmation lists the preview. Execution keeps using
session.remove. - Epoch bump.
Questions for maintainers
- Is the explicit-request, pre-Ready conversion of
runtime.sqliteacceptable? The alternative is to only report the reclaimable space and never convert. - Should
archived_atland as a small standalone PR first, since step 2 (retention) reuses it? - Should bulk delete become a Host-side batch command (possibly together with fix(storage): batch retired Session artifact cleanup #4984), or stay a renderer loop over
session.remove?
Risks
- Protocol compatibility. Each milestone bumps the compatibility epoch.
- Overlapping changes. It touches the same files as fix(desktop): invalidate artifact previews after deletion #5394 and fix(storage): batch retired Session artifact cleanup #4984 (retirement coordinator, artifact store).
- Blocking the Host.
node:sqliteis synchronous, so vacuum steps must stay small to avoid blocking the Host event loop. - Retained usage history. It is deliberately kept when sessions are deleted (
usage_model_call_attempts), so the database may stay larger than users expect. The UI should call this out.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
garvit-arora commentedon Sep 29, 2026
take
garvit-arora commentedon Sep 29, 2026
Plan (M2 reclamation): add bounded incremental_vacuum lane for context-offload.sqlite in HostStorageMaintenance; support explicit user-requested runtime.sqlite INCREMENTAL conversion + compact at next Host start (pre-Ready, after free-disk check); passive wal_checkpoint in maintenance loop; wire Data settings to show space reclaimed. Branch: feat/5825-m2-storage-reclamation. M1 visibility (#5832) stays separate.
liugddx commentedon Sep 30, 2026
Status update:
context-offload.sqlitelane and the report first, and splittingruntime.sqlitecompaction out (review on feat(storage): reclaim context-offload SQLite free pages in Host maintenance #5855).archived_at+ scoped cleanup) stays with me. This issue remains the tracker for all three, so PRs should sayPart of #5825rather thanFixes.Open question 1 now blocks the rest of M2: where should a full
VACUUMofruntime.sqliterun? #5855 has measured both placements:The options I see:
A maintainer call here would unblock the follow-up.
feat(sessions): record when a task was archived (#5884)
liugddx commentedon Oct 2, 2026
Status update:
archived_at) and feat(desktop): clean up archived tasks by age and project #5896 (age and project filters, plus a Host-side batch removal preview and an age guard).context-offload.sqliteincremental-vacuum lane. Recent reviews found no correctness blockers; it's waiting on a PR description update.One open item. Converting
runtime.sqliteto incremental vacuum still has no decision on where its one fullVACUUMshould run:Once #5855 lands I'll close this tracker, and move the
runtime.sqlitequestion to its own issue so it isn't lost.Next steps from #5776 continue elsewhere: retention is #5899 / #5902.
fix(storage): reduce context-offload reclamation to the bounded vacuum
test(storage): pin the reclamation checkpoint and idle cadence
liugddx commentedon Oct 8, 2026
Closing. All three steps have landed:
incremental_vacuumlane forcontext-offload.sqlite. @garvit-arora built the lane; I rebased it over feat(storage): opt-in retention for archived tasks #5902 and applied the last review round.archived_at) and feat(desktop): clean up archived tasks by age and project #5896 (age and project filters, Host-side batch preview).The one open item, where
runtime.sqlite's one-time fullVACUUMshould run before it can switch to incremental vacuum, now lives in #6000 with the measurements and options. Retention, the next step from #5776, landed in #5902, with notices in #5961.One note for the record: the #5855 squash commit (
c542bc757) went in subject-only, so it does not carry theGenerated-by: Claude Codetrailer that the follow-up commits had. Claude Code made a substantive contribution to that PR's final revision, as its description states.