Skip to content

Clean up retained versioned-cache namespaces without racing live workers #75

Description

@Ethan-Arrowood

Problem

VersionedCacheHandler (shipped in 2.4.0, #72) writes each production build's incremental cache to <cacheDirectory>/<app-path-hash>/<build-artifact-hash>/. The default cacheDirectory is <harper-root>/.nextjs-cache. Nothing ever removes an old namespace. Each deploy of an opted-in app leaves one behind: its render and fetch entries, plus the ownership markers under entries/{fetch,render}/. Disk use grows with every deploy and never shrinks.

Today the README asks operators to remove unused namespaces while their workers are stopped. No tooling helps with that, and an operator has no easy way to tell which namespaces a running worker still uses.

Why not a timed sweep

I dropped a timed sweep from #72 during review, because it can't know which workers are alive:

  • It can delete a namespace that a still-running outgoing worker writes to.
  • Deleting a namespace's ownership markers lets a later cache miss fall back to an older build seed after tag invalidation. That breaks the invariant recorded in src/DESIGN.md.
  • A liveness check just before deleting doesn't help, because it can't rule out a worker that starts at the same moment.

What a fix needs

  • A per-namespace lease or liveness record. It should be written in bindVersionedCache before the worker serves, and kept current for the worker's lifetime, so a sweep can prove no live worker holds the namespace.
  • A sweep that removes only namespaces with no live holder, for apps the plugin knows about. It must never remove the current build's namespace. Today HARPER_NEXTJS_SWEEP_OLD_BUILDS covers static builds only.
  • Possibly a signal from Harper for when an outgoing worker has actually stopped after deploy_component replaces the app. deploy_component: old-version workers write into the new release after the swap harper#3031 has the swap timing.
  • Tests: a namespace held by a live worker survives a sweep, an abandoned one is removed, and the current build's namespace is never touched, including across a restart.

Refs #72, HarperFast/harper#3031.

🤖 Generated by Claude Opus 5.5; posted via @Ethan-Arrowood.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    P2

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions