Skip to content

PC-015 — Architectural: drop Redis in favor of SQLite + in-process state #57

@itigges22

Description

@itigges22

Status (at mirror time): Open (tracking) • Severity: N/A (refactor)

One-liner: Drop Redis in favor of SQLite + in-process state. Deferred refactor


Cross-ref: this fits the Phase 0 setup-simplicity theme below
(atlas-bootstrap, prebuilt images, doctor). If Phase 0 lands first
Redis goes from "annoying setup step" to "annoying daemon to remove
later"; doing PC-015 before Phase 0 means Phase 0 has one fewer thing
to install.

  • Status: Open (tracking only — no implementation in this sprint)
  • Severity: N/A (refactor for setup simplicity, not a bug)
  • Location: geometric-lens/cache/, geometric-lens/router/,
    geometric-lens/main.py (task queue + metrics)

Redis is currently load-bearing for the Pattern Cache (sorted sets,
JSON blobs, hit/miss counters), the co-occurrence graph, the router's
Thompson sampling state, the task queue, and metrics. Audit shows the
actual Redis surface is purely key-value + sorted sets + lists — no
pub/sub, no streams, no Lua, no clustering. Every command in use maps
cleanly to SQLite (FTS5 even gives BM25 natively if we ever want to
retire the in-memory PatternMatcher).

Costs Redis adds today:

  • An extra service in docker-compose.yml with healthcheck and
    network dep.
  • A host kernel prereq (PC-011 / vm.overcommit_memory=1).
  • AOF persistence config and tuning surface.
  • A synchronous network hop on every cache op.
  • Version skew between client lib and server.

What porting would touch:

  • cache/pattern_store.py → SQLite tables: patterns(id, json, tier, score, last_modified) with indexes on (tier, score DESC).
  • cache/co_occurrence.pycooccur_edges(from_id, to_id, count).
  • router/feedback_recorder.py + router/route_selector.py → either
    a router_state table or a JSON file with file-lock (state is
    small).
  • main.py task queue + metrics → either drop (consumer audit needed
    per PC-014) or move to SQLite + JSONL.

Estimated 4 files, ~200-400 LOC. Not in scope for the current sprint;
keep Redis intact. Revisit once the cache fixes (PC-001..PC-010) have
proven out and the topology question (PC-014) is resolved.

Fix

Deliberately deferred. No implementation planned for this sprint.



Mirrored from local ISSUES.md. ISSUES.md is gitignored; this issue is the canonical public record. PC-### numbering preserved for cross-references.

Metadata

Metadata

Assignees

No one assigned

    Labels

    deferredTracked but deferred to roadmap (Phase 1+)reliabilityPC-### reliability ticket from ISSUES.md

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions