refactor(database,app): unify INDEX/MEMORY/SKILL on Resource + Entry backbone

[sairin1202]

Summary

Consolidates index, memory, and skill onto a single Resource + Entry, lane-tagged backbone (ADR 0006) and wires the previously model-only index and skill lanes into the full memorize/retrieve pipeline. Each lane now runs the same code path but with its own entry types and grouping strategy, instead of all sharing the memory entry set.

• Model: lane-tagged Resource / Entry / ResourceEntry replace the former MemoryCategory / MemoryItem / CategoryItem. Lane = source/index/memory/skill; RETRIEVAL_LANES = index/memory/skill. Migrated across all three backends (inmemory, sqlite, postgres).
• Per-lane entry types: index = description, memory = profile/event/knowledge, skill = tool/log (new LANE_ENTRY_TYPES table + prompt modules).
• Config: clean break to per-lane LaneConfig (enabled / grouping / entry_types / summary settings) under MemorizeConfig.lanes; index/memory/skill enabled by default.
• Memorize: loops over enabled lanes; supports per_resource grouping (index, 1:1 description docs) and adaptive grouping (memory/skill, LLM-grouped + summarized docs).
• Retrieve: single-pass per-lane recall for both rag and llm methods, returning {lanes: {index, memory, skill}, resources} with backward-compatible top-level categories/items mirroring the memory lane.
• Docs: ADR 0006 marked Accepted and rewritten for three lanes; architecture.md updated.

Test plan

• ruff check src tests — clean
• mypy src/memu — clean (150 files)
• Full non-network suite: pytest (126 passed, 1 skipped)
• Per-lane persistence conformance (index per_resource + skill adaptive) across inmemory/sqlite
• New per-lane RAG recall conformance test (test_rag_recall_lanes_returns_per_lane_shape)
• Postgres backend tests (require a live DB; not run locally — change is backend-agnostic, lane is a string column)

[Copilot]

Pull request overview

This PR refactors memU’s storage and app layers to a unified lane-based backbone, replacing the historical MemoryCategory/MemoryItem/CategoryItem model with Resource + Entry + ResourceEntry across all backends and workflows, and aligning the markdown exporter + docs with that architecture (ADR 0006).

Changes:

• Introduces lane-tagged Resource (raw + generated docs) and lane-tagged Entry (searchable atoms), linked via ResourceEntry membership edges.
• Updates SQLite/Postgres/InMemory repositories and protocols to use the new backbone (incl. lane-filtered list/clear/vector-search APIs).
• Removes legacy prompt blocks and prunes unused/obsolete code paths; updates tests and documentation to match the new data model.

Reviewed changes

Copilot reviewed 68 out of 68 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/test_tool_memory.py	Updates tool-memory tests to use Entry/entry_kind instead of MemoryItem/memory_type.
tests/test_sqlite.py	Updates debug printing to the new response field names (title, entry_kind, text).
tests/test_openrouter.py	Updates debug printing to the new response field names (title, entry_kind, text).
tests/test_memory_fs_synthesis.py	Updates synthesis wiring/tests to new Resource.summary + Entry.text model and lane="source".
tests/test_memory_files.py	Updates memory-fs seeding/export assertions to new repos (resource_repo, entry_repo, resource_entry_repo) and slugs.
tests/test_folder_memorize.py	Updates folder memorize tests to create/link Entry and ResourceEntry instead of item/category relations.
tests/test_backend_conformance.py	Updates cross-backend conformance tests to validate Entry + ResourceEntry behavior and lane-scoped clears.
src/memu/utils/tool.py	Updates tool-memory helpers to operate on Entry (entry_kind == "tool").
src/memu/utils/references.py	Removes legacy fetch_referenced_items helper.
src/memu/prompts/memory_type/skill.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/prompts/memory_type/profile.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/prompts/memory_type/knowledge.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/prompts/memory_type/event.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/prompts/memory_type/behavior.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/prompts/category_summary/category.py	Removes PROMPT_LEGACY, keeps block-based prompt format.
src/memu/memory_fs/exporter.py	Refactors exporter to read memory docs from Resource(lane="memory") and entries from EntryRepo.
src/memu/database/state.py	Updates shared DB state cache fields to entries + relations only (no categories/items).
src/memu/database/sqlite/sqlite.py	Rewires SQLite store to ResourceRepo + EntryRepo + ResourceEntryRepo.
src/memu/database/sqlite/schema.py	Updates SQLite schema model container to Resource + Entry + ResourceEntry.
src/memu/database/sqlite/repositories/resource_repo.py	Expands resource table to include lane/doc fields and adds doc helpers + lane filtering.
src/memu/database/sqlite/repositories/resource_entry_repo.py	Adds SQLite membership-edge repository implementation.
src/memu/database/sqlite/repositories/memory_category_repo.py	Removes obsolete SQLite category repository.
src/memu/database/sqlite/repositories/entry_repo.py	Adds SQLite entry repository implementation (incl. ref_id lookup and lane-filtered vector search).
src/memu/database/sqlite/repositories/category_item_repo.py	Removes obsolete SQLite category-item relation repository.
src/memu/database/sqlite/repositories/base.py	Replaces in-memory “matches_where” helpers with lane-filter helpers for DB queries.
src/memu/database/sqlite/repositories/init.py	Updates SQLite repository exports for new repos.
src/memu/database/sqlite/models.py	Replaces category/item models with Entry + ResourceEntry and expands Resource columns.
src/memu/database/sqlite/init.py	Builds scoped SQLite SQLAlchemy models and wires them into SQLiteStore.
src/memu/database/repositories/resource.py	Updates ResourceRepo protocol with lane filtering and doc helpers (get_or_create_doc, update_resource).
src/memu/database/repositories/resource_entry.py	Adds protocol for entry↔resource membership edges.
src/memu/database/repositories/memory_item.py	Removes obsolete MemoryItemRepo protocol.
src/memu/database/repositories/memory_category.py	Removes obsolete MemoryCategoryRepo protocol.
src/memu/database/repositories/entry.py	Adds EntryRepo protocol (lane filtering + ref_id + vector search).
src/memu/database/repositories/category_item.py	Removes obsolete CategoryItemRepo protocol.
src/memu/database/repositories/init.py	Updates repo exports to Entry/ResourceEntry/Resource.
src/memu/database/postgres/schema.py	Updates Postgres schema model container to Resource + Entry + ResourceEntry.
src/memu/database/postgres/repositories/resource_repo.py	Updates Postgres ResourceRepo implementation for lanes/docs and lane-filtered vector search.
src/memu/database/postgres/repositories/resource_entry_repo.py	Adds Postgres membership-edge repository implementation.
src/memu/database/postgres/repositories/memory_item_repo.py	Removes obsolete Postgres MemoryItem repo.
src/memu/database/postgres/repositories/memory_category_repo.py	Removes obsolete Postgres MemoryCategory repo.
src/memu/database/postgres/repositories/entry_repo.py	Adds Postgres EntryRepo implementation (pgvector + local fallback + salience).
src/memu/database/postgres/repositories/category_item_repo.py	Removes obsolete Postgres category-item repo.
src/memu/database/postgres/repositories/base.py	Removes unused merge helpers and legacy in-memory where matching.
src/memu/database/postgres/repositories/init.py	Updates Postgres repository exports for new repos.
src/memu/database/postgres/postgres.py	Rewires Postgres store to ResourceRepo + EntryRepo + ResourceEntryRepo.
src/memu/database/postgres/models.py	Replaces legacy category/item SQLModels with PostgresEntryModel + PostgresResourceEntryModel.
src/memu/database/postgres/init.py	Updates Postgres DB builder wiring for new models.
src/memu/database/models.py	Introduces Lane, expands Resource fields, adds Entry + ResourceEntry, removes legacy models.
src/memu/database/interfaces.py	Updates DB protocol to expose entry_repo + resource_entry_repo and caches.
src/memu/database/inmemory/repositories/resource_repo.py	Updates in-memory resource repo for lane/docs and adds get_resource + lane filtering.
src/memu/database/inmemory/repositories/resource_entry_repo.py	Adds in-memory membership-edge repository implementation.
src/memu/database/inmemory/repositories/memory_item_repo.py	Removes obsolete in-memory MemoryItem repo.
src/memu/database/inmemory/repositories/memory_category_repo.py	Removes obsolete in-memory MemoryCategory repo.
src/memu/database/inmemory/repositories/entry_repo.py	Adds in-memory EntryRepo implementation (incl. salience and ref_id lookup).
src/memu/database/inmemory/repositories/category_item_repo.py	Removes obsolete in-memory CategoryItem repo.
src/memu/database/inmemory/repositories/init.py	Updates in-memory repository exports for new repos.
src/memu/database/inmemory/repo.py	Rewires in-memory store to ResourceRepo + EntryRepo + ResourceEntryRepo.
src/memu/database/inmemory/models.py	Updates scoped in-memory models to Entry + ResourceEntry.
src/memu/database/inmemory/init.py	Updates in-memory DB builder wiring for new models.
src/memu/database/init.py	Updates public exports to Entry/ResourceEntry/Resource repositories and records.
src/memu/app/service.py	Removes legacy category-init plumbing and old provider summary helper.
src/memu/app/memory_files.py	Ensures memory-fs builder lists only lane="source" resources for raw file descriptions.
src/memu/app/crud.py	Updates CRUD pipeline to list/clear/update entries and memory-doc resources; uses ResourceEntryRepo for links.
docs/architecture.md	Updates architecture docs to describe the lane-based backbone and new repository contracts.
docs/adr/0006-unified-resource-entry-lane-backbone.md	Adds ADR describing the unified Resource+Entry lane model.

Comments suppressed due to low confidence (1)

src/memu/utils/tool.py:60

• Docstring arg description still says "MemoryItem" after the refactor to Entry, which is misleading for callers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.