feat(memory_fs): optional markdown memory file system export#435
Open
sairin1202 wants to merge 2 commits into
Open
feat(memory_fs): optional markdown memory file system export#435sairin1202 wants to merge 2 commits into
sairin1202 wants to merge 2 commits into
Conversation
Add a read-only, opt-in artifact layer that renders the structured memory store into browsable markdown files (MEMORY.md, index.md, skill.md, and categories/<slug>.md), treating the memory model as an actual file system. - New `memu.memory_fs.MemoryFileExporter` walks folders (categories), files (items), and sources (resources) and renders them to disk. - Diff detection via a sidecar `.memufs_manifest.json` of per-file content hashes, so each export only rewrites changed artifacts — no DB schema change. - `MemoryService.export_memory_files(user=...)` entrypoint, gated by `memory_files_config.enabled` (default off), serialized via a per-service lock. - Rendered content avoids volatile values so an unchanged store re-exports as a no-op; stale artifacts are pruned. Fully additive: no change to memorize/retrieve/CRUD behavior or public API. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an opt-in, read-only “memory file system” export path to render the structured memory store (categories/resources/items) into a browsable set of markdown artifacts on disk, wired through MemoryService and documented in the architecture guide.
Changes:
- Introduces
memu.memory_fs.MemoryFileExporterwith manifest-based diffing/pruning to writeMEMORY.md,index.md,categories/<slug>.md, andskill.md. - Adds
MemoryFilesConfigand a newMemoryService.export_memory_files(...)entrypoint gated bymemory_files_config.enabledand serialized via anasyncio.Lock. - Adds a focused test suite covering contents, idempotency, pruning, scoping, disabled guard, and manifest round-trips.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_memory_files.py |
Adds tests validating artifact contents, idempotency, pruning, scope filtering, disabled behavior, and manifest handling. |
src/memu/memory_fs/exporter.py |
Implements the exporter, markdown rendering, and manifest-based sync/prune logic. |
src/memu/memory_fs/__init__.py |
Exposes MemoryFileExporter, ExportResult, and slugify as the package API. |
src/memu/app/settings.py |
Introduces MemoryFilesConfig (enabled flag + output directory). |
src/memu/app/service.py |
Wires config + exporter and adds export_memory_files() to MemoryService. |
docs/architecture.md |
Documents the new memory file system export feature and its operational characteristics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+138
to
+142
| def _item_counts_per_category(items: list[MemoryItem], *, database: Database) -> dict[str, int]: | ||
| in_scope_ids = {item.id for item in items} | ||
| counts: dict[str, int] = {} | ||
| relations = getattr(database, "relations", None) or [] | ||
| for relation in relations: |
Comment on lines
+151
to
+154
| "---", | ||
| f"name: {category.name}", | ||
| f"description: {self._inline(description)}", | ||
| f"updated_at: {category.updated_at.isoformat()}", |
Comment on lines
+231
to
+237
| for rel_path in manifest: | ||
| if rel_path in new_manifest: | ||
| continue | ||
| stale = self.output_dir / rel_path | ||
| if stale.exists(): | ||
| stale.unlink() | ||
| result.removed.append(rel_path) |
Comment on lines
+176
to
+180
| slug = slug_by_category[category.id] | ||
| description = self._inline((category.description or "").strip()) | ||
| count = item_counts.get(category.id, 0) | ||
| suffix = f" — {description}" if description else "" | ||
| lines.append(f"- [{category.name}]({CATEGORIES_DIRNAME}/{slug}.md){suffix} ({count} items)") |
Rework the exporter so output matches the canonical memory tree (INDEX.md / MEMORY.md / skill/<name>/SKILL.md) and the description-trunk model: - Every source becomes one multimodal description (the shared trunk); INDEX, MEMORY, and SKILL are three sibling bypasses (none upstream of another). - INDEX.md: navigable table of contents (folders, skills, per-source descriptions) that links out instead of duplicating summaries. - MEMORY.md: living memory aggregated from category (folder) summaries. - skill/<skill_name>/SKILL.md: each skill-type item as a standalone doc; folder name parsed from frontmatter `name:` (heading / short-id fallbacks). - Prune now-empty skill dirs when a skill is removed; default output dir -> ./data/memory. Still read-only, diff-driven via the sidecar manifest, and disabled by default. Co-authored-by: Cursor <cursoragent@cursor.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a read-only, opt-in artifact layer that renders the structured memory store into browsable markdown files, treating the memory model as an actual file system. This is the first, fully-additive foundation step toward the file-system-as-memory direction (it does not change any
memorize/retrieve/CRUD behavior or public API).Generated artifacts:
MEMORY.md— top-level overview of all folders (categories)index.md— source-file description index (one entry perResource)categories/<slug>.md— per-folder summary file (e.g.preferences.md)skill.md— standalone aggregation of skill-type memoriesHow it works
memu.memory_fs.MemoryFileExporterwalks folders (MemoryCategory), files (MemoryItem), and sources (Resource) and renders them to disk. It is read-only against the database..memufs_manifest.jsonof per-file content hashes — each export only rewrites artifacts whose rendered content changed, and prunes stale ones. No database schema change required.MemoryService.export_memory_files(user=...), gated bymemory_files_config.enabled(default off), with scope filtering via the user model and writes serialized through a per-serviceasyncio.Lock.Design notes / deferred decisions
This PR deliberately keeps scope tight. The following were fixed with conservative defaults and can be iterated later:
MemoryItemextraction/synthesis is unchanged (skill.md aggregates existing skill items rather than replacing the item pipeline).Resource(avoids cross-backend schema churn).Test plan
uv run python -m pytest tests/test_memory_files.py -v(6 passed): artifact contents, diff idempotency, stale pruning, user-scope isolation, disabled-guard, manifest round-trip85 passed, 1 skipped(skip = postgres;test_lazyllmexcluded — optional dep not installed, pre-existing)uv run ruff checkanduv run mypyclean on changed filesMade with Cursor