refactor(app): decouple embedding, remove Rust, split flows, narrow DB protocol#437
Open
sairin1202 wants to merge 2 commits into
Open
refactor(app): decouple embedding, remove Rust, split flows, narrow DB protocol#437sairin1202 wants to merge 2 commits into
sairin1202 wants to merge 2 commits into
Conversation
…B protocol - Decouple embedding from chat LLM: route all vectorization through the dedicated memu.embedding clients (separate per-profile cache), drop embed() from the sdk/httpx llm clients, and add grok/openrouter embedding backends. - Remove the unused Rust scaffolding (lib.rs/Cargo/maturin) and switch the build backend to hatchling. - Delete legacy retrieve dead code (_embedding_based_retrieve/_llm_based_retrieve and their orphaned helpers). - Narrow the Database protocol to repositories only (no raw record dicts) so scope filtering cannot be bypassed. - Split oversized flows: extract the LLM retrieve pipeline into retrieve_llm.py and the memorize parsers into memorize_parse.py. - Finish skill/ decoupling in memory_fs (always synthesized from descriptions, independent of memory-item extraction) and update docs/tests. Co-authored-by: Cursor <cursoragent@cursor.com>
… skill/ with root indexes Split the memory file system output so each root index points at a sibling payload directory: - resource/<file_name>: raw source bytes copied verbatim from Resource.local_path - memory/<slug>.md: one file per MemoryCategory (description + summary) - skill/<slug>/SKILL.md: unchanged INDEX.md now links into resource/, MEMORY.md is an overview that links to each memory/<slug>.md, and _sync/_content_hash are bytes-aware so binary raw copies diff correctly. Docs (README, architecture), settings, the example, and tests are updated to the new layout. Also folds in the in-progress branch work (embedding decoupling, folder memorize, examples/prompts cleanup). Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Architecture cleanup stacked on top of #436 (base:
feat/memory-fs-synthesis).memu.embeddingclients with a separate per-profile cache;embed()is removed from the sdk/httpx chat clients. Addedgrok+openrouterembedding backends so provider coverage is preserved (lazyllm_backendstays unified, as it has no standalone embedding client).lib.rs/Cargo.toml/Cargo.lock/tests/rust_entry_test.pyand the_core/_rust_entryreferences; switch the build backend frommaturintohatchling._embedding_based_retrieve/_llm_based_retrieveand their now-orphaned helpers.Databaseprotocol: expose repositories only (no raw record dicts), so business logic can't bypass scope (where) filtering; drop the redundant mirror attributes on the concrete backends.retrieve_llm.py(RetrieveLlmMixin) and the memorize parsers intomemorize_parse.py(MemorizeParseMixin), composed as sibling mixins onMemoryService.retrieve.py1210→640,1220.memorize.py1420→skill/decoupling inmemory_fs: skills are always synthesized from the per-source descriptions, independent of memory-item extraction; docs/tests updated.Test plan
uv run ruff check src/memuuv run ruff format --check src/memuuv run mypy srcuv run python -m pytest tests/ --ignore=tests/test_lazyllm.py(97 passed, 1 skipped)uv lock --lockedconsistent;uv run deptry srccleanMade with Cursor