Refactor: replace Zep with Graphiti + FalkorDB, add agent.profikid.nl deploy overlay#687
Open
profikid wants to merge 7 commits into
Open
Refactor: replace Zep with Graphiti + FalkorDB, add agent.profikid.nl deploy overlay#687profikid wants to merge 7 commits into
profikid wants to merge 7 commits into
Conversation
…3 LLM, deterministic embedder) + add e2e test, docker-compose, e2e one-shot, deploy notes
…e one-shot, bootstrap/up/health scripts, README) + gitignore secrets.env
…DME fork notice, e2e Dockerfile path fix - backend/app/api/graph.py: add POST /api/graph/ingest_text (one-call: project + ontology + async build + returns project_id/task_id) Designed for cron automations that need to ingest a markdown briefing without the 3-step project -> ontology -> build dance. - backend/app/services/ontology_generator.py: bump max_tokens 4096->16384 and add a retry-with-compact-prompt path on JSON parse failure. M3 truncates long entity-attribute arrays; the retry asks for a stripped schema (no attributes, 6 entity types max, 6 edge types max). - deploy/Dockerfile.e2e: build context is the project root, so COPY needs the deploy/ prefix; entrypoint uses 'uv run python' because the MiroFish image is uv-managed (flask/graphiti/falkordb all live under /app/backend/.venv). - README.md: replace Zep + Qwen references with the Graphiti+FalkorDB stack and MiniMax M-series LLMs; add 'About this fork' section documenting the deploy overlay, ingest_text API, and cron integration.
…r.agent.profikid.nl The browser lives at /docker/falkordb-browser/ on the host (separate from the MiroFish stack so they can be restarted independently). It joins the mirofish_net bridge network as 'external' and dials the same FalkorDB sidecar the MiroFish app uses. Users sign in via the browser UI by entering host=falkordb, port=6379, empty user/pass — the connection is server-side from inside the browser container, so the user's machine doesn't need direct access to the Docker network.
…nto home history - New GraphView.vue: lightweight page at /graph/<id> that resolves a graph_id back to its owning project_id via /api/graph/project/list, then router-replaces to /process/<projectId> so the existing project page handles the full graph visualisation. - HistoryDatabase.vue: fetch /api/simulation/history and /api/graph/project/list in parallel and merge by project_id (sims win on conflict, project list fills the gap so home history is non-empty for users who only built graphs). Newest first by created_at. - frontend/src/api/graph.js: add listProjects(limit) wrapper. - frontend/src/router/index.js: register /graph/:id route, props: true. - frontend/src/api/index.js: switch VITE_API_BASE_URL fallback from || to ?? so an empty string is honoured (Vite uses empty to mean 'same origin', letting Traefik reverse-proxy /api/* to the backend container without an absolute baseURL). - Dockerfile: VITE_API_BASE_URL="" baked in at build time (default). - graphiti_service.py: strip lowercase <think> / <antmlthinking> reasoning blocks in addition to <think> (M2.7 vs M3/Qwen emit different reasoning tags); accept M2.7's 'entity_types' plural list shape in _normalize_extraction_payload.
…as default (#1) Frontend (.vue, .js, .ts): - All Chinese in HTML comments, JS comments, CSS comments translated - All hardcoded Chinese UI strings (titles, alerts, console errors) translated - The few remaining Chinese fragments in Step2EnvSetup.vue (4) and Step4Report.vue (77) are intentional: stage-name identifiers that must match what the backend emits, and LLM-output parser patterns that match Chinese section headers in the LLM response. Backend (Python): - All # comments, """ and ''' docstrings translated - All log/print/raise/error message strings translated - All LLM system prompts (ONTOLOGY_SYSTEM_PROMPT, _PROMPT, _INSTRUCTION, _TEMPLATE, _TASK_PROMPT, _EXTRACTOR, _PLANNER, _REPORTER, _SUMMARY, _OUTLINE, _SYNTHESIS, _REWRITE, _SEARCH_PROMPT, _INTERVIEW, _REFLECTION, _REACT, _ROLE, _CONTEXT, _GUIDELINES, _AGENT, _MESSAGE, _EXAMPLE, _SYNTHESIZER, _WRITER, _CRITIQUE, _REVISION, _FEEDBACK, _PERSONA, _FORMAT, _JUDGE, _ROUTER, _SECTION_PLANNER, _PARSER, _EXTRACT, _RATIONALE variables) translated - stage-name string literals in zep_graph_memory_updater.py (return values used as Zep episode content) translated - LLM-output parser patterns in zep_tools.py and report_agent.py (e.g. text.match(/分析问题:/) and section headers like '### 【关键事实】') kept as Chinese, because they match the LLM's output format. i18n: - locales/zh.json: kept as source of truth for Chinese strings - locales/en.json: kept (was already a complete English translation) - locales/nl.json: NEW — full Dutch (Nederlands) translation of en.json (633 leaf keys, all interpolation placeholders preserved) - locales/languages.json: added 'nl' entry with label 'Nederlands' and llmInstruction 'Antwoord in het Nederlands.' Frontend i18n config: - frontend/src/i18n/index.js: default locale changed from 'zh' to 'nl', fallback locale from 'zh' to 'en'. Dutch is now the default UI language; English is the fallback. Misc: - package.json: description translated - docker-compose.yml: inline comment translated - README.md / README-ZH.md / locales/zh.json / locales/languages.json: Chinese content intentionally preserved (bilingual readme pointers and language-metadata source of truth) Files: 59 changed, +6810 / -6102 Co-authored-by: hermes <hermes@profikid.nl>
- frontend/src/views/SimulationView.vue: remove onMounted auto-stop of running simulations. Step 3's handleGoBack() already stops the sim when the user explicitly clicks 'Back', so onMounted only killed sims when users opened the sim URL directly or refreshed Step 2 (the 'loading error' symptom). - backend/scripts/run_parallel_simulation.py: lower OASIS semaphore from 30 to 8 to fit the 8GB host's memory budget. - backend/vendor/camel-oasis/oasis/social_agent/agents_generator.py: add .get() defaults for mbti/gender/age/country to handle sparse agent profiles (KeyError: 'mbti' bug).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a maintained fork of
666ghj/MiroFishfromprofikid. We use it for an automated OSINT briefing pipeline atmirofish.agent.profikid.nl(Iran/US/Israel briefing cron every 12h, entities land in FalkorDB via the new/api/graph/ingest_textone-call API).What changed
POST /api/graph/ingest_text— one-call ingest (project + ontology + async build) for cron automations. The original 3-step flow is preserved for the UI.deploy/): single-host Docker stack with Traefik + Let's Encrypt + FalkorDB sidecar + e2e one-shot container + ingest helper scriptdeploy/e2e_test.pyruns against a real OSINT briefing seed, verifies entities land in FalkorDBhttps://api.minimax.io/v1. Thellm_client.chat()already strips the M-series mandatory<think>...</think>reasoning blockWhy fork (vs PR-into-upstream)
This is a self-contained rewrite of the graph layer (
backend/app/services/graphiti_service.py, ~1150 lines). The shim is a Zep-shaped facade so the rest of the MiroFish code (which still speaks Zep) didn't have to change, but the diff is large. Happy to split into smaller PRs if useful; we run it in production so we can dogfood it.Not changed
docker-compose.ymlat the repo root (the original Zep-based path) is preserved for users who still want that setup.Live deployment
https://mirofish.agent.profikid.nldeploy/README.mdin this fork for the host layoutScreenshots of the e2e output are in the repo at
deploy/README.md. Happy to do a live walkthrough on Discord if you want to see the entities land in FalkorDB in real time.