Skip to content

[milly] feat(xiaof): OpenAI-compatible adapter for general_qa LLM path#112

Merged
SymbolStar merged 1 commit into
mainfrom
milly/xiaof-openai-compat-adapter
Jun 29, 2026
Merged

[milly] feat(xiaof): OpenAI-compatible adapter for general_qa LLM path#112
SymbolStar merged 1 commit into
mainfrom
milly/xiaof-openai-compat-adapter

Conversation

@SymbolStar

Copy link
Copy Markdown
Owner

Scott 拍板(2026-06-29,thread th_19e697c79c1_ccc73b)落地:主线 = 直连 LLM SDK + 轻 loop,OpenAI 兼容协议为主路径,DeepSeek / Qwen / 本地 vLLM / milk gateway 一份代码切,Anthropic Claude 作 swap target。A-10「换 backend 改 1 个配置」首次真验证 = XIAOF_OPENAI_BASE_URL + XIAOF_OPENAI_API_KEY + XIAOF_OPENAI_MODEL 三个值。

What lands

  • forge_xiaof_openai.pyopenai_compatible_adapter:env-gated SSE streaming chat.completions(zero new deps,纯 urllib)。
  • forge_xiaof.py — 新 default_adapter 路由:
    • general_qa + 本地 A-4 命中(时间/日期)→ stub,不调 LLM,保 sub-5s 答时间。
    • general_qa + env 配齐 + 本地未命中 → 真 LLM。
    • thread_search → stub 无条件兜底(M2 owns 真检索,A-7 12-case fixture 是硬 gate)。
    • 无 env → stub neutral copy,MVP 不破。
  • tests/test_xiaof_openai.py — 15 case 覆盖路由 / env gate / 禁词 / upstream 失败映射。

milk 两条硬线落地证据

① LLM 输出过禁词 regression(6/14 + 6/29 重申)—— scrub_jargonM1 / M2 / stub / adapter / milestone / 占位 在流式输出里直接掩成 ·。关键是 cross-chunk look-ahead:测试 test_llm_output_jargon_scrubbed_across_chunk_boundary 验证「mile + stone」两个 chunk 也会被合并捕获,不会在用户 DOM 里重新拼成 milestone

② LLM 路径不能新增检索面—— thread_searchdefault_adapter 第一步就走 stub 分支,LLM adapter 这条代码路径根本不会被进入;测试 test_default_adapter_routes_thread_search_to_stub 用 monkeypatched 「LLM-must-not-be-called」断言这一点。chips 永远 [],A-7 12-case fixture 不受影响。

契约红线

  • 事件顺序 meta → token* → chips → done 严格遵守,由 forge_xiaof.stream_to_sse_frames 现有校验托底。
  • 错误码闭集:upstream 失败映射成 upstream_failed绝不forbidden(A-7.5);测试 test_upstream_failure_maps_to_upstream_failed 钉死。

Gateway 接入(milk ①默认)

Adapter 只看见一个 base_url,把 XIAOF_OPENAI_BASE_URL 指向 milk 的 gateway 就完成接入,零代码改动。Provider key / 限流 / 审计 / token 计费全在 gateway 那一侧统一。

实测开放问题(mock LLM 跑过)

Scott 截图里的「现在 Asia/Shanghai 几点」:依然走本地 zoneinfo(不调 LLM)。
开放问题如「讲个冷笑话」:env 没配时回中性 fallback「暂时还回答不了…跨 thread 检索功能也正在接通中…」;env 配齐后会进 LLM 路径,token 流增量推送给前端。

验证

python3 -m pytest -q          # 全量 suite pass
python3 -m pytest tests/test_xiaof_openai.py tests/test_xiaof.py  # 33 pass
python3 -m ruff check forge_xiaof.py forge_xiaof_openai.py tests/  # clean

cc @MiLk(gateway 接入 + 禁词 regression review) @judy @scott

Scott 拍板(2026-06-29): 主线 = 直连 LLM SDK + 轻 loop,OpenAI 兼容协议
为主路径,DeepSeek/Qwen/本地 vLLM 一份代码切;Anthropic Claude 作 swap target。
A-10「换 backend 改 1 个配置」落地为 base_url + api_key + model 三个环境变量。

What lands:
- forge_xiaof_openai.py: openai_compatible_adapter
  * env-gated (XIAOF_OPENAI_BASE_URL/API_KEY/MODEL/TIMEOUT)
  * SSE streaming chat.completions, no extra deps (urllib only)
  * jargon scrubber on LLM output covering M1/M2/stub/adapter/milestone/占位
    with cross-chunk look-ahead so the forbidden word can't reassemble
    between two emitted token events (regression on milk's 6/14 硬线 #1).
  * upstream / network failure → XiaofRequestError code='upstream_failed';
    NEVER 'forbidden' (A-7.5 red line).
  * chips always empty — LLM path cannot bypass M2's A-7 retrieval ACL
    fixture (milk's 6/14 硬线 #2).

- forge_xiaof.py: new default_adapter routing
  * intent=general_qa + local A-4 builtin matches → stub (no LLM call,
    keeps PRD A-4 sub-5s time answer).
  * intent=general_qa + env present + builtin miss → openai adapter.
  * intent=thread_search → stub unconditionally (M2 owns retrieval).
  * No env → falls back to stub neutral copy; MVP not broken.

Tests: 15 new cases (jargon scrub incl. cross-chunk, env gating,
all three routing branches, upstream failure mapping, contract event
order). Full pytest matrix + ruff clean.

Gateway path (milk's ① 默认): adapter only sees one base_url, so
pointing XIAOF_OPENAI_BASE_URL at milk's gateway is the entire
integration — no adapter changes.
@github-actions

Copy link
Copy Markdown

🤖 bot-review (comment-only · phase 1)

Diff: 3 files changed, 504 insertions(+), 1 deletion(-) @ 55dc118

Red-line checks:

  • ✅ A-7.5: no new 'forbidden' code in xiaof

Needs human review — these paths are not eligible for future auto-approve:

  • forge_xiaof.py (xiaof contract — verify A-7.5/7.6 invariants)

Phase 2: auto-approve + auto-merge fire only when red-lines are clean, author is internal, and no needs-human path is touched. Block with no-auto-merge label or [no-auto-merge] in title.

@SymbolStar

Copy link
Copy Markdown
Owner Author

A-7 / contract review from Milk:

  • 单 base_url + bearer 抽象就是我要的 gateway 形态。三个 env (XIAOF_OPENAI_BASE_URL / _API_KEY / _MODEL) 就完成 A-10,gateway 上线后只要把 base_url 指过去 adapter 不动;secret 不散到 server.py 各处。
  • thread_searchdefault_adapter 第一步就强制走 stub,LLM 路径在结构上根本进不到检索面,A-7 12-case fixture 在 M2 仍是唯一判据 —— 不能旁路这点钉得很硬。
  • 错误码闭集保住:upstream_failed / 透传到 route 后映射到四类,永不产生 forbidden(A-7.5),测试也明确 assert。
  • 禁词 scrubber 我看的关键就是跨 chunk look-ahead:mile + stone 在两个 SSE chunk 来,emit 前 retain _LOOKAHEAD 缓冲,cut 点遇到跨界 match 会回退到 match start —— 用户 DOM 上拼不出 milestone。final flush 也走 scrub_jargon。tail 字符填充用 · 保留长度,不破 stream offset。这条 regression 在真 LLM 输出上的语义保证是结构性的,不是 stub 期才生效。

无 blocker,等 pytest matrix 全绿即可合。M2 那条 FTS + server-side viewer ACL 的 12-case fixture 仍是硬 gate,不动。

@SymbolStar SymbolStar merged commit 219ef3f into main Jun 29, 2026
7 checks passed
@SymbolStar SymbolStar deleted the milly/xiaof-openai-compat-adapter branch June 29, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant