Skip to content

fix(doctor): read speaker labels from harvested text at Step 1.5 - #193

Open
titanwings wants to merge 1 commit into
atomic/pr-2-view-back-refsfrom
atomic/pr-3-doctor-shape
Open

titanwings wants to merge 1 commit into
atomic/pr-2-view-back-refsfrom
atomic/pr-3-doctor-shape

Conversation

@titanwings

Copy link
Copy Markdown
Owner

Why

doctor --require-shape 是 SKILL.md Step 1.5 的门禁。它应该在 Collect 之后、Derive 之前拦住「会议流水蒸不出一个人」的材料。

旧实现从 evidence/derived/voice.json 读说话人。那份文件要到 retrospect 才有。按文档顺序跑时 speakers 是 0,C-SPAN 语料被当成「没有说话人标注」并判 PASS。门禁在该出现的位置是空的。

数据形状如下。

输入 调用 字面期望
us-house 语料,只 harvest,不跑 retrospect doctor --require-shape --json 退出 1,shape[0].verdict 为 FAIL,speakers 18,attributed 85,reasons 含「只有 3% 的单元能归到某个说话人」
同上,不带 --require-shape doctor --json 退出 0,shape[0].verdict 仍为 FAIL,ok 为 true
synthetic-interview,只 harvest doctor --require-shape --json 退出 0,verdict PASS,speakers 2,top_speaker 林工
一份没有说话人标注的短 markdown,harvest 后 doctor --json shape FAIL,reasons 含「可引用单元只有 N 个,低于 20」,notes 含「没有说话人标注」

Scope

只改 corpusShape。从 knowledge/text/*.md 的锚点行读解析层写出的 [k00NN] <at> <speaker>: 前缀,统计单元数、可归属比例、最活跃者占比。evidence/derived/voice.json 存在时只交叉核对,不一致则 warning,verdict 仍以 text 为准。

不改 deliveredAnchors。阈值不变。可引用单元小于 20、多人材料可归属小于 40%、最活跃者小于 20% 判 FAIL。reasons 原文不变。

tests/doctor-coverage.test.mjs 三条形状测试改为 harvest 之后直接 doctor。不再手写 voice.json 夹具。

tests/gates/baseline/audit-tests.txt 只删这一行。

  • R24 Step 1.5: doctor --require-shape run after Collect and before Derive fails the C-SPAN floor corpus

该行对应 tests/audit/retrospect-doctor.test.mjs 的 T43。T44、T45 保持绿。基线数据行从 17 减到 16。

Blast Radius

harvest 之后立刻跑 doctor 才会从 PASS 变成 FAIL。这是要修的行为。retrospect 之后的 C-SPAN 路径仍是 FAIL,reasons 原文不变,speakers 仍是 18,attributed 仍是 85。

面试语料同一顺序仍是 PASS。acceptance-us-house 仍出现 FAIL · units=2744 speakers=18 attributed=85

栈停在 dot-skill-test,dot-skill 保留。

Verification

T43 由红转绿。T44、T45 保持绿。基线从 17 行减到 16 行。

改前,Node v22.14.0,HEAD 8784cac

PASS audit-tests  (5.3s, exit 1)
     ratchet failing 17, baseline 17

T43 失败原文:

before retrospect: speakers=0 attributed=0 notes=["材料里没有说话人标注:按单一对象处理(对本人文章/邮件/单人口述是正常的)"]

'PASS' !== 'FAIL'

改后,Node v22.14.0,HEAD d05717b

PASS audit-tests  (7.6s, exit 1)
     ratchet failing 16, baseline 16

改后,Node v20.20.2:

PASS audit-tests  (9.4s, exit 1)
     ratchet failing 16, baseline 16

node scripts/run-gates.mjs --without playwright 在 Node v20.20.2 与 Node v22.14.0 都是 8/8 gates passed。npm test 是 # tests 403、# fail 0。

DISTILLY_PLAYWRIGHT_ROOT=/tmp/pw-root node scripts/run-gates.mjs --requires playwright 在 Node v22.14.0 上是 3/3 gates passed。三份语料都是 验收结果:18/18 通过。us-house 含 FAIL · units=2744 speakers=18 attributed=85

You see,Node v22.14.0。node bin/distilly.mjs 实测。us-house harvest 后不跑 retrospect,doctor --require-shape --json 退出 1,verdict FAIL,speakers 18,attributed 85。synthetic-interview 同一顺序退出 0,verdict PASS,speakers 2,top_speaker 林工。

新用例在 base 上失败,在 head 上通过。base 是 Node v22.14.0,旧 corpusShape 加新测试。

the corpus shape check passes on a corpus that can carry a person 失败:

Expected values to be strictly equal:

0 !== 2

the corpus shape check catches a multi-speaker corpus nobody can be cut out of 失败:

Expected values to be strictly equal:

'PASS' !== 'FAIL'

head 上 node --test tests/doctor-coverage.test.mjs 是 # tests 9、# fail 0。

Rollback

回退 d05717b。已 harvest 的 text 不用迁移。栈停在 dot-skill-test,dot-skill 保留。

corpusShape now counts speakers from knowledge/text prefixes so
doctor --require-shape works after Collect, before Derive.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants