Repository navigation
fix: 증상 근거 규칙의 「아파」 누락과 통화 시나리오 시험 도구 - #12
Merged
Merged
Conversation
…ones Twelve synthetic call scenarios (sleep, walking, medication change, cough, negated and real chest pain, breathlessness after stairs vs at rest, child-only symptoms, disease name from child, short calls) with expected analysis state, facts, summary words and safety notice. scenario_audio.py synthesizes each line with ElevenLabs (cached per line) and assembles parent/child tracks plus a 16 kHz device-raw track. run_call_scenarios.py runs the real pipeline on a temporary SQLite DB: fake mode replaces only STT and Gemini; --real replays generated audio through Deepgram and Gemini. Prints an expected/actual table and writes it to .scenario-results/. Fake mode currently fails chest-pain-emergency: the symptom grounding pattern matches 아프 but not 아파, so the fact is dropped (pinned in tests).
The symptom grounding pattern only matched 「아프」, so a parent saying 「가슴이 너무 아파」, 「머리가 아파」 or 「허리가 아팠어」 had a correct Gemini symptom fact dropped as ungrounded. The call scenario harness caught it on chest-pain-emergency; the known-gap entry is removed now that it passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
변경 내용
backend/evals/call_scenarios.json,scripts/scenario_audio.py(ElevenLabs 화자별 음성, 문장 캐시),scripts/run_call_scenarios.py(가짜 모드 기본,--real은 Deepgram·Gemini 실제 호출).검증