You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When OJAD is unavailable, MarkAccent currently degrades to flat pitch (accent_marking_type=0 on every mora) — see the graceful-degradation path added in #59 / PR #53. Meanwhile UniDic's per-morpheme accent kernel (aType) is already tokenised and carried on every word as lexical_kernel / lexical_kernel_alts, but it is not used to synthesise any pitch.
This issue proposes an OJAD-down fallback that converts lexical_kernel into a per-mora pitch contour, so degraded responses carry an approximate (dictionary-form) accent instead of nothing.
When OJAD returns nothing, align_accent takes its m == 0 branch and every token goes through _fallback_word → _build_word_result(..., ojad_span=[]), which emits a single hard-coded flat entry:
lexical_kernel is only passed through as metadata (api/accent/align.py:444); it never drives accent_marking_type. So in degraded mode the client sees furigana + flat pitch (plus whatever apply_accent_patches POS rules fire independently, e.g. ます→FALL).
Confirmed locally: with OJAD unreachable, all four test articles return 200 with accent_marking_type=0 throughout and the warning field set.
Proposed feature
In the empty-OJAD-span path only, when lexical_kernel is not None, derive a per-mora contour from the kernel using the standard Tokyo-dialect mapping (same 0=LOW / 1=HIGH / 2=FALL-kernel convention documented in api/accent/user_patches.py):
None kernel (particles, auxiliaries, override-constructed tokens) stays flat as today.
Possible implementation
Add a helper, e.g. _kernel_to_accents(furigana, kernel) -> list[AccentInfo], in api/accent/align.py, mapping the kernel position to per-mora accent_marking_type.
Count morae with the existing convention (small kana ゃゅょ etc. attach to the preceding mora — reuse whatever user_patches mora-counting uses so the kernel index lands on the right mora).
Use lexical_kernel (primary); ignore lexical_kernel_alts for the first cut.
Call it from the empty-span branches in _build_word_result (align.py:431) and have align_accent's m == 0 branch (align.py:~534) route through it, instead of the flat fallback_accent.
Gate it so it only affects the OJAD-down / empty-span case. OJAD-normal behaviour must stay byte-identical (the per-mora OJAD contour still wins whenever a voiced span exists).
Consider exposing it as opt-in (request flag, e.g. lexical_pitch_fallback, or config) so callers who prefer "flat + lexical_kernel metadata" can keep today's behaviour.
Keep / adjust the existing warning wording to say pitch is lexical-form approximated rather than absent.
Verify interaction with apply_accent_patches (runs after align in pipeline.py): the POS ます/たい patches must not double-mark a FALL that the kernel mapping already produced.
Caveats / non-goals
UniDic aType is lexical (isolated dictionary-form) pitch, not connected-speech contour. Cross-word phrasing (downstep deletion across a phrase, 助詞 attachment, 連語 sandhi) is not modelled — this is a best-effort approximation, strictly inferior to OJAD, only meant to beat "completely flat".
kernel_absorbed is about OJAD-present cases and is orthogonal here.
With OJAD reachable: output unchanged (diff-clean against current behaviour).
With OJAD unavailable: words with lexical_kernel = N render a kernel-consistent contour (heiban / atamadaka / nakadaka verified on a few fixtures, e.g. 端=heiban, 箸=頭高, 橋=尾高).
None-kernel tokens (particles) stay flat.
ruff check / ruff format clean; existing import sanity holds.
Summary
When OJAD is unavailable,
MarkAccentcurrently degrades to flat pitch (accent_marking_type=0on every mora) — see the graceful-degradation path added in #59 / PR #53. Meanwhile UniDic's per-morpheme accent kernel (aType) is already tokenised and carried on every word aslexical_kernel/lexical_kernel_alts, but it is not used to synthesise any pitch.This issue proposes an OJAD-down fallback that converts
lexical_kernelinto a per-mora pitch contour, so degraded responses carry an approximate (dictionary-form) accent instead of nothing.Likely a separate PR on top of PR #53.
Current behaviour
When OJAD returns nothing,
align_accenttakes itsm == 0branch and every token goes through_fallback_word→_build_word_result(..., ojad_span=[]), which emits a single hard-coded flat entry:lexical_kernelis only passed through as metadata (api/accent/align.py:444); it never drivesaccent_marking_type. So in degraded mode the client sees furigana + flat pitch (plus whateverapply_accent_patchesPOS rules fire independently, e.g. ます→FALL).Confirmed locally: with OJAD unreachable, all four test articles return
200withaccent_marking_type=0throughout and thewarningfield set.Proposed feature
In the empty-OJAD-span path only, when
lexical_kernel is not None, derive a per-mora contour from the kernel using the standard Tokyo-dialect mapping (same0=LOW / 1=HIGH / 2=FALL-kernelconvention documented inapi/accent/user_patches.py):0[LOW, HIGH, HIGH, …, HIGH]→[0,1,1,…,1]1[FALL, LOW, …, LOW]→[2,0,…,0]N≥2[LOW, HIGH, …, HIGH(mora N)=FALL, LOW, …]→[0,1,…,2,0,…]Nonekernel (particles, auxiliaries, override-constructed tokens) stays flat as today.Possible implementation
_kernel_to_accents(furigana, kernel) -> list[AccentInfo], inapi/accent/align.py, mapping the kernel position to per-moraaccent_marking_type.user_patchesmora-counting uses so the kernel index lands on the right mora).lexical_kernel(primary); ignorelexical_kernel_altsfor the first cut._build_word_result(align.py:431) and havealign_accent'sm == 0branch (align.py:~534) route through it, instead of the flatfallback_accent.lexical_pitch_fallback, or config) so callers who prefer "flat +lexical_kernelmetadata" can keep today's behaviour.warningwording to say pitch is lexical-form approximated rather than absent.apply_accent_patches(runs after align inpipeline.py): the POS ます/たい patches must not double-mark a FALL that the kernel mapping already produced.Caveats / non-goals
aTypeis lexical (isolated dictionary-form) pitch, not connected-speech contour. Cross-word phrasing (downstep deletion across a phrase, 助詞 attachment, 連語 sandhi) is not modelled — this is a best-effort approximation, strictly inferior to OJAD, only meant to beat "completely flat".kernel_absorbedis about OJAD-present cases and is orthogonal here.Acceptance criteria
lexical_kernel = Nrender a kernel-consistent contour (heiban / atamadaka / nakadaka verified on a few fixtures, e.g. 端=heiban, 箸=頭高, 橋=尾高).None-kernel tokens (particles) stay flat.ruff check/ruff formatclean; existing import sanity holds.References
pipeline.pyOJADtry/except OJADUnavailableError)api/accent/align.py:423,:431,m == 0branch:~534api/accent/models.pylexical_kernel/lexical_kernel_alts/kernel_absorbed0/1/2convention:api/accent/user_patches.pyschema docstring