[Epic] Make MarkAccent a standalone pitch-accent service (drop OJAD dependency)

[torrid-fish]

Epic / tracking issue. Long-running theme, not a single PR. Individual work items get their own issues/PRs and link back here.

Vision

API-tools is converging on pitch-accent marking (/api/MarkAccent/) as its primary purpose; the other endpoints (DictQuery, SentenceQuery, UsageQuery) see little use. The goal of this epic is to make the accent service standalone — no runtime calls to external services — so it is robust, reproducible, low-latency, and deployable offline.

The tokeniser half is already standalone (fugashi + NINJAL UniDic CWJ, PR #53). The remaining external dependency in the accent path is OJAD (gavo.t.u-tokyo.ac.jp), which supplies the connected-speech pitch contour. This epic tracks removing it.

Why

• Robustness: OJAD is an HTML scrape against a university server — unreachable from some networks (observed: TCP connect timeout), no SLA, ToS-grey. Same fragility class that produced the scrape-drift bugs on the other endpoints (DictQuery returns 404 for valid words (先生): EDRDG/JMdictDB markup changed name="e" → name="g" #54, Invalid symbols block the whole result parsing #57, Furigana parsing for date with number tend to drift #58).
• Latency / scale: in-process inference vs per-chunk HTTP round-trip + a semaphore to avoid hammering OJAD.
• Reproducibility & offline deploy: pinned local model + dictionary, no network at request time.

The technical gap

OJAD does connected-speech contour: accent-phrase (アクセント句) segmentation, compound accent (複合語アクセント), particle attachment, downstep. UniDic's aType (already on every token as lexical_kernel) is only lexical / dictionary-form pitch — it cannot express phrasing. So the standalone replacement is not "use aType"; it is a local prosody engine.

flat (today's OJAD-down)  →  #61 aType lexical  →  rule-based 複合語アクセント  →  OpenJTalk  →  OpenJTalk + marine
      worst                    approximation          partial                      replaces OJAD     closest to / beyond OJAD

Workstreams

• Floor — lexical fallback (Fallback to UniDic aType-derived lexical pitch when OJAD is unavailable #61): when no engine/OJAD is available, synthesise per-mora pitch from lexical_kernel instead of returning flat. Lowest-quality tier, but a guaranteed baseline. (separate issue: Fallback to UniDic aType-derived lexical pitch when OJAD is unavailable #61)
• Spike — local prosody engine: evaluate pyopenjtalk (OpenJTalk full-context accent), optionally run_marine=True (MARINE neural accent), as an OJAD replacement. Dict is naist-jdic (≈tens of MB, ≪ UniDic). Deliverable: accent-nucleus agreement vs an OJAD gold set + latency + image-size delta.
• Mapping layer: OpenJTalk full-context labels (/A:, accent-phrase fields) → the service's per-mora accent_marking_type (0/1/2). Reuse the existing mora-grouping convention.
• Integration: introduce the engine behind the existing get_ojad_result seam so the pipeline (align_accent, overrides, POS patches, postprocess) stays unchanged. Make the contour source pluggable: openjtalk | ojad | lexical.
• OJAD → optional: keep OJAD selectable for comparison, then default to local; eventually remove the scrape.
• Evaluation harness: a gold set of sentences with OJAD output captured on a network that reaches OJAD, to score any local engine offline (nucleus-position agreement %, by word class).

Open decisions

1. Quality bar: "not worse than OJAD" (→ likely need MARINE + eval) vs "good enough, simpler" (OpenJTalk only)?
2. MARINE: pulls in torch (heavy image). Worth it?
3. Image-size budget: already ~1.3 GB (UniDic). OpenJTalk dict is small; torch is not.
4. Repo split: spin the accent service into its own repo, leaving Dict/Sentence/Usage behind? (Under consideration — affects packaging/CD.)
5. Keep OJAD at all post-migration (as an optional high-accuracy backend) or remove entirely?

Explicitly out of scope (parked)

• Standalone-ifying DictQuery / SentenceQuery / UsageQuery, and their known bugs (DictQuery returns 404 for valid words (先生): EDRDG/JMdictDB markup changed name="e" → name="g" #54, Invalid symbols block the whole result parsing #57, Furigana parsing for date with number tend to drift #58). These stay external for now; revisit only if the repo-split decision pulls them along.

References

• Tokeniser already local: PR feat(accent): local UniDic + POS-driven patches #53 (Evaluate local UniDic (fugashi/Sudachi) for in-process accent + lower latency #50)
• OJAD graceful degradation: OJAD graceful fallback #59
• Lexical fallback floor: Fallback to UniDic aType-derived lexical pitch when OJAD is unavailable #61
• OJAD scrape seam: api/accent/ojad.py, consumed in api/accent/pipeline.py

[torrid-fish]

Spike result: pyopenjtalk (OpenJTalk) as an offline OJAD replacement — promising

Ran OpenJTalk locally (ephemeral env, no changes to pyproject/lock) and reconstructed accent with the standard ESPnet-style full-context prosody parser.

Key evidence — the 端 / 箸 / 橋 minimal triple, disambiguated by context, fully offline

端を:  ^ h a [ sh i o #        → はし rise, no fall  = heiban    (端)  ✓
箸で:  # h a ] sh i d e #      → は]し fall@mora1    = atamadaka (箸)  ✓
橋を:  # h a [ sh i ] o #      → はし]を fall@mora2   = odaka     (橋)  ✓

Other hand-checked sentences also correct: 今日=きょ]う (頭高), 日本語=にほんご (平板), 勉強=べんきょう (平板), 人工知能=じんこうち]のう.

So OpenJTalk gives connected-speech accent (accent-phrase segmentation # + nucleus ]), not just lexical pitch — i.e. it genuinely covers what we use OJAD for, in-process.

Footprint / availability

• open_jtalk_dic ≈ 23 MB (vs UniDic 1.3 GB) — negligible image-size delta.
• Pure CPU, no network at request time.
• MARINE (neural accent refinement, run_marine=True): NOT available out of the box — needs marine + torch (heavy). Defer unless eval shows OpenJTalk-alone accuracy is insufficient.

Mapping to our schema

OpenJTalk per-accent-phrase nucleus → our per-mora accent_marking_type (0=LOW/1=HIGH/2=FALL) is the same Tokyo-dialect math as #61, just driven by OpenJTalk's phrasing instead of a single word kernel. The only careful part is mora grouping (small kana ゃゅょ, っ, ー) — already called out as the "mapping layer" workstream above.

What this spike did NOT establish

• Quantitative agreement vs OJAD: OJAD is unreachable from the dev/CI network (TCP connect timeout), so no gold baseline could be captured here. Need to run the OJAD half on an OJAD-reachable network and score nucleus-position agreement. The harness is ready for that (OpenJTalk side + an OJAD <TODO> slot).
• MARINE quality (not installed).

Recommendation

OpenJTalk is a credible drop-in for the get_ojad_result seam. Proposed next steps:

1. Capture an OJAD gold set (a few hundred sentences) on an OJAD-reachable network.
2. Score OpenJTalk (and OpenJTalk+MARINE) nucleus agreement vs OJAD, by word class.
3. Decide MARINE (accuracy vs +torch image cost) and whether OJAD stays as an optional high-accuracy backend.

[torrid-fish]

Update: accuracy vs human gold (not just OJAD-vs-OpenJTalk agreement)

The earlier spike measured mutual agreement (≈90% realized-pitch) between OJAD and OpenJTalk. That overstated OpenJTalk's quality. Using a free, human-annotated ground truth — the JSUT gold set (sarulab-speech/jsut-label, BASIC5000, CC-BY-SA) — we can now score accuracy.

Result (200 sentences, 5355 gold morae)

Backend	coverage	exact per-mora	realized-pitch
OJAD	71.6%	85.5%	89.9%
OpenJTalk (vanilla)	76.3%	56.1%	66.7%

• OJAD's 89.9% sits in the literature band (rule-based ≈87%, neural ≈94%) → validates the harness; OJAD is a strong reference.
• OpenJTalk-vanilla is ~23 pts behind. It's a viable offline fallback, not yet an OJAD-quality replacement.
• Coverage ~72–76%: accuracy is over kana-aligned morae; the unaligned ~25% is reading-notation divergence (は/わ, を/お, ー), reported transparently.

What's on spike/openjtalk-accent now

• scripts/download_jsut_gold.sh — fetch the gold set (labels only, no audio).
• scripts/eval_accent_backends.py --gold data/gold/jsut-label --limit N — accuracy-vs-gold scoring for both backends.
• data/corpora/handpicked_v1.txt (+README) — the probe corpus, now versioned data.
• Docs: docs/openjtalk-vs-ojad-eval.md (Part 2 = gold accuracy), docs/accent-benchmarks-survey.md, docs/accent-annotation.md (surface/furigana/accent convention).

Benchmark survey takeaways: tdmelodic would improve OpenJTalk readings but is non-commercial / eval-only; MARINE is the most promising shippable lever.

---

Next experiment — does MARINE close the gap?

Goal: measure whether OpenJTalk + MARINE (pyopenjtalk[marine], run_marine=True, DNN accent estimation) narrows the 66.7% → 89.9% gap enough to justify dropping/optionalising OJAD.

Hypothesis: MARINE lifts OpenJTalk realized-pitch accuracy from ~67% toward ~80%+ (its paper/JSUT-trained range), at the cost of a torch dependency (image-size hit).

Method (reuses the existing gold harness — minimal new code):

1. pip install pyopenjtalk[marine] (torch); add a third backend openjtalk-marine that calls extract_fullcontext(text, run_marine=True). The shared fullcontext.py parser already handles its labels unchanged.
2. Run the same gold eval over all three backends on an identical sample (start --limit 200, then scale the OpenJTalk/MARINE side — no network — to the full 5000).
3. Report realized-pitch + exact accuracy vs gold for OJAD / OpenJTalk / OpenJTalk+MARINE side by side.

Secondary (tightens the numbers): add a kana-normalisation pass (は↔わ, を↔お, へ↔え, ー) before the difflib alignment to raise gold coverage above ~72%, so accuracy isn't measured on a biased subset.

Decision criteria:

• MARINE within a few points of OJAD (say ≥85% realized-pitch) → pursue local-only with OJAD as optional high-accuracy backend.
• Still well below OJAD → keep OJAD primary, OpenJTalk(±MARINE) as the offline-degradation fallback (slots into the fix(accent): degrade gracefully when OJAD is unavailable #60 graceful-degradation path), and revisit readings (tdmelodic eval-only) / a learner-targeted accuracy bar.

Cost note: torch materially increases image size (already ~1.3 GB from UniDic) — factor into the keep-OJAD-or-not call.

[torrid-fish]

MARINE experiment result — does the DNN estimator close OpenJTalk's gap to OJAD?

Ran OpenJTalk + MARINE (pyopenjtalk[marine], run_marine=True) as a third backend on the same JSUT gold harness, in an isolated venv (torch; numpy pinned <2 — MARINE relies on pre-2.0 uint8 wrapping — so the main env was untouched).

Accuracy vs gold (realized-pitch)

Backend	@200 (3-way)	@1000 (offline)	raw, no pipeline (~94% cov)
OJAD	89.8%	—	—
OpenJTalk	66.6%	66.6%	67.5%
OpenJTalk + MARINE	69.9%	70.3%	72.1%

MARINE adds only ~+4 pts — it does not close the ~20-pt gap to OJAD.

A raw engine-vs-gold check (bypassing our align/override/postprocess, so morae line up ~1:1 with gold → 94% coverage) gives the same ~+4–5 pt gain, so our OJAD-tuned pipeline is not blunting MARINE — the gain is genuinely small. (The metric caps ~90% even for OJAD due to convention/alignment noise, so read these as relative: OJAD ≫ MARINE > vanilla OpenJTalk.)

Costs / problems

• torch ≈ 1.1 GB (default wheel pulled the full CUDA stack → 4.8 GB venv; CPU-only torch ≈200 MB, still ≫ the 23 MB OpenJTalk dict).
• Latency ~73 ms/sentence (vs ~few ms vanilla, ~1.8 s OJAD).
• Robustness bug: MARINE AssertionErrors (Invalid sequence sizes in njd_results, marine_results) on inputs with ・, full-width digits, or long katakana-name sentences — multiple BASIC5000 sentences failed outright.

Recommendation

Keep OJAD primary; vanilla OpenJTalk as the offline-degradation fallback (the #60 path). MARINE's ~+4 pts don't justify +1.1 GB, 7× latency, and an input-robustness regression.

If higher offline accuracy is ever needed, the remaining levers are: reconcile the systematic verb-ending / 尾高·平板 marking differences, and/or tdmelodic for readings (eval-only — non-commercial license).

On the branch (spike/openjtalk-accent, pushed)

• Gated ACCENT_BACKEND=openjtalk-marine (default behaviour unchanged) + --backends/MARINE-guard/kana-norm in scripts/eval_accent_backends.py. torch is NOT added to runtime deps — MARINE is an isolated eval-only install.
• Full write-up (3-way + offline + raw diagnostic) in docs/openjtalk-vs-ojad-eval.md Part 2.