Skip to content

feat(carmenta): close the commercial gaps: forms, scanned filings, PDFs (0.11) - #42

Open
Ttimmahlax wants to merge 3 commits into
masterfrom
feat/commercial-gaps
Open

Ttimmahlax wants to merge 3 commits into
masterfrom
feat/commercial-gaps

Conversation

@Ttimmahlax

Copy link
Copy Markdown
Collaborator

Closes the production gaps a real caller hit running Carmenta on scanned financial-disclosure filings. The gap list and measured results are in docs/plans/commercial-gaps.md.

What ships

Carmenta 0.11

  • Offline weights: manifests ship in the crate, with offline() and preload().
  • Charset-constrained decoding.
  • Quarter-turn orientation: 144/144 correct on turned pages, at 48% of the cost of a read.
  • Opt-in form-rule removal.
  • Full-width to ASCII fold on non-CJK lines.
  • Lexicon matching.
  • Checkbox detection: 100% recall and state on the corpus.
  • Label-to-value pairing: 220/221 correctly-read values paired.

ffai-media 0.6.8

  • decode_image(&[u8]).
  • Opt-in pdf feature. Pages are composited as a viewer shows them, so redactions are painted in rather than read through.
  • Text layers carry a redaction_risk flag.
  • Every allocation driven by untrusted PDF values is bounded.

CLI 0.7

  • ffai ocr reads PDFs.
  • New flags: --charset, --remove-rules, --auto-orient, --checkboxes.
  • ffai models --verify.

Bench

  • Digit error rate in OCR notes.

Disclosure corrections

  • The public 0.041 Text^Edit figure was Unlimited-OCR's number, not ours. Carmenta's is 0.108 on the same subset.
  • The top-level README now uses official-evaluator numbers throughout, replacing those from the withdrawn biased scorer.

Versions

Every changed crate is one step past its crates.io version. The table is in CHANGELOG.md.

  • ffai-core 0.8.0 is breaking: OcrOptions gained public fields.
  • Merging this publishes through release-plz.

Checks run locally

All pass:

  • the three blocking clippy gates;
  • fmt;
  • lint_invariants;
  • the hardening-table check;
  • workspace cargo check --all-targets;
  • lib and integration tests for the changed crates (244 tests).

🤖 Generated with Claude Code

tim-almond-house and others added 3 commits September 25, 2026 06:31
…Fs (0.11)

A real caller ran Carmenta on scanned financial-disclosure filings and hit a
run of production gaps (docs/plans/commercial-gaps.md). This batch closes them.

Carmenta 0.11:
- offline weights: manifests ship in the crate, offline()/preload(), and
  resolution that needs only the runtime files
- charset-constrained CTC decoding, and confidence that stays unrenormalized
- quarter-turn orientation (144/144 on turned pages, 48% of a read)
- opt-in form-rule removal
- full-width to ASCII fold on non-CJK lines
- lexicon matching with look-alike-weighted edit distance
- checkbox detection (100% recall and state on the corpus)
- label-to-value pairing (220/221)

ffai-media 0.6.7:
- decode_image(&[u8])
- opt-in pdf feature: pages are composited as a viewer shows them, so
  redactions and overlays are painted in rather than read through
- redaction_risk on text layers
- every allocation from untrusted PDF values is bounded

CLI 0.7: PDF input, --charset, --remove-rules, --auto-orient, --checkboxes,
and `ffai models --verify`.

Bench: digit error rate in OCR notes.

Disclosure: corrects the public 0.041 Text^Edit figure. That number was
Unlimited-OCR's; Carmenta's is 0.108 on the same subset. The top-level README
now uses official-evaluator numbers throughout, replacing those from the
withdrawn biased scorer.

Versions: core 0.8.0, carmenta 0.11.0, cli 0.7.0, and patch bumps for the
dependents (see CHANGELOG).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… crate past crates.io

master's release commit already published media 0.6.7, models 0.6.5,
bench 0.7.6, mercury 1.0.4, diana 0.7.7, argus 0.7.4, carmenta-wasm 0.2.2,
mercury/argus-wasm 0.1.2 and wasm 0.1.4 -- the same numbers this branch had
chosen. Left as they were, release-plz would have treated them as already
published and the new code (the PDF feature among it) would never ship.

Every changed crate is now one step past its registry version, with per-crate
CHANGELOG entries and the workspace table updated to match.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pdf feature adds lopdf 0.44 and fax 0.3. lopdf's non-optional encryption
support brings aes, cbc, ecb, cipher, md-5, stringprep and small helpers.
All of them are exempted at their locked versions, which is the repo's
convention for dependency deltas. The ffai-* rows move to their published
versions, and two stale entries (shlex, simdutf8) are pruned.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants