Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ C/C++ speech-to-text inference library. Runs diverse STT model families via [GGU
| Multitalker Parakeet Streaming 0.6B v1 | `multitalker-parakeet-streaming-0.6b-v1` | diarize, streaming, token timestamps | [docs/models/multitalker-parakeet-streaming-0.6b-v1.md](docs/models/multitalker-parakeet-streaming-0.6b-v1.md) |
| Nemotron 3.5 ASR Streaming 0.6B | `nemotron-3.5-asr-streaming-0.6b` | streaming, token timestamps | [docs/models/nemotron-3.5-asr-streaming-0.6b.md](docs/models/nemotron-3.5-asr-streaming-0.6b.md) |
| Nemotron Speech Streaming EN 0.6B | `nemotron-speech-streaming-en-0.6b` | streaming, token timestamps | [docs/models/nemotron-speech-streaming-en-0.6b.md](docs/models/nemotron-speech-streaming-en-0.6b.md) |
| Parakeet | `parakeet-ctc-0.6b`, `parakeet-ctc-1.1b`, `parakeet-primeline`, `parakeet-rnnt-0.6b`, `parakeet-rnnt-1.1b`, `parakeet-tdt-0.6b-v2`, `parakeet-tdt-0.6b-v3`, `parakeet-tdt-1.1b`, `parakeet-tdt_ctc-1.1b`, `parakeet-tdt_ctc-110m`, `parakeet-ultra`, `parakeet-unified-en-0.6b` | streaming, token timestamps | [docs/models/parakeet.md](docs/models/parakeet.md) |
| Parakeet | `parakeet-ctc-0.6b`, `parakeet-ctc-1.1b`, `parakeet-primeline`, `parakeet-rnnt-0.6b`, `parakeet-rnnt-1.1b`, `parakeet-tdt-0.6b-v2`, `parakeet-tdt-0.6b-v3`, `parakeet-tdt-1.1b`, `parakeet-tdt_ctc-1.1b`, `parakeet-tdt_ctc-110m`, `parakeet-ultra`, `parakeet-unified-en-0.6b`, `orukeet` | streaming, token timestamps | [docs/models/parakeet.md](docs/models/parakeet.md) |
| Qwen3-ASR | `qwen3-asr-0.6b`, `qwen3-asr-1.7b` | - | [docs/models/qwen3-asr.md](docs/models/qwen3-asr.md) |
| SenseVoice Small | `sensevoice-small` | - | [docs/models/sensevoice-small.md](docs/models/sensevoice-small.md) |
| Voxtral (2507) | `voxtral-mini-3b-2507`, `voxtral-small-24b-2507` | translate | [docs/models/voxtral.md](docs/models/voxtral.md) |
| Voxtral Realtime (2602) | `voxtral-mini-4b-realtime-2602` | streaming | [docs/models/voxtral-realtime.md](docs/models/voxtral-realtime.md) |
| Whisper | `breeze-asr-25`, `whisper-base`, `whisper-base.en`, `whisper-large`, `whisper-large-v2`, `whisper-large-v3`, `whisper-large-v3-turbo`, `whisper-medium`, `whisper-medium.en`, `whisper-small`, `whisper-small.en`, `whisper-tiny`, `whisper-tiny.en` | segment timestamps, translate | [docs/models/whisper.md](docs/models/whisper.md) |
| Whisper | `whisper-base`, `whisper-base.en`, `whisper-large`, `whisper-large-v2`, `whisper-large-v3`, `whisper-large-v3-turbo`, `whisper-medium`, `whisper-medium.en`, `whisper-small`, `whisper-small.en`, `whisper-tiny`, `whisper-tiny.en`, `breeze-asr-25` | segment timestamps, translate | [docs/models/whisper.md](docs/models/whisper.md) |
<!-- /catalog -->

**Speaker diarization models** (no transcription; verified by DER/JER rather than WER):
Expand Down
98 changes: 98 additions & 0 deletions catalog/orukeet.json

Large diffs are not rendered by default.

192 changes: 192 additions & 0 deletions docs/models/orukeet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Orukeet

<!-- catalog:intro -->
Upstream: [`oruk/orukeet`](https://huggingface.co/oruk/orukeet) at [`b59c13a`](https://huggingface.co/oruk/orukeet/commit/b59c13a).

A tuned version of parakeet-tdt-0.6b-v3 from Oruk AI. Turns speech into
text in 25 European languages, works offline, and runs as fast as v3. Give
it a 16 kHz mono WAV and it gives you back text with punctuation and
capitals. Swap it in anywhere you use v3. Does not stream and does not
translate.
<!-- /catalog -->

## What it does

Speech to text in 25 European languages, offline. It is
`parakeet-tdt-0.6b-v3` with new weights from Oruk AI: same size, same speed,
same 25 languages, slightly more accurate. If you already run v3, point at
this file instead and you are done.

It does not stream and it does not translate.

<!-- catalog:pin -->
Licensed CC-BY-SA-4.0. Ported from upstream commit [`b59c13a`](https://huggingface.co/oruk/orukeet/commit/b59c13a), pinned 2026-10-06. Validated against the NeMo reference at transcribe.cpp commit [`65856fac`](https://github.com/handy-computer/transcribe.cpp/tree/65856fac) on 2026-10-06.
<!-- /catalog -->

## Download

<!-- catalog:downloads -->
| Quantization | Download | Size | WER (LibriSpeech test-clean) |
| --- | --- | ---: | ---: |
| F32 | [orukeet-F32.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-F32.gguf) | 2.51 GB | 1.86% |
| F16 | [orukeet-F16.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-F16.gguf) | 1.26 GB | 1.86% |
| Q8_0 | [orukeet-Q8_0.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-Q8_0.gguf) | 740 MB | 1.86% |
| Q6_K | [orukeet-Q6_K.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-Q6_K.gguf) | 610 MB | 1.86% |
| Q5_K_M | [orukeet-Q5_K_M.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-Q5_K_M.gguf) | 549 MB | 1.83% |
| Q4_K_M | [orukeet-Q4_K_M.gguf](https://huggingface.co/handy-computer/orukeet-gguf/resolve/main/orukeet-Q4_K_M.gguf) | 485 MB | 1.95% |
<!-- /catalog -->

<!-- catalog:recipe -->
WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 8, timestamps none, language hint `en`, decoded on cuda. Measured at transcribe.cpp `37273409`.
<!-- /catalog -->

<!-- catalog:prose field=wer.notes -->
A bit more accurate than v3 on English and on 20 of the 25 languages.
parakeet-ultra is more accurate than both on almost everything.
<!-- /catalog -->

<!-- catalog:accuracy -->
**FLEURS test**

| Language | Metric | Q8_0 |
| --- | --- | ---: |
| bg | WER | 11.14% |
| cs | WER | 10.22% |
| da | WER | 16.58% |
| de | WER | 5.64% |
| el | WER | 33.12% |
| en | WER | 4.43% |
| es | WER | 3.75% |
| et | WER | 15.16% |
| fi | WER | 11.88% |
| fr | WER | 6.17% |
| hr | WER | 11.26% |
| hu | WER | 13.53% |
| it | WER | 3.11% |
| lt | WER | 17.72% |
| lv | WER | 19.53% |
| mt | WER | 17.18% |
| nl | WER | 6.93% |
| pl | WER | 7.35% |
| pt | WER | 4.56% |
| ro | WER | 10.86% |
| ru | WER | 6.70% |
| sk | WER | 8.99% |
| sl | WER | 24.00% |
| sv | WER | 13.26% |
| uk | WER | 6.53% |
<!-- /catalog -->

## Quick Start

```bash
cmake -B build
cmake --build build

build/bin/transcribe-cli \
-m models/orukeet/orukeet-Q8_0.gguf \
samples/jfk.wav
```

Audio has to be 16 kHz mono WAV. Convert anything else first:

```bash
ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav
```

It figures out the language on its own. Add `--language de` (or any of the
25 codes) if you want to force one.

## Performance

Same speed as v3.

### Apple M4 Max

<!-- catalog:perf machine=m4-max -->
Compute latency (mel + encode + decode), speedup over realtime in parentheses; profile `asr-publication-v2`: mean over 3 iterations after 1 warmup.

| Backend | Sample | Q8_0 | Q4_K_M |
| ------- | ------------ | ---------------: | ---------------: |
| Metal | jfk (11.0s) | 59 ms (185.68×) | 65 ms (170.12×) |
| Metal | dots (35.3s) | 159 ms (221.73×) | 164 ms (215.85×) |
| CPU | jfk (11.0s) | 304 ms (36.17×) | 305 ms (36.03×) |
| CPU | dots (35.3s) | 969 ms (36.45×) | 1.03 s (34.29×) |

Apple M4 Max: transcribe.cpp `d91374b3` on 2026-10-06.
<!-- /catalog -->

### AMD Ryzen 7 PRO 4750U

<!-- catalog:perf machine=ryzen-4750u -->
Compute latency (mel + encode + decode), speedup over realtime in parentheses; profile `asr-publication-v2`: mean over 3 iterations after 1 warmup.

| Backend | Sample | Q8_0 | Q4_K_M |
| ------- | ------------ | --------------: | --------------: |
| Vulkan | jfk (11.0s) | 454 ms (24.20×) | 455 ms (24.17×) |
| Vulkan | dots (35.3s) | 1.36 s (25.99×) | 1.38 s (25.59×) |
| CPU | jfk (11.0s) | 730 ms (15.07×) | 776 ms (14.18×) |
| CPU | dots (35.3s) | 2.90 s (12.17×) | 2.95 s (11.97×) |

AMD Ryzen 7 PRO 4750U (Radeon RADV RENOIR): transcribe.cpp `8bab590e` on 2026-10-06.
<!-- /catalog -->

Rerun the benchmark yourself:

```bash
uv run scripts/bench/run.py --profile --models orukeet
```

## How we checked it

Compared tensor by tensor against NeMo running Oruk's own `.nemo` file,
using the same tolerances as v3. Everything matches and the test transcript
is identical. Batches of 2, 4 and 8 give the same text as one at a time.

| Field | Value |
| --- | --- |
| Reference | NeMo, `oruk/orukeet` `orukeet-v0.1.0.nemo` |
| Dump script | `scripts/dump_reference_parakeet_nemo.py` |
| Manifest | `tests/golden/parakeet/orukeet.manifest.json` |
| Tolerances | `tests/tolerances/parakeet.json` |
| Command | `uv run scripts/validate.py all --family parakeet --variant orukeet --model models/orukeet/orukeet-v0.1.0.nemo` |

## Build it yourself

```bash
# get the source checkpoint
hf download oruk/orukeet orukeet-v0.1.0.nemo --local-dir models/orukeet

# convert to F32 GGUF
uv run --project scripts/envs/parakeet \
scripts/convert-parakeet.py models/orukeet/orukeet-v0.1.0.nemo \
models/orukeet/orukeet-F32.gguf --repo-id oruk/orukeet

# make the smaller files
uv run scripts/quantize-all.py models/orukeet/orukeet-F32.gguf

# check it against NeMo
uv run scripts/validate.py all --family parakeet --variant orukeet \
--model models/orukeet/orukeet-v0.1.0.nemo

# measure WER
uv run scripts/wer/ingest.py librispeech
uv run scripts/wer/run.py \
--model models/orukeet/orukeet-F32.gguf \
--manifest samples/wer/librispeech-test-clean.manifest.jsonl \
--out reports/wer/orukeet-F32.librispeech-test-clean.b1.jsonl
uv run scripts/wer/score.py reports/wer/orukeet-F32.librispeech-test-clean.b1.jsonl
```

Oruk also ships a `transcribe-cpp/orukeet-Q8_0.gguf` in their own repo.
It was made with this same converter and works too; ours is just rebuilt
from the F32 so all the sizes come from one place.

## Limits

- No streaming.
- No translation.
- Timestamps are per token, not per word.
- Licence is CC-BY-SA-4.0. The rest of the parakeet family is CC-BY-4.0.
If you make a derivative of the weights, it has to carry the same licence.
- Long files go through in one pass. Nothing cuts them up for you.
11 changes: 8 additions & 3 deletions docs/models/parakeet.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ of three decoder heads — TDT (transducer with a duration prediction
head), classical RNN-T, or CTC — and a TDT+CTC hybrid that ships both
heads in one checkpoint. All variants take 16 kHz mono PCM through a
mel frontend; English-only across the family, except
`parakeet-tdt-0.6b-v3`, Moondream's post-trained `parakeet-ultra` and the
German fine-tune `parakeet-primeline`, which cover 25 European languages.
`parakeet-tdt-0.6b-v3`, Moondream's post-trained `parakeet-ultra`, Oruk's
retune `orukeet` and the German fine-tune `parakeet-primeline`, which
cover 25 European languages.

For the architecture deep-dive, validation contract, and porting notes,
see the family doc at
Expand All @@ -24,6 +25,9 @@ Most users want one of three:
and on every FLEURS language. A drop-in replacement for
`parakeet-tdt-0.6b-v3`, which remains available.
See [parakeet-ultra.md](parakeet-ultra.md) for how it segments long audio.
- **Multilingual, alternative → `orukeet`.** Oruk AI's retune of v3. Same
size and speed, a bit more accurate than v3, less accurate than
`parakeet-ultra`. CC-BY-SA-4.0 weights. See [orukeet.md](orukeet.md).
- **German → `parakeet-primeline`.** primeLine's German fine-tune of
v3. Same size and speed; tuned for German while keeping the other 24
v3 languages usable.
Expand Down Expand Up @@ -54,12 +58,13 @@ WER is on LibriSpeech test-clean for the **Q8_0** preset, measured by
transcribe.cpp's WER pipeline. See each per-variant doc for the full
quant matrix and the comparison to NVIDIA's self-reported numbers.

<!-- catalog:family variants=parakeet-tdt-0.6b-v2,parakeet-ultra,parakeet-tdt-0.6b-v3,parakeet-primeline,parakeet-tdt-1.1b,parakeet-tdt_ctc-110m,parakeet-tdt_ctc-1.1b,parakeet-rnnt-0.6b,parakeet-rnnt-1.1b,parakeet-ctc-0.6b,parakeet-ctc-1.1b,parakeet-unified-en-0.6b -->
<!-- catalog:family variants=parakeet-tdt-0.6b-v2,parakeet-ultra,parakeet-tdt-0.6b-v3,orukeet,parakeet-primeline,parakeet-tdt-1.1b,parakeet-tdt_ctc-110m,parakeet-tdt_ctc-1.1b,parakeet-rnnt-0.6b,parakeet-rnnt-1.1b,parakeet-ctc-0.6b,parakeet-ctc-1.1b,parakeet-unified-en-0.6b -->
| Variant | Params | Languages | Q8_0 size | Benchmark | Q8_0 | Capabilities | Doc |
| --- | ---: | --- | ---: | --- | ---: | --- | --- |
| `parakeet-tdt-0.6b-v2` | 618M | en | 730 MB | LibriSpeech test-clean (WER) | 1.69% | token timestamps | [parakeet-tdt-0.6b-v2.md](parakeet-tdt-0.6b-v2.md) |
| `parakeet-ultra` | 627M | 25 languages + auto-detect | 740 MB | LibriSpeech test-clean (WER) | 1.80% | token timestamps | [parakeet-ultra.md](parakeet-ultra.md) |
| `parakeet-tdt-0.6b-v3` | 627M | 25 languages + auto-detect | 740 MB | LibriSpeech test-clean (WER) | 1.94% | token timestamps | [parakeet-tdt-0.6b-v3.md](parakeet-tdt-0.6b-v3.md) |
| `orukeet` | 627M | 25 languages + auto-detect | 740 MB | LibriSpeech test-clean (WER) | 1.86% | token timestamps | [orukeet.md](orukeet.md) |
| `parakeet-primeline` | 627M | 25 languages + auto-detect | 740 MB | FLEURS de (WER) | 5.98% | token timestamps | [parakeet-primeline.md](parakeet-primeline.md) |
| `parakeet-tdt-1.1b` | 1.1B | en | 1.27 GB | LibriSpeech test-clean (WER) | 1.38% | token timestamps | [parakeet-tdt-1.1b.md](parakeet-tdt-1.1b.md) |
| `parakeet-tdt_ctc-110m` | 114M | en | 135 MB | LibriSpeech test-clean (WER) | 2.43% | token timestamps | [parakeet-tdt_ctc-110m.md](parakeet-tdt_ctc-110m.md) |
Expand Down
22 changes: 22 additions & 0 deletions docs/porting/families/parakeet.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,28 @@ uv run --project scripts/envs/parakeet-kestrel scripts/wer/run_reference_parakee
| Speaker diarization | n/a | n/a | n/a | OUT OF SCOPE — not a diarizer | SKIP — not exposed by runtime |
| Translation | n/a | n/a | n/a | OUT OF SCOPE — not advertised | SKIP — not exposed by runtime |

### orukeet (oruk/orukeet)

Weights-only fine-tune of `parakeet-tdt-0.6b-v3` (Oruk AI, r3 checkpoint).
Half of the encoder's 9-tap depthwise conv taps are frozen fitted Gabor
values, stored as ordinary F32 conv weights; architecture, tokenizer and
frontend are byte-for-byte v3, so no runtime work. Reference is NeMo on
the `.nemo` archive (the publisher's benchmark pipeline), same env as v3.
License is CC-BY-SA-4.0, not CC-BY-4.0. Intake:
`reports/porting/parakeet/orukeet/intake.json`.

| Capability | Mode | Command / test | Expected observable | Target | Status |
|---|---|---|---|---|---|
| Transcribe | explicit en | `build/bin/transcribe-cli -m models/orukeet/orukeet-F32.gguf --language en samples/jfk.wav` | English transcript with PnC; LibriSpeech test-clean WER within Stage 7 band of the NeMo oracle | MUST PASS | PASS — jfk byte-equal to NeMo (validate.py compare 18/18 within the finalized parakeet tolerances, transcript exact); LibriSpeech test-clean gate in Stage 7 |
| Transcribe | auto | `build/bin/transcribe-cli -m models/orukeet/orukeet-F32.gguf samples/jfk.wav` | English transcript, no hint | MUST PASS | PASS — no-hint jfk byte-equal to NeMo |
| Transcribe | explicit non-English (de/ru/uk clips) | `… --language de samples/german.wav` | transcript in source language with correct script | MUST PASS | PASS — german.wav identical with and without --language de; ru-short / uk-short correct Cyrillic transcripts |
| Language detection | auto, non-English | `build/bin/transcribe-cli -m models/orukeet/orukeet-F32.gguf samples/ru-short.wav` (no `--language`) | Russian transcript without a hint; same dispatcher path as v3 | MUST PASS | PASS — ru-short / uk-short / german transcribed in-language with no hint (same dispatcher path as v3) |
| Offline batch | batch | `uv run scripts/batch_parity.py --model <gguf> --list <list.txt> --batch-sizes 2,4,8 --backend cpu` | per-item transcripts equal to serial | MUST PASS | PASS — text byte-equal serial vs 2/4/8 on 7 clips incl. dots (35 s), CPU (golden tests/golden/batch/orukeet.cpu.json, list tests/golden/batch/orukeet.list). product-names.wav (56 s) flips one near-tie token ('QuidQuil' vs 'Quid Quill') at batch 4/8 in a mixed-length batch; parakeet-tdt-0.6b-v3 Q8_0 shows the identical flip on the same clip, so it is a pre-existing runtime property of the family on long padded clips, not this checkpoint, and the clip is excluded from the golden |
| Timestamps (token/word/segment) | output | `… --timestamps word samples/jfk.wav` | monotonic word times, same TDT duration math as v3 | MUST PASS | PASS — 22 monotonic word spans on jfk, same TDT duration math as v3 |
| Streaming | streaming | n/a (offline full-context model, `streaming: false`) | n/a | OUT OF SCOPE — no streaming training | SKIP — not exposed by runtime (offline-only model) |
| Speaker diarization | n/a | n/a | n/a | OUT OF SCOPE — not a diarizer | SKIP — not exposed by runtime |
| Translation | n/a | n/a | n/a | OUT OF SCOPE — not advertised | SKIP — not exposed by runtime |

## Open decisions before Stage 3 (convert)

These decisions block converter design for the new variants and should
Expand Down
1 change: 1 addition & 0 deletions reports/porting/parakeet/forward-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,4 @@ cache to mis-index.
- **TDT global symbol budget** (`stt.parakeet.tdt.symbol_budget="global"`; ref `ParakeetTdt.generate_encoded`): one budget of `max_symbols * T_valid` steps, blank+duration-0 advances one frame, no per-frame cap.
- **VAD head + long-form segmenter** (`vad.{proj,ctx,out}`, `stt.parakeet.vad.*`, `stt.parakeet.segmenter.*`; ref `vad.py`, `segment.py`, `asr/audio.py` `AudioChunks.chunks`): `build_vad_graph` (subsampler -> proj 1x1 + SiLU -> ctx k=5 + SiLU -> out 1x1 -> sigmoid) per 120 s scan block with per-block mel normalization; host `longform::pause_segments` (float64, ties-to-even round); `run_longform` decodes each segment like a short clip and stitches (space-joined text, timestamps shifted by segment start and clamped to its end, one result segment per decoded segment). Gate: `vad.b<k>.{mel,pre_encode,proj,ctx,prob}`, `vad.prob`, `longform.segments` (exact), `seg.<j>.enc.{mel.in,final}`, stitched transcript exact, word timestamps.
- **Correctness regime**: `TRANSCRIBE_NO_FLASH=1` via manifest `cpp_env` (flash casts the rel-pos score bias to F16).
- **`orukeet`** (Oruk AI, reference NeMo): weights-only fine-tune of `parakeet-tdt-0.6b-v3`. Identical geometry, KVs, tokenizer and frontend (24 layers, d_model=1024, n_mels=128, `use_bias=false`, TDT head, pred_n_layers=2, SPM 8192+blank byte-identical to v3). 12,288 of the 24,576 encoder depthwise-conv taps are frozen fitted Gabor values stored as ordinary F32 conv weights, so the forward map is v3's with no deltas; the only converter change is the `orukeet` VARIANT_PROFILES entry (`stt.variant=tdt-0.6b-orukeet`, cc-by-sa-4.0). Validated with the finalized family tolerances unchanged (18/18 on jfk, transcript exact).
Loading