Tracking: Whisper-class offline ASR on Android — engine-side P0 #1282

Goal

Make SKaiNET the obvious engine for a fully offline, Whisper-class transcription app on Android. The reference case is an existing app on LiteRT (Whisper large-v3-turbo split encoder/decoder on OpenCL FP16, Parakeet TDT 0.6B, Silero VAD on ONNX Runtime, a 350M LLM for titles on LiteRT-LM): three native runtimes in one APK, hand-written mel/FFT/resampler/tokenizers/decode loops, and model-file surgery plus byte-patched runtime AARs to work around GPU numerics bugs. Its author's Pixel 7 Pro numbers are the bar:

Target for this milestone: CPU-only SKaiNET Whisper large-v3-turbo at or below 82 s on the same class of device, token-identical to whisper.cpp, with the audio side and VAD served by SKaiNET libraries.

What 0.54.0 already provides

• JNI NEON backend (skainet-backend-jni-cpu) for every GGML quant format, runtime dotprod dispatch, threaded packed matmuls, self-installing dispatch (Ship the aarch64-verified NEON kernels to mobile: Apple targets + Android JNI for skainet-backend-native-cpu (measured 21 → 1.0 → 0.11 tok/s cliff) #920 → 0.50–0.52).
• Off-heap mapped weights on Android (AndroidGguf.loader, WeightForm(residency = MAPPED)), memory plan and device fit check before load; a 1.0 GB Q4_K_M LLM decodes at 61–66 ms/step on a Pixel 8a.
• Lstm with explicit state ([Feature]: LSTM layer (Lstm + explicit-state step API) in skainet-lang-core #823); transformer-core KV cache; tape → StableHLO export at billion-parameter scale (0.53.0).

Engine-side P0 (this repo)

• Dense FP32 (and FP16) matmul on the Android JNI tier: attention runs scalar today #1275 — dense FP32/FP16 matmul on the JNI tier (attention is scalar on Android today)
• Native fused LayerNorm / GELU / softmax / conv1d kernels for the JNI and FFM tiers #1276 — native fused LayerNorm / GELU / softmax / conv1d kernels
• Schedule on Android: verify CoroutineSchedule on ART and measure scheduled SDPA on device #1277 — Schedule on Android verified and measured for SDPA
• Native Silero VAD model on Lstm with weights from the upstream ONNX file #1278 — native Silero VAD model on Lstm, weights from the upstream ONNX file
• ONNX importer: LSTM, Gather, Unsqueeze/Squeeze, Cast, Where, If (Silero VAD import as the gate) #1279 — ONNX importer: LSTM, Gather, Unsqueeze/Squeeze, Cast, Where, If
• TensorDump forward hook + TensorCompare CLI: localise numeric divergence between two runs #1280 — TensorDump + TensorCompare for divergence localisation
• FP16 simulation execution context + FP32-accumulating LayerNorm lowering rule (variance overflow repro) #1281 — FP16 simulation context + FP32-accumulating LayerNorm lowering rule

Out of this repo, tracked with the ASR projects

• Whisper v3 128-mel MelConfig in the audio library (HF-exact constants).
• Whisper large-v3-turbo model + self-attention KV cache + timestamp rules + long-form windowing in the Whisper project, on top of the current engine pin.
• Consumer docs: the artifact picker must list skainet-backend-jni-cpu; the AndroidGguf KDoc still says packed tensors heap-stage (fixed in 0.50.0).

Follow-ups already identified (P1, separate issues when P0 lands)

• AndroidGguf.loader over an AssetFileDescriptor so a noCompress GGUF is mapped straight from the APK instead of copied out.
• Windowed mapping to lift the 2 GB single-region cap.
• HF tokenizer.json byte-level BPE loader for Whisper vocab.
• Parakeet TDT cartridge reusing Lstm and the 128-mel config.

Exit criterion

whisper-tiny.en runs on a Pixel-class phone through the JNI tier with mapped weights and the M2-A5 harness names the kernel behind every matmul; the encoder is ≥ 5× faster than the 0.54.0 scalar path; Silero VAD matches ONNX Runtime within 1e-4 with no ORT dependency.