Repository navigation
Vulkan vs XNNPACK: per model divergences across devices #1488
Description
Tracking issue for cases where the Vulkan vs XNNPACK verdict changes between devices, so one per-platform registry default cannot be right for the whole fleet.
One comment per device pair. Each carries the full paired dataset, the models that flip, and what they imply for defaults.
A Vulkan vs XNNPACK ratio is a GPU/CPU quotient, so it does not transfer across devices: a phone with a strong CPU relative to its GPU reads as "Vulkan loses" on models where another phone reads the opposite. The ratio also moves with input length, because Vulkan carries a fixed per-inference dispatch cost. Peak memory is so far the only axis that holds steady across devices.
Measured with apps/benchmarks (#1363) so every device runs the same protocol and the same synthetic inputs.
Device pairs
- Mali-G76 (Galaxy S10+) vs Mali-G77 MP11 (Galaxy S20+) - 7 of 16 models flip
Activity
[-]Vulkan vs XNNPACK diverges per device: 7 of 16 models flip verdict between Mali-G76 and Mali-G77[/-][+]Vulkan vs XNNPACK: per model divergences across devices[/+]msluszniak commentedon Sep 23, 2026
Mali-G76 (Galaxy S10+) vs Mali-G77 MP11 (Galaxy S20+)
7 of 16 models flip verdict between these two phones.
Root cause
Sorting all 44 measurements by backend:
Mali-G76 is not meaningfully worse than Mali-G77. The Exynos 9820 CPU is much worse than the 990's. Only the denominator moved. The S20+ is therefore the worst case for Vulkan in our fleet, not a broken device.
The CPU gap is not uniform: tiny models are identical across the two phones (minilm 22.6 vs 22.2 ms, clip-text 74.1 vs 74.9), and the gap only opens on sustained work (whisper 1.37 to 1.74, supertonic 1.85). That is sustained-clock behaviour, so short-input benchmarks understate how well Vulkan does on older hardware.
Per model
Whisper is split because the baseline differs: the EN sizes have an
XNNPACK_INT8export and are compared against it, the multilingual sizes only have fp32. Against fp32 Vulkan wins everywhere. "Vulkan vs XNNPACK" has no single answer without naming the XNNPACK precision.Two exceptions to the root cause
paddle-ppocrv6-smallVulkan is 1.61x slower on S10+ while its XNNPACK sibling is only 1.10x. The one place in 44 measurements where the G76 GPU genuinely underperforms. Vulkan is the current Android default, chosen on the S20+, so it regresses on G76.Profiled since: no shader pathology. G76/Adreno on
detectis a flat 8.2x at 640, 960 and 1280 (134.0/16.1, 284.5/35.0, 494.8/60.8 ms), and the shippedconv2d_pw_tiledtile shape is already optimal on G76 (TILE_M=40.79x,TILE_N4=20.24x, both 0.18x, all byte-identical output). Remaining lead is kernel selection: on G76 30 convs fall back toconv2d_im2col_buffer+conv2d_gemm_buffer(215 ms) where Adreno usesconv2d_clamp(19.9 ms), and 60 dw convs pickconv2d_dw_output_tile_3x3_b1x1(33.6 ms) over_3x3(1.9 ms). Needs an S20+ to confirm.Profiling caveat for anyone repeating this: the Vulkan query pool costs Mali +269% and Adreno +13%, so query-pool timings are not comparable across vendors.
whisper-tinymultilingual reads 1.82x on Vulkan and 1.78x on XNNPACK, so both arms moved together. Decode-length difference between devices, not a backend effect. Its cross-device numbers are void; the within-device ratio holds.What this implies for defaults
Peak memory is the only device-independent axis: mpnet 0.61 on both, whisper 0.79/0.81, minilm 0.84/0.87, supertonic 0.36/0.45, clip image 0.41/0.45. Vulkan is roughly half the XNNPACK footprint at nearly the same ratio on both phones.
supertonic is the strongest candidate for a change: 1.84x on G76, 2.12x on Adreno 840, 0.36 to 0.45 of the memory, losing only on the S20+. Current Android default is XNNPACK.
all-minilm / multi-qa-minilm default to
VULKAN_FP16today, chosen on a 254 token benchmark where Vulkan was 2.4x faster. At this harness's ~62 tokens it is 1.50x slower on G76 and 2.52x on G77. Same phone, opposite verdict, so the fix is probably a length-aware choice rather than a flip.mpnet
VULKAN_INT8is 7.3x slower than fp16 on G76 and 8.2x on G77, confirming across two GPU generations that ET Vulkan's quantized tiled kernels are Adreno-tuned and pathological on Mali (they are 1.34x faster than fp16 on Adreno 840). Accuracy was never the issue, 0.9983 to 0.9994 cosine. Correctly shipped as explicit opt-in. Falsified fix, do not retry: raisingTILE_M41 to 4 kept output bit-identical and was 1.6x slower, workgroups dropped 288 to 72, so the bottleneck is occupancy not thebitfieldExtractunpack.whisper-en-small
VULKAN_FP16peaks at 1466 MB on S10+ for no gain over int8 (6012 vs 5830 ms). Heaviest arm measured and an OOM risk on 4 GB devices.Full dataset
image-embeddings/clip-vit-base-patch32-vulkan-fp16image-embeddings/clip-vit-base-patch32-xnnpack-fp32ocr/paddle-ppocrv6-small-vulkanocr/paddle-ppocrv6-small-xnnpackocr/paddle-ppocrv6-small-xnnpack-fp32speech-to-text/whisper-base-vulkan-fp16speech-to-text/whisper-base-vulkan-int8speech-to-text/whisper-base-xnnpack-fp32speech-to-text/whisper-en-base-vulkan-fp16speech-to-text/whisper-en-base-vulkan-int8speech-to-text/whisper-en-base-xnnpack-fp32speech-to-text/whisper-en-base-xnnpack-int8speech-to-text/whisper-en-small-vulkan-fp16speech-to-text/whisper-en-small-vulkan-int8speech-to-text/whisper-en-small-xnnpack-fp32speech-to-text/whisper-en-small-xnnpack-int8speech-to-text/whisper-en-tiny-vulkan-fp16speech-to-text/whisper-en-tiny-vulkan-int8speech-to-text/whisper-en-tiny-xnnpack-fp32speech-to-text/whisper-en-tiny-xnnpack-int8speech-to-text/whisper-small-vulkan-fp16speech-to-text/whisper-small-vulkan-int8speech-to-text/whisper-small-xnnpack-fp32speech-to-text/whisper-tiny-vulkan-fp16speech-to-text/whisper-tiny-vulkan-int8speech-to-text/whisper-tiny-xnnpack-fp32text-embeddings/all-minilm-l6-v2-vulkan-fp16text-embeddings/all-minilm-l6-v2-xnnpack-fp32text-embeddings/all-mpnet-base-v2-vulkan-fp16text-embeddings/all-mpnet-base-v2-vulkan-int8text-embeddings/all-mpnet-base-v2-xnnpack-fp32text-embeddings/clip-vit-base-patch32-text-vulkan-fp16text-embeddings/clip-vit-base-patch32-text-xnnpack-fp32text-embeddings/distiluse-base-multilingual-cased-v2-vulkan-fp16text-embeddings/distiluse-base-multilingual-cased-v2-xnnpack-fp32text-embeddings/multi-qa-minilm-l6-cos-v1-vulkan-fp16text-embeddings/multi-qa-minilm-l6-cos-v1-xnnpack-fp32text-embeddings/multi-qa-mpnet-base-dot-v1-vulkan-fp16text-embeddings/multi-qa-mpnet-base-dot-v1-vulkan-int8text-embeddings/multi-qa-mpnet-base-dot-v1-xnnpack-fp32text-embeddings/paraphrase-multilingual-minilm-l12-v2-vulkan-fp16text-embeddings/paraphrase-multilingual-minilm-l12-v2-xnnpack-fp32text-to-speech/supertonic-vulkan-fp16text-to-speech/supertonic-xnnpack-fp32Method and caveats
RNE
@ms/perf-benchmarks(#1363),nativeLibsVersion0.10.0, release build, 20 timed iterations after 3 warmups, median of the whole pipeline, host thermal gate 37C, one repeat, synthetic deterministic inputs. Only ratios beyond ~1.4x are outside this harness's spread.S10+ (SM-G975F, Android 12, Exynos 9820, Mali-G76) run 2026-09-23: 44 ok, 0 failed, 0 skipped.
S20+ (SM-G986B, Android 13, Exynos 990, Mali-G77 MP11) from #1363 (comment)
S10+ peak is total PSS on a device with ~151 MB MemFree, so mmap'd
.ptepages get evicted and the S10+ MB column is a floor, not a measurement.Unstable rows are all XNNPACK on long work (
whisper-en-small-xnnpack-fp32IQR 2524 ms,supertonic-xnnpack-fp322231 ms), which if anything understates Vulkan there. Device was charging throughout, raising the thermal floor equally for both arms.Two
--onlyids matched nothing and were silently dropped byselectCases: bothxnnpack-8da4wtext-embedding arms. Harness bug, and it leaves the distiluse and paraphrase 8da4w comparisons incomplete.