feat(models): wire the published Vulkan exports and bump nativeLibsVersion - #1479
Conversation
There was a problem hiding this comment.
I tested selected models from each model family and they work correctly. One note that might be important is that on S20+ I didn't observe any speedup compared to xnnpack version and in some cases Vulkan is noticeably slower (e.g. FastSAM S ~800ms on cpu vs ~1s on Vulkan).
I guess we can merge it after conflict resolution.
You didn't observe any speedup at all on any model, or there were some models with regressions and no speedup on s20+? |
|
Some models were slightly faster, e.g. the Yolo variants across different tasks (on the order of ~10% and the speedup was stable and consistent). I didn't observe any huge speedups (like the CoreML vs xnnpack case on iOS) and on the large models e.g. FastSAM vulkan was slower + I observed that the UI (of the whole phone not just the app) was slightly laggy when the heavy inference was running (probably because of using the same GPU resources as rendering). |
|
@msluszniak I've added the S20+ benchmark harness results to #1363 PR comment and they also seem to reproduce the general behavior:
|
…rsion Adds the 44 Vulkan .pte files published under the HF v0.11.0 tag to the registry: style transfer (4), YOLO26 detection (15), YOLO26-seg (15), YOLO26-pose (3), RF-DETR detector/segmentation/keypoint, FastSAM s/x, BlazeFace and SDXS. They become the Android default through the existing `variants` resolution. A 0.11 tag carries only the files cut under it, so the new URLs resolve through NEXT_VERSION_TAG while each model's older exports stay on VERSION_TAG. FEATURE_MAP and the native-libraries doc gain `vulkan` for the features that now reference it, so an app opting in by feature still downloads the backend its models need. nativeLibsVersion moves to 0.10.4, which is the first libs release whose core carries the constant-segment fix these models need to load.
eb5908e to
bcf6521
Compare
…1494) (#1497) ## Description Cherry-picks five fixes from `main` onto `release/0.10` for the v0.10.3 patch, and pins `nativeLibsVersion` to `0.10.4`. - 3bf8168 `fix(libs): stop shipping backends the app opted out of` (#1466) - e76e0b9 `fix(llm): keep every terminal token out of the chat response` (#1486) - daafa64 `fix(llm): pin the model load mode instead of inheriting upstream defaults` (#1492) - 10bfbcf `fix(llm): chunk prefill at the bound the graph declares` (#1491) - 8406b98 `fix(install): make the native lib download work on Windows` (#1494) `v0.10.4-libs` carries the XNNPACK weights-cache fixes (PReLU use-after-free, and the retained weights that got Kokoro XNNPACK killed on iPhone). Docs hunks from #1466 and #1494 are left out, per the patch-release rule. Every touched file matches `main` except `package.json` and `download-libs.js`, which lacks #1479's Vulkan task map. ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [x] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [x] iOS - [x] Android ### Testing instructions - `yarn jest __tests__`, `yarn typecheck`, `yarn lint`, `scripts/run-native-tests.sh` - `apps/speech` Release on iPhone 16 and Galaxy S26 Ultra: Kokoro EN_US XNNPACK synthesizes, and a 12-layer PReLU `.pte` matches eager output. ### Screenshots ### Related issues ### Checklist - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have updated the documentation accordingly - [x] My changes generate no new warnings ### Additional notes The version bump follows in a separate `Release v0.10.3` PR once this merges. --------- Co-authored-by: Mateusz Słuszniak <msluszniak1@gmail.com> Co-authored-by: Bartosz Hanc <bartosz.hanc02@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Description
Wires the 44 Vulkan
.ptefiles published under the HFv0.11.0tag: style transfer (4), YOLO26 detection (15), YOLO26-seg (15), YOLO26-pose (3), RF-DETR detector/segmentation/keypoint, FastSAM s/x, BlazeFace and SDXS. They become the Android default through the existingvariantsresolution.New URLs resolve through
NEXT_VERSION_TAG(v0.11.0), since a tag carries only the files cut under it and each model's older exports stay onVERSION_TAG.nativeLibsVersionmoves to0.10.4: its core carries the constant-segment fix without which RF-DETR fails to load (0x23), and its Vulkan backend is 1.8-3.8x faster on Mali for the conv models added here.FEATURE_MAPand the native-libraries doc gainvulkanforobjectDetection,keypointDetection,instanceSegmentation,styleTransfer,textToImageandsegmentAnything.Five published Vulkan exports are deliberately not wired, because each measured slower than the Android default it would replace:
efficientnet-v2-s(fp32 export, 2.4x slower than XNNPACK int8 on Adreno),selfie-segmentationand its landscape checkpoint (1.2x slower on Adreno),lraspp(0.86x vs int8 on Adreno) andssdlite320(loses on both GPUs).Introduces a breaking change?
Type of change
Tested on
Testing instructions
Every one of the 44 URLs was HEAD-checked against HF for a 200 and a plausible content-length.
yarn workspace react-native-executorch test(4082 tests),tsc --noEmitandyarn lintpass; the registry's ownmodelVariantsinvariants cover the Android default resolution.Checklist