Skip to content

feat(models): wire the published Vulkan exports and bump nativeLibsVersion - #1479

Merged
msluszniak merged 1 commit into
mainfrom
@ms/wire-vulkan-models
Sep 23, 2026
Merged

msluszniak merged 1 commit into
mainfrom
@ms/wire-vulkan-models

Conversation

@msluszniak

Copy link
Copy Markdown
Member

Description

Wires the 44 Vulkan .pte files published under the HF v0.11.0 tag: style transfer (4), YOLO26 detection (15), YOLO26-seg (15), YOLO26-pose (3), RF-DETR detector/segmentation/keypoint, FastSAM s/x, BlazeFace and SDXS. They become the Android default through the existing variants resolution.

New URLs resolve through NEXT_VERSION_TAG (v0.11.0), since a tag carries only the files cut under it and each model's older exports stay on VERSION_TAG.

nativeLibsVersion moves to 0.10.4: its core carries the constant-segment fix without which RF-DETR fails to load (0x23), and its Vulkan backend is 1.8-3.8x faster on Mali for the conv models added here.

FEATURE_MAP and the native-libraries doc gain vulkan for objectDetection, keypointDetection, instanceSegmentation, styleTransfer, textToImage and segmentAnything.

Five published Vulkan exports are deliberately not wired, because each measured slower than the Android default it would replace: efficientnet-v2-s (fp32 export, 2.4x slower than XNNPACK int8 on Adreno), selfie-segmentation and its landscape checkpoint (1.2x slower on Adreno), lraspp (0.86x vs int8 on Adreno) and ssdlite320 (loses on both GPUs).

Introduces a breaking change?

  • Yes
  • No

Type of change

  • New feature (change which adds functionality)

Tested on

  • iOS
  • Android

Testing instructions

Every one of the 44 URLs was HEAD-checked against HF for a 200 and a plausible content-length. yarn workspace react-native-executorch test (4082 tests), tsc --noEmit and yarn lint pass; the registry's own modelVariants invariants cover the Android default resolution.

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

@msluszniak msluszniak self-assigned this Sep 18, 2026
@msluszniak msluszniak added platform: android Issues and tasks related to Android performance Related to all issues and tasks focused on improving performance labels Sep 18, 2026
@msluszniak
msluszniak requested a review from barhanc September 21, 2026 15:53

@barhanc barhanc left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested selected models from each model family and they work correctly. One note that might be important is that on S20+ I didn't observe any speedup compared to xnnpack version and in some cases Vulkan is noticeably slower (e.g. FastSAM S ~800ms on cpu vs ~1s on Vulkan).

I guess we can merge it after conflict resolution.

@msluszniak

msluszniak commented Sep 22, 2026 •

Copy link
Copy Markdown
Member Author

I tested selected models from each model family and they work correctly. One note that might be important is that on S20+ I didn't observe any speedup compared to xnnpack version and in some cases Vulkan is noticeably slower (e.g. FastSAM S ~800ms on cpu vs ~1s on Vulkan).

I guess we can merge it after conflict resolution.

You didn't observe any speedup at all on any model, or there were some models with regressions and no speedup on s20+?

@barhanc

barhanc commented Sep 22, 2026

Copy link
Copy Markdown
Member

Some models were slightly faster, e.g. the Yolo variants across different tasks (on the order of ~10% and the speedup was stable and consistent). I didn't observe any huge speedups (like the CoreML vs xnnpack case on iOS) and on the large models e.g. FastSAM vulkan was slower + I observed that the UI (of the whole phone not just the app) was slightly laggy when the heavy inference was running (probably because of using the same GPU resources as rendering).

@barhanc

barhanc commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

@msluszniak I've added the S20+ benchmark harness results to #1363 PR comment and they also seem to reproduce the general behavior:

  • Vulkan Supertonic is noticealby slower than xnnpack (~5.6s vs ~5s).
  • Similar speed regression on large Whispers.
  • On small model like PaddleOCR there is a speedup of the order 10-20%.

…rsion

Adds the 44 Vulkan .pte files published under the HF v0.11.0 tag to the
registry: style transfer (4), YOLO26 detection (15), YOLO26-seg (15),
YOLO26-pose (3), RF-DETR detector/segmentation/keypoint, FastSAM s/x,
BlazeFace and SDXS. They become the Android default through the existing
`variants` resolution.

A 0.11 tag carries only the files cut under it, so the new URLs resolve
through NEXT_VERSION_TAG while each model's older exports stay on
VERSION_TAG.

FEATURE_MAP and the native-libraries doc gain `vulkan` for the features
that now reference it, so an app opting in by feature still downloads the
backend its models need.

nativeLibsVersion moves to 0.10.4, which is the first libs release whose
core carries the constant-segment fix these models need to load.
@msluszniak
msluszniak force-pushed the @ms/wire-vulkan-models branch from eb5908e to bcf6521 Compare September 23, 2026 09:45
@msluszniak
msluszniak merged commit b4c60d9 into main Sep 23, 2026
6 checks passed
@msluszniak
msluszniak deleted the @ms/wire-vulkan-models branch September 23, 2026 11:45
msluszniak added a commit that referenced this pull request Sep 25, 2026
…1494) (#1497)

## Description

Cherry-picks five fixes from `main` onto `release/0.10` for the v0.10.3
patch, and pins `nativeLibsVersion` to `0.10.4`.

- 3bf8168 `fix(libs): stop shipping backends the app opted out of`
(#1466)
- e76e0b9 `fix(llm): keep every terminal token out of the chat
response` (#1486)
- daafa64 `fix(llm): pin the model load mode instead of inheriting
upstream defaults` (#1492)
- 10bfbcf `fix(llm): chunk prefill at the bound the graph declares`
(#1491)
- 8406b98 `fix(install): make the native lib download work on Windows`
(#1494)

`v0.10.4-libs` carries the XNNPACK weights-cache fixes (PReLU
use-after-free, and the retained weights that got Kokoro XNNPACK killed
on iPhone). Docs hunks from #1466 and #1494 are left out, per the
patch-release rule. Every touched file matches `main` except
`package.json` and `download-libs.js`, which lacks #1479's Vulkan task
map.

### Introduces a breaking change?

- [ ] Yes
- [x] No

### Type of change

- [x] Bug fix (change which fixes an issue)
- [ ] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)

### Tested on

- [x] iOS
- [x] Android

### Testing instructions

- `yarn jest __tests__`, `yarn typecheck`, `yarn lint`,
`scripts/run-native-tests.sh`
- `apps/speech` Release on iPhone 16 and Galaxy S26 Ultra: Kokoro EN_US
XNNPACK synthesizes, and a 12-layer PReLU `.pte` matches eager output.

### Screenshots

### Related issues

### Checklist

- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have updated the documentation accordingly
- [x] My changes generate no new warnings

### Additional notes

The version bump follows in a separate `Release v0.10.3` PR once this
merges.

---------

Co-authored-by: Mateusz Słuszniak <msluszniak1@gmail.com>
Co-authored-by: Bartosz Hanc <bartosz.hanc02@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Related to all issues and tasks focused on improving performance platform: android Issues and tasks related to Android

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants