Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 0 additions & 136 deletions .github/workflows/export-vulkan.yml

This file was deleted.

87 changes: 85 additions & 2 deletions .github/workflows/export-xnnpack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ on:
description: "Recipe: empty for the config's 8da4w, or fp32 (linears not quantized; the solve only writes the gate's fp32 reference)"
default: ""
type: string
vulkan:
description: Also build each window's Vulkan (GPU) file from the same solved codes, gated through that window's XNNPACK file (GPTQ 8da4w runs of models Vulkan can take)
default: true
type: boolean
publish_hf:
description: Publish to the Hugging Face org in config/pipeline.yaml
default: true
Expand Down Expand Up @@ -58,11 +62,21 @@ jobs:
CONTEXTS: ${{ inputs.contexts }}
MODEL_ID: ${{ inputs.model_id }}
REVISION: ${{ inputs.revision }}
VULKAN: ${{ inputs.vulkan }}
GPTQ: ${{ inputs.gptq }}
QMODE: ${{ inputs.qmode }}
# One entry per window with the memory its Modal Sandbox asks for (export-matrix over
# xnnpack.runner_tiers: sizing.host_need_bytes plus modal.memory_margin). A window past
# what Modal allows is left out here.
# what Modal allows is left out here. With --with-vulkan each entry also says whether
# its job builds the Vulkan file (the model can take Vulkan and the window fits Modal),
# and asks for whichever of the two exports needs more. Never for a rounded or fp32 run:
# a GPU file is built only from solved codes, gated through its XNNPACK twin.
run: |
entries=$(python3 -m pipeline export-matrix "$MODEL_ID" --revision "$REVISION" --backend xnnpack --contexts "$CONTEXTS")
WITH_VULKAN=""
if [ "$VULKAN" = "true" ] && [ "$GPTQ" = "true" ] && { [ -z "$QMODE" ] || [ "$QMODE" = "8da4w" ]; }; then
WITH_VULKAN="--with-vulkan"
fi
entries=$(python3 -m pipeline export-matrix "$MODEL_ID" --revision "$REVISION" --backend xnnpack --contexts "$CONTEXTS" $WITH_VULKAN)
echo "$entries"
echo "entries=$entries" >>"$GITHUB_OUTPUT"
# The solve holds the fp32 model (16 GB for a 4B) and one layer's Hessians: about 12
Expand Down Expand Up @@ -127,6 +141,16 @@ jobs:
# test through ExecuTorch's C++ runner, then the decision gate against the fp32 model's own
# answers, and only then does this job publish. They used to be two jobs because the
# runner fails on aarch64 (finding 30) and the export ran on ARM; on Modal both are x86.
#
# Then, for a window targets marked (export-matrix --with-vulkan: a GPTQ 8da4w run of a
# model Vulkan can take), the Vulkan file, in a Sandbox of its own (vulkan_memory_mib): the
# same solved codes in the same torchao linears, with the XNNPACK report just published as
# its --gate-proxy. The host has no Vulkan runtime, so that verdict is the one it carries,
# and export_xnnpack.check_gate_proxy refuses one built any other way. The CPU file is
# published before the GPU Sandbox starts, so a GPU failure or timeout never costs it;
# publishing it also withdraws, in the same commit, a Vulkan window whose gate was the file
# it replaces (publish.orphaned_gpu_window), so the Hub never holds a GPU file citing a
# gate that is not there. A Vulkan failure fails the job at the end.
runs-on: ubuntu-latest
timeout-minutes: 350
env:
Expand All @@ -140,6 +164,8 @@ jobs:
GPTQ: ${{ inputs.gptq }}
CONTEXT: ${{ matrix.entry.context }}
MEMORY_MIB: ${{ matrix.entry.memory_mib }}
VULKAN: ${{ matrix.entry.vulkan && 'true' || 'false' }}
VULKAN_MEMORY_MIB: ${{ matrix.entry.vulkan_memory_mib }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

Expand All @@ -156,6 +182,7 @@ jobs:
# job had: a solved run with no fp32 reference refuses to publish rather than falling
# through to an unguarded one.
- name: Export, smoke-test and gate on Modal
id: export
run: |
echo "ARTIFACT=xnnpack-${MODEL_ID//\//--}-${CONTEXT}" >>"$GITHUB_ENV"
puts=()
Expand Down Expand Up @@ -205,6 +232,7 @@ jobs:
if-no-files-found: warn

- name: Publish to Hugging Face
id: publish_xnnpack
if: inputs.publish_hf && env.SKIPPED != '1'
run: |
if [ -z "$HF_TOKEN" ]; then
Expand All @@ -214,7 +242,62 @@ jobs:
python -m pipeline publish-hf "$WORK_DIR/out" --backend xnnpack

- name: Publish GitHub release
id: release_xnnpack
if: inputs.publish_release && env.SKIPPED != '1'
env:
GH_TOKEN: ${{ github.token }}
run: python -m pipeline publish-release "$WORK_DIR/out" --backend xnnpack

# Its own Sandbox and its own clock: whatever happens here, the XNNPACK file above is
# already out. Exit 4 is the Vulkan export's own size gate (no file for this window).
- name: Build the Vulkan file on Modal
if: ${{ !cancelled() && env.VULKAN == 'true' && env.SKIPPED != '1' && steps.export.outcome == 'success' && steps.publish_xnnpack.outcome != 'failure' }}
timeout-minutes: 130
run: |
proxy=$(ls "$WORK_DIR"/out/xnnpack/export-report-*.json | head -1)
set +e
python -m pipeline modal-run --memory-mib "$VULKAN_MEMORY_MIB" --timeout-minutes 120 \
--requirements requirements/export-xnnpack.txt \
--put "$WORK_DIR/in/codes.pt" --put "$proxy" --get "$WORK_DIR/vulkan-out" \
--script 'python -m pipeline export-vulkan "$MODEL_ID" --revision "$REVISION" --context "$CONTEXT" \
--out "$OUT" --work "$WORK/vulkan" --codes "$IN/codes.pt" --gate-proxy "$(ls "$IN"/export-report-*.json)"'
rc=$?
set -e
if [ "$rc" -eq 0 ]; then
echo "VULKAN_BUILT=1" >>"$GITHUB_ENV"
python -m pipeline summary "$WORK_DIR/vulkan-out" --backend vulkan >>"$GITHUB_STEP_SUMMARY"
elif [ "$rc" -eq 4 ]; then
echo "::warning title=Vulkan window skipped::$MODEL_ID at $CONTEXT tokens cannot be exported for Vulkan"
else
echo "VULKAN_FAILED=1" >>"$GITHUB_ENV"
echo "::error title=Vulkan export failed::exit $rc; the XNNPACK file is published, this job fails at the end"
fi

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: ${{ !cancelled() && (env.VULKAN_BUILT == '1' || env.VULKAN_FAILED == '1') }}
with:
name: vulkan-${{ env.ARTIFACT }}
path: ${{ env.WORK_DIR }}/vulkan-out
retention-days: 14
if-no-files-found: warn

# Only once its twin is on the Hub: a GPU file is never published citing a gate that
# is not there to read.
- name: Publish the Vulkan file to Hugging Face
id: publish_vulkan
if: ${{ !cancelled() && inputs.publish_hf && env.VULKAN_BUILT == '1' && steps.publish_xnnpack.outcome == 'success' }}
run: python -m pipeline publish-hf "$WORK_DIR/vulkan-out" --backend vulkan

# Likewise only beside its twin's release, and, when the Hub is published to, only once the
# GPU file is there too (a skipped step is not a success).
- name: Publish the Vulkan GitHub release
if: ${{ !cancelled() && inputs.publish_release && env.VULKAN_BUILT == '1' && steps.release_xnnpack.outcome == 'success' && (!inputs.publish_hf || steps.publish_vulkan.outcome == 'success') }}
env:
GH_TOKEN: ${{ github.token }}
run: python -m pipeline publish-release "$WORK_DIR/vulkan-out" --backend vulkan

- name: Fail on a Vulkan export that did not build
if: ${{ !cancelled() && env.VULKAN_FAILED == '1' }}
run: |
echo "::error::the Vulkan export failed after the XNNPACK file was published; see the Vulkan step"
exit 1
14 changes: 9 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,16 +39,20 @@ there and regenerate; never hand-edit the SVGs. `docs/brand/README.md` has the u

- **`watch-hf.yml`** (hourly) β†’ `watch.py`: lists each watched org's newest repos, checks every unseen one (by name
first with no network, then by config), and runs `gh workflow run` for each backend's export workflow. State is
`seen.json` on the orphan **`state` branch**. Dispatch is staged (`watch.STAGES`: XNNPACK for every model, then Vulkan, then QNN, then
MediaTek; the next stage starts when the previous has nothing queued or running) and idempotent (a run in flight for
the model counts as dispatched); `requeue` puts cancelled dispatches back. The first run only seeds the state; existing models are exported via the
`backfill` input. `watch.WORKFLOWS` maps backend β†’ workflow.
`seen.json` on the orphan **`state` branch**. Dispatch is staged (`watch.STAGES`: XNNPACK, with Vulkan in the same
run, for every model, then QNN, then MediaTek; the next stage starts when the previous has nothing queued or running) and
idempotent (a run in flight for the model counts as dispatched); `requeue` puts cancelled dispatches back. The first run
only seeds the state; existing models are exported via the `backfill` input. `watch.WORKFLOWS` maps backend β†’
workflow, and `watch.BUILT_WITH` names a backend built in another's run (Vulkan in XNNPACK's).
- **`solve` job in `export-xnnpack.yml`** (on Modal) β†’ `solve.py`: GPTQ for the int4 codes, once per model because the codes are
per linear and the window changes only the KV cache and the masks. Calibration is `calibration.py`: committed
prompts in the model's own chat template, each continued by the fp32 model's own greedy reply. The codes reach every
window's export as an artifact; `export_with_codes.py` injects them into torchao's tensors before lowering and also
carries the LFM2 state fix (`lfm2_state.py`), which the solve applies too.
- **`export-xnnpack.yml`** / **`export-vulkan.yml`** (each window in a Modal Sandbox via `remote.py`) β†’ `export_xnnpack.py` (`backend="xnnpack"|"vulkan"`, same recipe, XNNPACK or Vulkan delegate; Vulkan gets the structural check): `hub.fetch` β†’ `eligibility.evaluate` β†’ `sizing.choose_context` β†’
- **`export-xnnpack.yml`** (each window in a Modal Sandbox via `remote.py`) β†’ `export_xnnpack.py` (`backend="xnnpack"`,
then in the same Sandbox `backend="vulkan"` from the same codes with the XNNPACK report as `--gate-proxy`, for GPTQ
8da4w runs of models Vulkan can take; there is no Vulkan workflow of its own, since one could only solve again or round
to nearest; Vulkan gets the structural check): `hub.fetch` β†’ `eligibility.evaluate` β†’ `sizing.choose_context` β†’
download β†’ `convert.py` (HF safetensors β†’ ExecuTorch checkpoint layout) β†’ generated `params.json` + `export_llm`
YAML β†’ subprocess `executorch.extension.llm.export.export_llm` β†’ `smoke.py` (greedy generation through the wheel's
`TextLLMRunner`, the same C++ runner the app uses) β†’ `export-report-<window>.json` + `config.json`.
Expand Down
Loading
Loading