Commit b21aa5f
authored
fix(driver/cuda): gate FlashInfer Mamba SM90 SSU on min arch, not any arch (#438)
The SM90 SSU vertical/horizontal kernels use CCCL's experimental TMA
intrinsics (cuda::device::experimental::cp_async_bulk_tensor_* /
fence_proxy_async_shared_cta), which <cuda/barrier> exposes only behind
__cccl_lib_experimental_ctk12_cp_async_exposure — defined solely when
__CUDA_MINIMUM_ARCH__ >= 900. That macro is the *minimum* of the whole
CUDA_ARCHITECTURES list and is constant across device passes, so a fat
multi-arch build (e.g. the Docker image's 80;86;89;90) pins it to 800 and
#ifdef's the intrinsics out of every pass — even the sm_90 one — breaking
the build with "namespace cuda::device::experimental has no member ...".
flashinfer_mamba.cu built fine at v0.3.0 (no mamba kernel); the SSU kernels
arrived in v0.4.0 and the gate enabled FLASHINFER_MAMBA_ENABLE_SM90 whenever
*any* targeted arch was 90/100, which a mixed build can never satisfy.
Enable it only when *every* targeted arch is sm_90+ (a pure Hopper/Blackwell
build, e.g. the per-compute-capability release binaries); mixed builds fall
back to the simple SSU algorithm. flashinfer's kAuto dispatch already selects
kSimple on every device (incl. Hopper) when FLASHINFER_MAMBA_ENABLE_SM90 is
unset, so this is correct — just not the TMA-optimized path in a fat binary.
Hopper/FA3 attention gating is unchanged (it compiles fine in fat builds; it
uses CUTLASS TMA, not the CCCL experimental gate).
Unbreaks the slim CUDA Docker image build added in #363. Verified e2e on
yecl-gpu-02 (RTX 4090, CUDA 12.9): image builds (4.11 GB runtime), pie doctor
detects the GPU with cuda_native compiled in, and `pie run text-completion`
returns a coherent completion.1 parent 12b27d8 commit b21aa5f
1 file changed
Lines changed: 12 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
| 122 | + | |
| 123 | + | |
122 | 124 | | |
123 | 125 | | |
124 | 126 | | |
125 | 127 | | |
126 | 128 | | |
127 | 129 | | |
128 | 130 | | |
129 | | - | |
130 | 131 | | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
138 | 136 | | |
139 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
140 | 144 | | |
141 | 145 | | |
142 | 146 | | |
| |||
0 commit comments