Skip to content

Commit 15d3009

Browse files
committed
vulkan: portable CPU-proxy fallback for cross-driver -sm tensor AllReduce
The GPU-side cross-device ordering imports each peer's OPAQUE_FD timeline semaphore, but OPAQUE_FD payloads are driver-private, so the import only works when all devices share a driver (e.g. two NVIDIA GPUs). On mixed drivers or vendors it is out of spec. Add a portable fallback: a helper thread polls each peer's progress/upload timeline and host-signals a local timeline that the consumer's download is parked on (core timeline semaphores plus host signal/wait, no imported handle). Both the chunked pipeline (prefill) and the single-shot (decode) paths are bridged, so proxy mode no longer drops decode to the meta-backend butterfly. A capability gate (vkGetPhysicalDeviceExternalSemaphoreProperties plus a driverUUID match) selects the proxy deterministically on unsupported configs; GGML_VK_COMM_PROXY forces it, and the import try/catch stays as a safety net. Measured within ~4% of the native-import path on decode and on par for prefill, with byte-identical output. Assisted-by: Claude Opus 4.8
1 parent 35de20e commit 15d3009

1 file changed

Lines changed: 188 additions & 103 deletions

File tree

0 commit comments

Comments
 (0)