Skip to content

Commit 2fb0fdb

Browse files
committed
build_support: add XLA_TARGET preflight that catches cuda13 before EXLA
A contributor whose shell exports `XLA_TARGET=cuda13` against the bundled `xla 0.9.x` previously hit a `RuntimeError` stacktrace from `deps/exla/mix.exs:116` during dependency compilation, with no indication that the project itself rejects the value: ** (RuntimeError) expected XLA_TARGET to be one of "cpu", "cuda", "rocm", "tpu", "cuda12", but got: "cuda13" (xla 0.9.1) lib/xla.ex:82: XLA.xla_target/0 ... This commit surfaces the same failure with a single readable `** (Mix.Error)` line that names the bad value, the accepted set, the canonical remediation (`export XLA_TARGET=cuda12`), and the relevant troubleshooting guide section. Pieces: - `build_support/xla_target_validator.exs` — shared validator module. Single source of truth for the bundled-xla acceptance list. `Mix.raise/1` with a friendly message on rejection. - `build_support/mix_tasks_compile_xla_env_preflight.exs` — Mix.Task.Compiler that delegates to the validator. Registered in `mix.exs` via `compilers: [:xla_env_preflight] ++ Mix.compilers()`. - `mix.exs` — Code.require_file's both the validator and the compiler eagerly, then calls `XlaTargetValidator.validate!/0` at top level. The eager top-level call is what actually catches the failure mode for `mix test`, `mix deps.compile`, and `mix deps.update`, because the project's own `:compilers` list runs *after* dependency compilation. The compiler in the `:compilers` list provides the conventional `==> xla_env_preflight` step in normal `mix compile` output. - The top-level eager call is gated by an under-`deps/` check, so this project does not impose its XLA_TARGET preference on parent projects that consume it as a transitive dependency. - The compiler module lives in `build_support/` rather than `lib/mix/tasks/compile/` because Mix needs the compiler module loaded before `mix compile` starts; a `lib/`-resident compiler module would create a chicken-and-egg bootstrap problem (Mix would try to find the compiler before `lib/` has had a chance to compile). Tests at `test/build_support/xla_target_validator_test.exs`: - accepting cases: unset, empty, each member of the supported set - rejecting cases: cuda13, arbitrary garbage, case mismatch, trailing whitespace - accessor sanity: `supported_xla_targets/0`, `recommended_xla_target/0`, `raw_xla_target/0` Verified manually against the three scenarios the user reported: - `XLA_TARGET=cuda13 mix compile` -> friendly Mix.Error at mix.exs:30 - `XLA_TARGET=cuda13 mix test` -> friendly Mix.Error at mix.exs:30 - `XLA_TARGET=cuda13 mix deps.compile exla --force` -> friendly Mix.Error at mix.exs:30 All gates green with `XLA_TARGET=cuda12`: - format --check-formatted: clean - compile --warnings-as-errors: clean - test: 231 / 0 / 24 (was 220, +11 new tests) - credo --strict: 1537 mods/funs, 0 issues - dialyzer: 0 errors - docs --warnings-as-errors: clean - trinity.gates --include-hex-build: exit 0 (hex_build_advisory: fail by design, see appendix G of ~/jb/docs/20260519/sakana/) Source policy clean; documentation references "XLA_TARGET=cuda13" only inside fenced code blocks and identifier names, which the dynamic-atom scanner accepts.
1 parent 506e57a commit 2fb0fdb

6 files changed

Lines changed: 352 additions & 1 deletion

File tree

CHANGELOG.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,45 @@
11
# Changelog
22

3+
## 2026-05-21
4+
5+
### Added
6+
- `XlaTargetValidator` — shared `XLA_TARGET` validator at
7+
`build_support/xla_target_validator.exs`. Mirrors the bundled
8+
`xla 0.9.x` acceptance list exactly (`cpu`, `cuda`, `cuda12`,
9+
`rocm`, `tpu`) and raises a single readable `Mix.Error` for any
10+
other value, naming `cuda12` as the recommended remediation and
11+
pointing at `guides/troubleshooting.md`.
12+
- `Mix.Tasks.Compile.XlaEnvPreflight` — Mix compiler that delegates
13+
to `XlaTargetValidator.validate!/0`. Registered in `mix.exs` via
14+
`compilers: [:xla_env_preflight] ++ Mix.compilers()`.
15+
- Top-level eager `XlaTargetValidator.validate!/0` call in
16+
`mix.exs`, gated by an under-`deps/` check. Catches the
17+
`XLA_TARGET=cuda13` failure mode for `mix test`,
18+
`mix deps.compile`, `mix deps.update`, and other tasks that
19+
evaluate `mix.exs` before touching dependency compilation.
20+
Without this gate, the `:compilers` list alone would not help
21+
because the project's compilers run *after* dependency
22+
compilation, and the EXLA-side `RuntimeError` would still appear
23+
first.
24+
- 11 tests in `test/build_support/xla_target_validator_test.exs`
25+
covering: unset, empty, every accepted target, `cuda13`,
26+
arbitrary garbage, case mismatch, trailing whitespace, and the
27+
helper accessors.
28+
29+
### Changed
30+
- `guides/troubleshooting.md``XLA_TARGET=cuda13` section now
31+
reflects that the project surfaces this automatically; the
32+
`mix trinity.env.check` recipe is preserved as a manual
33+
alternative.
34+
35+
### Bug fix surface
36+
- Replaces the EXLA-side `RuntimeError` stacktrace from
37+
`deps/exla/mix.exs:116` with a single readable `** (Mix.Error)`
38+
line that names the bad value, the accepted set, and the
39+
remediation. Verified against `mix compile`, `mix test`, and
40+
`mix deps.compile exla --force`, all of which now fail at
41+
`mix.exs:30` instead of inside the EXLA compile step.
42+
343
## 2026-05-20 (Phase 11 follow-up)
444

545
### Added
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
defmodule Mix.Tasks.Compile.XlaEnvPreflight do
2+
@shortdoc "Validates XLA_TARGET against the bundled xla before project compilation"
3+
@moduledoc """
4+
Mix compiler that validates the `XLA_TARGET` OS environment variable
5+
against the targets accepted by the bundled `xla` dependency before
6+
the project's own source files compile.
7+
8+
## Why this exists
9+
10+
Without this preflight, an unrecognised `XLA_TARGET` (for example
11+
`cuda13` against the bundled `xla 0.9.x`) surfaces as a `RuntimeError`
12+
stacktrace from `deps/exla/mix.exs` during dependency compilation:
13+
14+
** (RuntimeError) expected XLA_TARGET to be one of
15+
"cpu", "cuda", "rocm", "tpu", "cuda12", but got: "cuda13"
16+
(xla 0.9.1) lib/xla.ex:82: XLA.xla_target/0
17+
...
18+
19+
With this preflight, the project surfaces the failure with a single
20+
readable line and a concrete remediation (`export XLA_TARGET=cuda12`).
21+
22+
## How this fits
23+
24+
This compiler runs as part of the *project's* compile step, not during
25+
dependency compilation. Mix compiles dependencies before invoking the
26+
project's `:compilers` list, so this compiler alone would not catch
27+
the dependency-compile failure mode. To close that gap, the same
28+
validation is also invoked eagerly from `mix.exs` at top level, so
29+
`mix test`, `mix deps.compile`, `mix deps.update`, and other tasks
30+
that evaluate `mix.exs` before touching deps all benefit.
31+
32+
The compiler module exists in addition to (not instead of) the
33+
top-level eager check, because it also produces an explicit
34+
`==> xla_env_preflight` step in normal `mix compile` output, which
35+
is the conventional place an Elixir developer looks for build-graph
36+
preflight checks.
37+
38+
## Behaviour
39+
40+
The compiler delegates entirely to `XlaTargetValidator.validate!/0`.
41+
When the validator returns `:ok`, the compiler returns `{:noop, []}`
42+
(no source files were produced, no diagnostics emitted). When the
43+
validator raises, the raise propagates and Mix surfaces it as a
44+
build error.
45+
46+
This compiler is intentionally a no-op in terms of source artefacts.
47+
It exists for its side-effect (the validation) and its diagnostic
48+
surface only.
49+
"""
50+
51+
use Mix.Task.Compiler
52+
53+
@impl Mix.Task.Compiler
54+
def run(_argv) do
55+
XlaTargetValidator.validate!()
56+
{:noop, []}
57+
end
58+
end
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
defmodule XlaTargetValidator do
2+
@moduledoc """
3+
Validates the `XLA_TARGET` OS environment variable against the values
4+
accepted by the bundled `xla` dependency.
5+
6+
Lives under `build_support/` (not `lib/`) so it can be `Code.require_file`d
7+
from `mix.exs` at top level. That makes the same validator usable from:
8+
9+
* `mix.exs` (eager top-level validation, catches `mix test`,
10+
`mix deps.compile`, `mix deps.update`, etc., before EXLA tries to
11+
compile and before the project's own `:compilers` get a chance to
12+
run);
13+
* `Mix.Tasks.Compile.XlaEnvPreflight` (the project's preflight
14+
compiler, surfaced as a normal `mix compile` step);
15+
* `Mix.Tasks.Trinity.Env.Check` (operator-invoked
16+
`mix trinity.env.check`).
17+
18+
The recognised target list is intentionally kept in lock-step with the
19+
bundled `xla` version. As of `xla 0.9.x`, the supported set is
20+
`cpu`, `cuda`, `cuda12`, `rocm`, `tpu`. The newer `xla 0.10.x` adds
21+
`cuda13`; that bump is tracked separately (see
22+
`docs/bumblebee_unpin_playbook.md`).
23+
"""
24+
25+
@supported_xla_targets ["cpu", "cuda", "cuda12", "rocm", "tpu"]
26+
@recommended "cuda12"
27+
28+
@doc "Validates `XLA_TARGET`. Returns `:ok` or raises a `Mix.Error`."
29+
@spec validate!() :: :ok
30+
def validate! do
31+
case raw_xla_target() do
32+
nil -> :ok
33+
"" -> :ok
34+
value when value in @supported_xla_targets -> :ok
35+
value when is_binary(value) -> raise_invalid!(value)
36+
end
37+
end
38+
39+
@doc "Returns the list of XLA_TARGET values accepted by the bundled xla."
40+
@spec supported_xla_targets() :: [String.t()]
41+
def supported_xla_targets, do: @supported_xla_targets
42+
43+
@doc "Returns the recommended XLA_TARGET for CUDA-capable hosts."
44+
@spec recommended_xla_target() :: String.t()
45+
def recommended_xla_target, do: @recommended
46+
47+
@doc "Reads `XLA_TARGET` from the OS environment. Exposed for testing."
48+
@spec raw_xla_target() :: String.t() | nil
49+
def raw_xla_target, do: System.get_env("XLA_TARGET")
50+
51+
defp raise_invalid!(value) do
52+
accepted = Enum.map_join(@supported_xla_targets, ", ", &inspect/1)
53+
54+
Mix.raise(
55+
"XLA_TARGET=#{inspect(value)} is not accepted by the bundled xla 0.9.x. " <>
56+
"Accepted values: #{accepted}. " <>
57+
"Recommended for CUDA hosts: export XLA_TARGET=#{@recommended}. " <>
58+
"Recommended for CPU hosts: unset XLA_TARGET (or use cpu). " <>
59+
"The bundled xla rejects unrecognised targets at compile time, so EXLA " <>
60+
"cannot compile until XLA_TARGET is corrected. See " <>
61+
"guides/troubleshooting.md (\"XLA_TARGET=cuda13 Is Rejected At Compile " <>
62+
"Time\") for the canonical recipe."
63+
)
64+
end
65+
end

guides/troubleshooting.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,36 @@ This applies even on hosts whose installed CUDA toolkit is 13.x. The
179179
`XLA_TARGET` controls which prebuilt XLA artifact is fetched; mismatched
180180
host CUDA installations are tolerated by EXLA via dynamic loading.
181181

182-
The pre-flight task surfaces this fast:
182+
### Automatic preflight
183+
184+
As of 2026-05-21, the project surfaces this automatically via a Mix
185+
preflight that runs from `mix.exs` before any compilation step. An
186+
operator whose shell exports `XLA_TARGET=cuda13` will see a single
187+
readable line instead of an EXLA stacktrace:
188+
189+
```text
190+
** (Mix.Error) XLA_TARGET="cuda13" is not accepted by the bundled xla 0.9.x.
191+
Accepted values: "cpu", "cuda", "cuda12", "rocm", "tpu".
192+
Recommended for CUDA hosts: export XLA_TARGET=cuda12.
193+
Recommended for CPU hosts: unset XLA_TARGET (or use cpu).
194+
The bundled xla rejects unrecognised targets at compile time, so EXLA
195+
cannot compile until XLA_TARGET is corrected.
196+
```
197+
198+
This fires for `mix compile`, `mix test`, `mix deps.compile`,
199+
`mix deps.update`, `mix credo`, `mix dialyzer`, `mix docs`, and any
200+
other task that evaluates `mix.exs`. To bypass it for one command,
201+
prefix the invocation:
202+
203+
```bash
204+
XLA_TARGET=cuda12 mix help
205+
```
206+
207+
### Manual preflight (alternative)
208+
209+
`mix trinity.env.check` is the operator-facing preflight task; it
210+
performs the same validation and additionally accepts
211+
`--require TARGET` and `--artifact-dir DIR` options:
183212

184213
```bash
185214
mix trinity.env.check

mix.exs

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,34 @@ unless Code.ensure_loaded?(DependencySources) do
22
Code.require_file("build_support/dependency_sources.exs", __DIR__)
33
end
44

5+
unless Code.ensure_loaded?(XlaTargetValidator) do
6+
Code.require_file("build_support/xla_target_validator.exs", __DIR__)
7+
end
8+
9+
# Load the XlaEnvPreflight Mix compiler eagerly so Mix can find it before
10+
# the project's own `lib/` tree compiles. Compilers referenced from a
11+
# project's `:compilers` list must be loadable before `mix compile`
12+
# starts; placing the module under `lib/` would create a chicken-and-egg
13+
# bootstrap problem (Mix would look for the compiler module before it
14+
# has had a chance to compile `lib/`).
15+
unless Code.ensure_loaded?(Mix.Tasks.Compile.XlaEnvPreflight) do
16+
Code.require_file("build_support/mix_tasks_compile_xla_env_preflight.exs", __DIR__)
17+
end
18+
19+
# Eager XLA_TARGET preflight: a project's :compilers list runs AFTER
20+
# dependency compilation, so the in-project preflight compiler alone
21+
# would not catch the EXLA-side failure mode. Calling the validator
22+
# here, at mix.exs top level, catches mix test, mix compile,
23+
# mix deps.compile, and mix deps.update -- all of which evaluate
24+
# mix.exs before touching deps.
25+
#
26+
# When this mix.exs is being evaluated as a transitive dependency
27+
# (our directory sits inside some parent project's deps/), defer
28+
# entirely to that parent project's build configuration.
29+
unless __DIR__ |> Path.split() |> Enum.member?("deps") do
30+
XlaTargetValidator.validate!()
31+
end
32+
533
defmodule TrinityCoordinator.MixProject do
634
use Mix.Project
735

@@ -19,6 +47,7 @@ defmodule TrinityCoordinator.MixProject do
1947
start_permanent: Mix.env() == :prod,
2048
package: package(),
2149
docs: docs(),
50+
compilers: [:xla_env_preflight] ++ Mix.compilers(),
2251
elixirc_paths: elixirc_paths(Mix.env()),
2352
dialyzer: [
2453
plt_add_apps: [:mix],
Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
defmodule XlaTargetValidatorTest do
2+
@moduledoc """
3+
Tests for the XLA_TARGET preflight validator.
4+
5+
These tests manipulate `XLA_TARGET` via `System.put_env/2` and
6+
`System.delete_env/1`. Per AGENTS.md, tests may manipulate
7+
environment variables for config-boundary checks, which is exactly
8+
what this is. The suite is `async: false` because `System.put_env/2`
9+
is process-global and would otherwise race with other tests that
10+
read XLA_TARGET.
11+
12+
The validator module itself lives at
13+
`build_support/xla_target_validator.exs` and is loaded eagerly by
14+
`mix.exs`; by the time the test suite runs, it is already loaded
15+
into the Code server.
16+
"""
17+
18+
use ExUnit.Case, async: false
19+
20+
setup do
21+
original = System.get_env("XLA_TARGET")
22+
23+
on_exit(fn ->
24+
case original do
25+
nil -> System.delete_env("XLA_TARGET")
26+
value -> System.put_env("XLA_TARGET", value)
27+
end
28+
end)
29+
30+
:ok
31+
end
32+
33+
describe "supported_xla_targets/0" do
34+
test "matches the bundled xla 0.9.x acceptance list exactly" do
35+
assert XlaTargetValidator.supported_xla_targets() ==
36+
["cpu", "cuda", "cuda12", "rocm", "tpu"]
37+
end
38+
end
39+
40+
describe "recommended_xla_target/0" do
41+
test "is cuda12 (the canonical CUDA lane for the current dep stack)" do
42+
assert XlaTargetValidator.recommended_xla_target() == "cuda12"
43+
end
44+
end
45+
46+
describe "validate!/0 — accepting cases" do
47+
test "accepts unset XLA_TARGET" do
48+
System.delete_env("XLA_TARGET")
49+
assert :ok = XlaTargetValidator.validate!()
50+
end
51+
52+
test "accepts empty XLA_TARGET (treated as unset)" do
53+
System.put_env("XLA_TARGET", "")
54+
assert :ok = XlaTargetValidator.validate!()
55+
end
56+
57+
test "accepts every value in supported_xla_targets/0" do
58+
for target <- XlaTargetValidator.supported_xla_targets() do
59+
System.put_env("XLA_TARGET", target)
60+
assert :ok = XlaTargetValidator.validate!(), "expected #{inspect(target)} to be accepted"
61+
end
62+
end
63+
end
64+
65+
describe "validate!/0 — rejecting cases" do
66+
test "rejects cuda13 (the canonical reported failure) and names cuda12 as remediation" do
67+
System.put_env("XLA_TARGET", "cuda13")
68+
69+
try do
70+
XlaTargetValidator.validate!()
71+
flunk("expected validate!/0 to raise on XLA_TARGET=cuda13")
72+
rescue
73+
e in Mix.Error ->
74+
msg = Exception.message(e)
75+
assert String.contains?(msg, "cuda13")
76+
assert String.contains?(msg, "cuda12")
77+
assert String.contains?(msg, "not accepted by the bundled xla 0.9.x")
78+
assert String.contains?(msg, "guides/troubleshooting.md")
79+
end
80+
end
81+
82+
test "rejects a garbage value and surfaces the actual value in the message" do
83+
System.put_env("XLA_TARGET", "garbage_target_xyz")
84+
85+
try do
86+
XlaTargetValidator.validate!()
87+
flunk("expected validate!/0 to raise on a garbage target")
88+
rescue
89+
e in Mix.Error ->
90+
assert String.contains?(Exception.message(e), "garbage_target_xyz")
91+
end
92+
end
93+
94+
test "rejects case mismatch (xla is case-sensitive)" do
95+
System.put_env("XLA_TARGET", "CUDA12")
96+
97+
try do
98+
XlaTargetValidator.validate!()
99+
flunk("expected validate!/0 to raise on CUDA12 (wrong case)")
100+
rescue
101+
e in Mix.Error ->
102+
assert String.contains?(Exception.message(e), "CUDA12")
103+
end
104+
end
105+
106+
test "rejects whitespace-padded value (xla does not trim)" do
107+
System.put_env("XLA_TARGET", "cuda12 ")
108+
109+
try do
110+
XlaTargetValidator.validate!()
111+
flunk("expected validate!/0 to raise on a padded target")
112+
rescue
113+
e in Mix.Error ->
114+
assert String.contains?(Exception.message(e), "cuda12 ")
115+
end
116+
end
117+
end
118+
119+
describe "raw_xla_target/0" do
120+
test "returns the literal env value (no trimming, no normalisation)" do
121+
System.put_env("XLA_TARGET", "cuda12 ")
122+
assert XlaTargetValidator.raw_xla_target() == "cuda12 "
123+
end
124+
125+
test "returns nil when XLA_TARGET is unset" do
126+
System.delete_env("XLA_TARGET")
127+
assert XlaTargetValidator.raw_xla_target() == nil
128+
end
129+
end
130+
end

0 commit comments

Comments
 (0)