Context
tests/surgery_cli.rs::empty_surgery_pipeline_matches_baseline_output runs the mlxcel CLI twice as separate processes on qwen2.5-0.5b-4bit (a baseline run vs a surgery pipeline with 0 operations) and asserts byte-equal generated output over 20 greedy tokens.
The property under test (an empty surgery pipeline must not alter output) is still sound; only the comparison window is fragile.
Evidence
Observed on GB10 (CUDA 13.0, MLX pin 57c66cac, 2026-07-10, main + PR #727 branch): the two runs agree for 19 of 20 tokens and diverge on the final token.
The divergent token lies after the model emits <|im_end|> and <|endoftext|>, i.e. in the post-EOS free-running tail where logits are near-tie.
Rerunning the single test on the same binary gives pass/fail/pass (2 of 3), so the comparison is nondeterministic across process invocations on the CUDA backend. Reduction-order variance in GPU kernels is enough to flip a near-tie argmax.
The failure aborts the remaining cargo test targets when it lands mid-suite.
Left/right sample from the failing assertion:
left: ...Human: How can I improve my writing
right: ...Human: How can I improve my public
Suggested Fix
Stop generation at the first EOS in this test. The tail after <|endoftext|> carries no signal for the surgery no-op property being tested, so truncating there removes the nondeterministic window without weakening the assertion.
Alternatives considered: compare only tokens up to and including the first EOS, or generate fewer tokens so the comparison window ends before EOS.
Acceptance Criteria
Context
tests/surgery_cli.rs::empty_surgery_pipeline_matches_baseline_outputruns the mlxcel CLI twice as separate processes on qwen2.5-0.5b-4bit (a baseline run vs a surgery pipeline with 0 operations) and asserts byte-equal generated output over 20 greedy tokens.The property under test (an empty surgery pipeline must not alter output) is still sound; only the comparison window is fragile.
Evidence
Observed on GB10 (CUDA 13.0, MLX pin 57c66cac, 2026-07-10, main + PR #727 branch): the two runs agree for 19 of 20 tokens and diverge on the final token.
The divergent token lies after the model emits
<|im_end|>and<|endoftext|>, i.e. in the post-EOS free-running tail where logits are near-tie.Rerunning the single test on the same binary gives pass/fail/pass (2 of 3), so the comparison is nondeterministic across process invocations on the CUDA backend. Reduction-order variance in GPU kernels is enough to flip a near-tie argmax.
The failure aborts the remaining
cargo testtargets when it lands mid-suite.Left/right sample from the failing assertion:
Suggested Fix
Stop generation at the first EOS in this test. The tail after
<|endoftext|>carries no signal for the surgery no-op property being tested, so truncating there removes the nondeterministic window without weakening the assertion.Alternatives considered: compare only tokens up to and including the first EOS, or generate fewer tokens so the comparison window ends before EOS.
Acceptance Criteria
empty_surgery_pipeline_matches_baseline_outputno longer compares post-EOS tokens (generation stops at first EOS, or the comparison is truncated at first EOS)