Add struct types to Halide - #9416
Open
alexreinking wants to merge 12 commits into
Open
alexreinking wants to merge 12 commits into
alexreinking wants to merge 12 commits into
Conversation
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 17:16
9f99483 to
069ce04
Compare
alexreinking
changed the base branch from
alexreinking/hoist-splitting
to
alexreinking/rfactor-hoisting
September 1, 2026 17:32
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 18:21
069ce04 to
50683f7
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 19:06
50683f7 to
e9123a1
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #9416 +/- ##
==========================================
- Coverage 70.92% 70.74% -0.18%
==========================================
Files 261 262 +1
Lines 79989 80594 +605
Branches 19494 19653 +159
==========================================
+ Hits 56733 57018 +285
- Misses 17446 17637 +191
- Partials 5810 5939 +129 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 3, 2026 17:10
e9123a1 to
425cb95
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 4, 2026 09:09
425cb95 to
2740239
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 4, 2026 20:16
1464bc0 to
168ce01
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 11, 2026 09:02
168ce01 to
e272037
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
2 times, most recently
from
September 15, 2026 15:06
166d1d5 to
1444b20
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 20, 2026 17:23
1444b20 to
91c252b
Compare
alexreinking
marked this pull request as ready for review
September 20, 2026 17:23
alexreinking
force-pushed
the
alexreinking/struct-types
branch
2 times, most recently
from
September 21, 2026 17:39
129cd6a to
53dcddc
Compare
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
abadams
reviewed
Sep 21, 2026
alexreinking
force-pushed
the
alexreinking/struct-types
branch
2 times, most recently
from
September 29, 2026 19:08
cf3799e to
011f44d
Compare
alexreinking
removed this pull request from stack #9420
September 29, 2026 20:30
alexreinking
added this pull request to stack #9485
September 29, 2026 20:32
3 of 5 tasks
alexreinking
removed this pull request from stack #9485
September 29, 2026 21:35
alexreinking
added this pull request to stack #9487
September 29, 2026 21:36
Introduce first-class packed struct types (Type::Struct) modeled faithfully in the type system: - A dedicated ABI type code halide_type_struct=5; a struct's packed byte size rides in the halide_type_t reserved field, so struct-typed buffers have correct element size/strides. StructTypeInfo (field layout) is interned like handle metadata, keeping Type at 8 bytes. is_uint()/etc. are honestly false for structs, so no numeric special-casing is needed. - field()/pack_struct() intrinsics with byte-addressed lowering (LowerStructTypes), plus per-field pack_struct ergonomics: an array field is filled by a gather() packet, a gather(extent, gen) generator, a single expression with one swept `_` placeholder (index arithmetic allowed), or a field() copy of a whole same-typed field. - Python bindings for the type, field/pack_struct/gather, and the per-field forms. - Tests: correctness (CPU + GPU), error cases, Python, and an ARM codegen test implementing ggml's q4_0/q8_0 dot product that verifies the packed qs arrays lower to dense 128-bit vector loads. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ConstantBounds.cpp/PyType.cpp: drop two unnecessary-copy clang-tidy findings.
- struct_type_dot_product.cpp: back the test buffers with 16-byte-aligned
storage instead of std::vector, whose allocator doesn't guarantee that on
32-bit ABIs.
- FuseGPUThreadLoops.cpp: relax the GPU shared/global allocation clustering
assert from "both types are powers of two bytes" to the actual requirement
("widest type is a whole multiple of the cluster's byte-granularity type"),
so a non-power-of-two struct size (e.g. 12 bytes) can share GPU memory with
other types.
- LowerStructTypes.cpp: build a packed float field (e.g. a struct's fp16
delta) via a scalar shift/or chain instead of concat_bits's
vector-shuffle-then-reinterpret lowering. Some AArch64 backends (LLVM < 23)
can't legalize a bitcast straight from a vector to a scalar half, and any
chain of pure bitcasts collapses back to that illegal form during
optimization -- only avoiding the vector shape in the first place works.
- struct_type.cpp: move constant_bounds_test() (dead code -- declared and
defined but never called since Halide's old internal-test runner was
removed) into a proper correctness test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- CodeGen_D3D12Compute_Dev: struct-backed groupshared/local arrays are
stored as raw bytes, but Load/Store still consulted the original
Type::Struct(...) for cast/promotion bookkeeping, crashing print_cast's
internal_assert(source_type.is_uint()) since a struct is neither int,
uint, nor float. Treat the storage element as Int(8) (matching what
print_type_maybe_storage actually emits) and scale the groupshared
array's declared element count by the struct's byte size.
- CodeGen_Vulkan_Dev: visit(Allocate) passed Type::Struct(...) directly
to SpvBuilder::declare_type, which has no notion of struct types
("SPIRV: Unsupported type"). Use UInt(8) as the element type for
declaration and later Load/Store bookkeeping instead.
- Serialization/Deserialization: add a Struct TypeCode and StructField
table to the flatbuffers schema so struct types (and their field
layout) round-trip instead of being rejected outright. This is needed
because CI runs JIT compiles through a serialize/deserialize
round-trip for regression testing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Struct-backed allocations are declared as byte-addressable (UInt(8)) arrays in the Vulkan backend, since SPIR-V natively supports 8-bit storage via SPV_KHR_8bit_storage. However, load_from_scalar_index() and store_at_scalar_index() only ever accessed a single storage element and relied on cast_type() (a numeric-conversion helper meant for genuine Halide Cast nodes) to bridge to the field's real type. For fields wider than one byte (e.g. a float32 struct field), this silently read/wrote only 1 of the needed 4 bytes, and converted the value numerically instead of bit-reinterpreting it, producing wrong results (e.g. gpu_struct_type correctness test failures on host-vulkan targets). Add load_from_narrow_storage()/store_at_narrow_storage(), which detect when a value's type is wider than the array's storage type and combine/split the value across the correct number of consecutive byte-sized storage elements via shift+bitwise-or (load) or shift+truncate (store), reinterpreting (bitcasting) the combined bits rather than numerically converting them. Narrower fields (e.g. int8) are unaffected and continue to use the existing single-element path. Verified via spirv-val/spirv-dis on the SPIR-V generated by the gpu_struct_type correctness test (dumped via HL_SPIRV_DUMP_FILE) for both the Register and GPUShared memory-space variants: the generated module is valid and now uses OpBitcast to reinterpret the reassembled bytes instead of OpConvertFToU/OpConvertUToF. Not yet validated by running on real Vulkan hardware. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
HLSL has no native 8-bit integer type, so print_type_maybe_storage() always emits a 32-bit `int` for any 8-bit-typed storage slot. This means struct-backed arrays (used to back Halide struct types) are declared with 32-bit-capacity "byte slots" in HLSL, even though each slot conceptually represents one byte. The scalar Store path performed a plain C-style assignment into these slots, and the scalar Load path used print_cast() (a numeric-cast helper intended for genuine Halide Cast nodes) to bridge back to the field's real type. For fields whose "kind" differs from a plain integer (notably float32 struct fields), this triggered numeric value conversion rather than bit-level reinterpretation: storing 0.5f truncated to 0, then converting 0 back to 0.0f -- exactly matching the `result(1, 0) = 0.000000 instead of 0.500000` failures observed on the host-d3d12compute and host-d3d12compute-hlsl_sm65 buildbot targets. An existing reinterpret mechanism (shared_promotion_required / promotion_str, using asint/asuint/asfloat via hlsl_reinterpret_name()) already existed for groupshared allocations when sm < 62, but was gated too narrowly: it didn't apply to private/register-space struct-backed allocations at all, and even for groupshared arrays the Load side computed the promoted type from op->type instead of the allocation's actual (struct) type, so it never triggered for a Float(32) field load either. Extend both the Store and Load paths so struct-backed allocations (is_struct_backed) always use bit-level reinterpretation (forcing promoted_type to Int(8), matching the emulated storage element) in any memory space, not just groupshared with sm < 62. Verified via HL_DEBUG_CODEGEN dumps of the generated HLSL for the gpu_struct_type correctness test (host-d3d12compute and host-d3d12compute-hlsl_sm65 targets, both the Register and GPUShared memory-space variants, the latter via a temporary git-reverted reorder of the test to reach it): stores now emit e.g. `_producer[0] = asint(_3);` and loads emit `float(asfloat(_producer[0]))`, correctly reinterpreting bits instead of converting values. Not yet validated by running on real D3D12/HLSL hardware (Windows buildbot CI only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 30, 2026 19:41
011f44d to
5462034
Compare
Member
|
Nice to haves:
Far future:
|
abadams
approved these changes
Oct 1, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds struct types to Halide. This allows for inputs and outputs to be defined in terms of popular wire formats, even if they aren't prepared for efficient computation. For example, you can define GGML's
q5_0andq8_0formats like so:Type q5_0 = Type::Struct({{"d", Float(16)}, {"qh", UInt(32)}, {"qs", UInt(8), 16}}); Type q8_0 = Type::Struct({{"d", Float(16)}, {"qs", Int(8), 32}});Then you can write a pipeline that consumes them:
ImageParam x{q4_0_type(), 1, "x"}; ImageParam y{q8_0_type(), 1, "y"}; Var b("b"), k("k"), u("u"); RDom r(0, x.dim(0).extent(), 0, 32, "r"); // block, quant // Dequantize one 4-bit weight to float: quant k is a nibble of the // packed byte k % 16 -- its low nibble for k < 16, its high nibble // otherwise -- biased by -8 to signed [-8, 7], times the block delta. Expr nib = field(x(b), "qs")[k % 16]; // uint8 Func x_wt("x_wt"); x_wt(k, b) = cast<float>(field(x(b), "d")) * (cast<int32_t>(select(k < 16, nib % 16, nib / 16)) - 8); // Dequantize one int8 activation to float: quant k is int8 k, times the // block delta. Func y_wt("y_wt"); y_wt(k, b) = cast<float>(field(y(b), "d")) * cast<int32_t>(field(y(b), "qs")[k]); // Dequantize each side and sum every product. Func qdot{"qdot"}; qdot() = 0.0f; qdot() += x_wt(r[1], r[0]) * y_wt(r[1], r[0]);As you can see, struct types support both individual elements, as well as fixed-size array elements. Fields are accessed with a named
field(expr, "name")intrinsic. The tests include additional cases for struct-typed fields and output structs.This adjusts the
Typeandhalide_type_trepresentations in the following ways:Type: likeHandle, aStructtype has a compile-time known field layout.halide_type_t: structs get their own kind, and the reserved field records the number of bytes in the struct.All layouts are expected to be packed. No padding is inserted around or in between field elements.
Breaking changes
None—it's a new feature.
Checklist