Skip to content

feat: juliac --trim compatibility#43

Merged
Beforerr merged 1 commit into
mainfrom
trim-support
Jun 10, 2026
Merged

feat: juliac --trim compatibility#43
Beforerr merged 1 commit into
mainfrom
trim-support

Conversation

@Beforerr

@Beforerr Beforerr commented Jun 5, 2026

Copy link
Copy Markdown
Member

Makes the open + inspect metadata workload build cleanly under juliac --trim=safe, and adds a regression test for it.

What & why

A --trim=safe build of CDFDataset(file) + metadata access (version, majority, compression, keys) previously failed the trim verifier with 34 errors. Root causes and fixes:

  • Record-size type passed as a value. Every record parser took the size type (Int32/Int64) as a bare/::Type argument. Julia does not specialize on Type/Function/Vararg value args, so it was boxed as abstract Type, making every sizeof/read_be a dynamic call. Now annotated ::Type{T} where T to force specialization.
  • Compression in the type parameter. CDFDataset{CT,FST} encoded file-level compression as a type param, but it is metadata only (the buffer is already decompressed) and nothing dispatches on it. The version×compression product was a 6-way union that exceeds Julia's split limit (4), forcing everything through abstract CDFDataset. Moved to a plain field → 2-way concrete union.
  • open(f, name, mode) do block. Routes through varargs splatting (_apply_iterate) that trim cannot resolve. Replaced with explicit open + try/finally (mmap stays valid after the fd closes, same as before).
  • getproperty lazy branches. Field accesses fell through to the :attrib/:adr branches before getfield, and the optimizer failed to DCE the dead attribute invoke — dragging the heterogeneous attribute machinery (Dict{String,Vector}, runtime julia_type) into builds that never read attributes. Reordered so fields short-circuit first.

Scope

Only the metadata path is trimmable. Variable data reading is inherently dynamic — element type, ndims, and VDR type are read from the file at runtime, so they cannot be statically resolved — and is intentionally out of scope.

@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.06173% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/dataset.jl 90.00% 2 Missing ⚠️
src/records/records.jl 75.00% 1 Missing ⚠️
src/records/vvr.jl 50.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Beforerr Beforerr force-pushed the trim-support branch 9 times, most recently from b47fbfa to 3c7bc7d Compare June 10, 2026 07:29
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main e04e56d... main / e04e56d...
elx/full_load 0.138 ± 0.0046 ms 0.132 ± 0.0049 ms 1.04 ± 0.052
elx/sum_array 3.38 ± 0.32 μs 3.37 ± 0.4 μs 1 ± 0.15
elx/sum_lazy 3.25 ± 1.3 μs 3.23 ± 1.2 μs 1.01 ± 0.55
elx/sum_var_access 4.49 ± 0.46 μs 4.58 ± 0.48 μs 0.98 ± 0.14
elx/var_access 0.881 ± 0.031 μs 0.891 ± 0.03 μs 0.989 ± 0.048
mms/full_load 29.8 ± 2.4 μs 29.3 ± 2.1 μs 1.02 ± 0.11
mms/sum_array 0.0958 ± 0.014 s 0.0958 ± 0.015 s 1 ± 0.21
mms/sum_slice 3.58 ± 0.082 ms 3.59 ± 0.45 ms 0.998 ± 0.13
mms/sum_var_access 3.59 ± 0.074 ms 3.58 ± 0.11 ms 1 ± 0.036
mms/var_access 0.621 ± 0.03 μs 0.631 ± 0.031 μs 0.984 ± 0.068
time_to_load 0.138 ± 0.0021 s 0.141 ± 0.0022 s 0.977 ± 0.021
Memory benchmarks
main e04e56d... main / e04e56d...
elx/full_load 3.78 k allocs: 0.159 MB 3.77 k allocs: 0.159 MB 1
elx/sum_array 5 allocs: 27.7 kB 5 allocs: 27.7 kB 1
elx/sum_lazy 5 allocs: 27.7 kB 5 allocs: 27.7 kB 1
elx/sum_var_access 12 allocs: 28.3 kB 12 allocs: 28.3 kB 1
elx/var_access 6 allocs: 0.531 kB 6 allocs: 0.531 kB 1
mms/full_load 0.25 k allocs: 12.6 kB 0.246 k allocs: 12.5 kB 1.01
mms/sum_array 0.524 k allocs: 31.6 MB 0.524 k allocs: 31.6 MB 1
mms/sum_slice 0.045 k allocs: 1.29 MB 0.045 k allocs: 1.29 MB 1
mms/sum_var_access 0.045 k allocs: 1.29 MB 0.045 k allocs: 1.29 MB 1
mms/var_access 7 allocs: 0.531 kB 7 allocs: 0.547 kB 0.971
time_to_load 0.145 k allocs: 11 kB 0.145 k allocs: 11 kB 1

@Beforerr Beforerr force-pushed the trim-support branch 3 times, most recently from fb5230f to e04e56d Compare June 10, 2026 13:33
Make the "open + inspect metadata" workload build cleanly under
`juliac --trim=safe`:

- Annotate every record-parsing function's record-size argument as
  `::Type{T} where T`. Julia does not specialize on bare/`::Type` value
  args (Type/Function/Vararg heuristic), so they were boxed as abstract
  `Type`, making every `sizeof`/`read_be` a dynamic call the trim verifier
  rejects.
- Move file-level compression from the `CDFDataset{CT,FST}` type parameter
  to a plain field. It is metadata only (the buffer is already decompressed)
  and nothing dispatches on it; as a type param the version×compression
  product was a 6-way union exceeding the split limit, forcing everything
  through abstract `CDFDataset`.
- Replace `open(f, name, mode) do` (varargs splat via `_apply_iterate`,
  trim-hostile) with explicit open + try/finally.
- Reorder `getproperty` so real field accesses short-circuit before the lazy
  `attrib`/`adr` branches, so a trim build that never reads attributes does
  not drag in the heterogeneous attribute machinery.
- Branch on the version literal so the record-size type reaches the loader as
  a concrete type parameter rather than a widened `Type` ternary.

Behavior-preserving: Aqua, JET, and the existing suite pass.

Add test/trim.jl (HTTP.jl/SciML-style): locate juliac, develop the package
into a temp project, build a `--trim=safe` probe, run it. Skips when juliac
is unavailable. Variable *data* reading stays out of scope — eltype, ndims
and VDR type are read from the file at runtime and cannot be statically
resolved.
@Beforerr Beforerr changed the title feat: juliac --trim compatibility for metadata path + trim test feat: juliac --trim compatibility Jun 10, 2026
@Beforerr Beforerr merged commit ea1bf08 into main Jun 10, 2026
6 of 7 checks passed
@Beforerr Beforerr deleted the trim-support branch June 10, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant