Skip to content

fix: RLE decompression for compressed variable records#47

Merged
Beforerr merged 1 commit into
mainfrom
push-osknqzxmvvzt
Jun 11, 2026
Merged

fix: RLE decompression for compressed variable records#47
Beforerr merged 1 commit into
mainfrom
push-osknqzxmvvzt

Conversation

@Beforerr

Copy link
Copy Markdown
Member

decompress_bytes! had an empty RLE branch: RLE-compressed CVVRs silently
left dest uninitialized. Route RLE through _rle_decompress (output size
is known from the read request) and throw ArgumentError for unsupported
codecs (Huffman/AHUFF) instead of @Assert.

No CDF writer in the ecosystem emits variable-level RLE (cdflib/pycdf
fixtures are gzip-only, which is how this went untested), so tests build
CVVRs by hand.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

decompress_bytes! had an empty RLE branch: RLE-compressed CVVRs silently
left dest uninitialized. Route RLE through _rle_decompress (output size
is known from the read request) and throw ArgumentError for unsupported
codecs (Huffman/AHUFF) instead of @Assert.

No CDF writer in the ecosystem emits variable-level RLE (cdflib/pycdf
fixtures are gzip-only, which is how this went untested), so tests build
CVVRs by hand.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main 9f2791e... main / 9f2791e...
elx/full_load 0.12 ± 0.0043 ms 0.117 ± 0.0042 ms 1.03 ± 0.052
elx/sum_array 3.38 ± 0.36 μs 3.48 ± 0.39 μs 0.971 ± 0.15
elx/sum_lazy 3.33 ± 0.42 μs 3.35 ± 0.45 μs 0.991 ± 0.18
elx/sum_var_access 3.75 ± 0.39 μs 3.82 ± 0.5 μs 0.982 ± 0.16
elx/var_access 0.31 ± 0.001 μs 0.31 ± 0.011 μs 1 ± 0.036
mms/full_load 27.1 ± 2.4 μs 27.1 ± 2.6 μs 1 ± 0.13
mms/sum_array 0.0956 ± 0.0027 s 0.0957 ± 0.0029 s 0.999 ± 0.041
mms/sum_slice 3.59 ± 0.071 ms 3.59 ± 0.46 ms 0.999 ± 0.13
mms/sum_var_access 3.58 ± 0.053 ms 3.59 ± 0.26 ms 0.998 ± 0.074
mms/var_access 0.11 ± 0.01 μs 0.11 ± 0 μs 1 ± 0.091
time_to_load 0.144 ± 0.00072 s 0.146 ± 0.0013 s 0.989 ± 0.01
Memory benchmarks
main 9f2791e... main / 9f2791e...
elx/full_load 3.73 k allocs: 0.167 MB 3.73 k allocs: 0.167 MB 1
elx/sum_array 5 allocs: 27.7 kB 5 allocs: 27.7 kB 1
elx/sum_lazy 5 allocs: 27.7 kB 5 allocs: 27.7 kB 1
elx/sum_var_access 8 allocs: 28.1 kB 8 allocs: 28.1 kB 1
elx/var_access 2 allocs: 0.328 kB 2 allocs: 0.328 kB 1
mms/full_load 0.243 k allocs: 14.8 kB 0.243 k allocs: 14.8 kB 1
mms/sum_array 0.524 k allocs: 31.6 MB 0.524 k allocs: 31.6 MB 1
mms/sum_slice 0.045 k allocs: 1.29 MB 0.045 k allocs: 1.29 MB 1
mms/sum_var_access 0.045 k allocs: 1.29 MB 0.045 k allocs: 1.29 MB 1
mms/var_access 2 allocs: 0.328 kB 2 allocs: 0.328 kB 1
time_to_load 0.145 k allocs: 11 kB 0.149 k allocs: 11.2 kB 0.983

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Beforerr Beforerr merged commit ecb8fd4 into main Jun 11, 2026
5 of 7 checks passed
@Beforerr Beforerr deleted the push-osknqzxmvvzt branch June 11, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant