perf(io): optimize CRC32 with lookup table and reduce ReaderIO heap allocation

[jfeng18]

Summary

• CRC32 checksum: replace bit-by-bit implementation (8 branch-iterations per byte) with a 256-entry lookup table (1 table lookup per byte), ~8x speedup for serialization/deserialization checksum computation. Also fixes a latent sign-extension bug for non-ASCII bytes.
• ReaderIO::MultiReadImpl: replace std::vector<uint64_t> heap allocation with a 1KB stack buffer for counts <= 128, with std::unique_ptr heap fallback for larger counts to maintain exception safety.

Test plan

• Footer tests: 21 assertions passed
• ReaderIO tests: 1,031 assertions passed
• Full unit test suite: 441 cases / 85M assertions passed (release)
• Adversarial review: CRC32 table verified against Python-generated reference (256 entries, all match), known CRC32 check value 0xCBF43926 confirmed
• CI format/lint check

🤖 Generated with Claude Code

[jfeng18]

Hi maintainers, could you please add kind/improvement and version/1.0 labels? I don't have label permissions as a fork contributor. Thanks!

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Require kind label

Wonderful, this rule succeeded.

• label~=^kind/

🟢 Require version label

Wonderful, this rule succeeded.

• label~=^version/

[gemini-code-assist]

Code Review

This pull request optimizes performance in ReaderIO::MultiReadImpl by using a stack-allocated buffer for small offset counts and optimizes Footer::calculate_checksum by implementing a lookup-table-based CRC32 calculation. The review feedback suggests using the custom allocator_ for heap allocations in ReaderIO to maintain consistent memory tracking, and using static constexpr for the CRC32 lookup table to guarantee compile-time initialization.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The current implementation uses new uint64_t[count] to allocate memory on the heap when count exceeds kStackThreshold. This bypasses the custom allocator_ provided to ReaderIO, which is used for memory tracking, custom alignment, or pool allocation in the vsag library. To maintain consistency and ensure proper memory management, please use the custom allocator_ for heap allocations.

    constexpr uint64_t kStackThreshold = 128;
    uint64_t local_buf[kStackThreshold];
    struct AllocatorDeleter {
        Allocator* allocator;
        void operator()(uint64_t* ptr) const {
            allocator->Deallocate(ptr);
        }
    };
    std::unique_ptr<uint64_t[], AllocatorDeleter> heap_buf(nullptr, AllocatorDeleter{allocator_});
    uint64_t* real_offsets;
    if (count <= kStackThreshold) {
        real_offsets = local_buf;
    } else {
        auto* ptr = static_cast<uint64_t*>(allocator_->Allocate(count * sizeof(uint64_t)));
        heap_buf.reset(ptr);
        real_offsets = heap_buf.get();
    }

[jfeng18]

Good point — fixed in commit 97726ea. Now using allocator_->Allocate/Deallocate for consistency with the rest of the IO layer.

[gemini-code-assist]

Using static const for the lookup table inside crc32_table() can be improved by using static constexpr. This guarantees compile-time initialization of the table and avoids any potential runtime overhead or thread-safety initialization guards (magic statics) that the compiler might otherwise generate.

Suggested change

	static const uint32_t*
	crc32_table() {
	static const uint32_t table[256] = {
	static const uint32_t*
	crc32_table() {
	static constexpr uint32_t table[256] = {

[jfeng18]

Done — changed to static constexpr uint32_t table[256] in the same commit (97726ea).

[wxyucs]

Hi @jfeng18, please check the review comments and resolve them.

[jfeng18]

Hi @wxyucs, review comments addressed in commit 97726ea:

• Use allocator_ (high): Fixed — now uses allocator_->Allocate/Deallocate instead of raw new[]/delete[]
• static constexpr (medium): Fixed — changed to static constexpr uint32_t table[256]