BE-607: HashQL: Refactor ModuleRegistry into builder/immutable split with allocator-generic stdlib construction

[indietyp]

🌟 What is the purpose of this PR?

Replaces the StandardLibrary type (12.9KB on the stack due to nested SmallVec) with a flat ModuleCache backed by MaybeUninit slots and a FiniteBitSet, and restructures ModuleRegistry construction around a builder pattern (PartialModuleRegistry). Reduces stdlib construction cost by ~20% in instructions and ~60% in L1D cache misses.

🔍 What does this change?

Module cache and construction:

• Replaced nested SmallVec<(TypeId, ModuleDef)> in StandardLibrary with a flat ModuleCache using Box<[MaybeUninit<ModuleDef>]> and FiniteBitSet<CacheId, u64> for initialization tracking
• Split StandardLibrary into StandardLibrary (coordinator), StandardLibraryContext (passed to define methods), and ModuleCache (shared across modules)
• Added CacheId enum with one variant per stdlib module in preorder traversal order, replacing runtime TypeId linear scan with compile-time indices
• ModuleDef tracks emitted_len for exact output capacity when building item slices
• Direct match in build() instead of [Option<ItemKind>; 2].flatten() for item emission
• Item slices allocated directly on the heap (bypassing InternSet) since module item slices are provably unique by ModuleId ownership

Registry restructuring:

• Introduced PartialModuleRegistry as a mutable builder that collects modules during construction
• ModuleRegistry is now immutable after construction, backed by a contiguous IdSlice instead of InternMap
• Removed RwLock from the root namespace (mutable access during construction, shared reference after)
• Added ModuleRegistry::builder() test helper and PartialModuleRegistry::insert_root_module() for tests that inject custom modules

Symbol and type cleanup:

• Replaced all heap.intern_symbol("...") calls in stdlib modules with pre-interned sym:: constants
• Replaced all raw string .opaque("::...") calls with sym::path:: nested constants
• Interned::empty() for zero-generic decl! expansions and special form types
• ModuleDef parameterized by allocator S for bump-scope 2.x support

Benchmark results (Apple Silicon, bump-scope 2.x):

Counter	Before	After	Delta
Instructions	440.0K	351.2K	-20.2%
L1D cache misses	1,213	487	-59.9%
Branch mispredictions	420	347	-17.4%
Walltime	23.49 us	19.57 us	-16.7%

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

• are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

🐾 Next steps

Potential further optimizations identified but not implemented:

• Cache primitive TypeIds (number, boolean, etc.) on TypeBuilder to avoid repeated intern round-trips
• Hoist repeated monomorphic TypeDefs in modules like math and special_form
• Pre-size hash maps for the finite stdlib construction
• Batch generic declaration construction in the decl! macro

🛡 What tests cover this?

• Existing hashql-core test suite (880 unit + 326 integration + 20 doc tests) all pass
• The module::namespace and module::resolver tests exercise custom module construction through the new PartialModuleRegistry builder API

❓ How to test this?

1. cargo test --package hashql-core
2. Verify all 1226 tests pass

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview, Comment	Jun 23, 2026 10:13am
petrinaut	Ready	Preview	Jun 23, 2026 10:13am

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Jun 23, 2026 10:13am

[cursor]

PR Summary

Medium Risk
Large refactor of core module resolution and stdlib registration; behavior should be equivalent but any mistake in builder invariants or cache indexing could break name resolution across the compiler.

Overview
Refactors how HashQL builds and stores the module graph and standard library, aiming for faster startup and simpler lookups after construction.

Module registry: Construction moves to PartialModuleRegistry (provision_module → insert_module → register → finish). The finished ModuleRegistry is immutable: modules live in a contiguous IdSlice, root names in a plain FastHashMap (no RwLock), and module items are heap slices instead of Interned slices. search_by_name uses DenseBitSet for visited modules. Item::absolute_path is removed in favor of absolute_path_rev (expander diagnostics reverse that when formatting paths).

Stdlib: StandardLibrary no longer keeps a large nested SmallVec of **ModuleDef**s. Each stdlib module gets a compile-time CacheId; ModuleCache holds MaybeUninit<ModuleDef> slots with a FiniteBitSet, and cross-module types use cache.request instead of runtime TypeId scans. define takes StandardLibraryContext + allocator-generic scratch; ModuleDef tracks emitted_len for exact item buffers. Stdlib code switches from ad-hoc heap.intern_symbol / string opaque paths to sym:: and sym::path:: constants; zero-generic decl! uses Interned::empty().

Tests / misc: Namespace and resolver tests build custom modules via ModuleRegistry::builder() and insert_root_module. darwin-kperf drops as5-1 from the M5 CPU name mapping.

^{Reviewed by Cursor Bugbot for commit df8bde2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[indietyp]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values #8895
• BE-615: HashQL: Introduce the concept of synthetic closures in the MIR #8894
• BE-607: HashQL: Refactor ModuleRegistry into builder/immutable split with allocator-generic stdlib construction #8887 👈 (View in Graphite)
• BE-538: HashQL: Permission system integration #8882
• BE-71: HashQL: Consolidate AST special-form expansion stages into one #8870
• BE-481: HashQL: Reify null and empty tuple as unit constants #8853
• BE-537, BE-537: hgraph: REST /hashql endpoint with E2E tests #8827
• BE-592: HashQL: Remove old eval graph path and core value types #8841
• BE-590: hgraph: Refactor entity query handlers into entity/query submodule #8840
• BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner #8839
• BE-581, BE-589: HashQL: Introduce pass::lower/pass::place pipeline facades and fix cached constructor closure bug #8838
• BE-585, BE-591: HashQL: Add MIR interpreter test suite and refactor diagnostic categories #8837
• BE-583, BE-584, BE-586: HashQL: Add HeapPool, improve spans, and propagate Critical severity through diagnostics #8836
• BE-582: HashQL: Change opaque types from invariant to covariant #8835
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Pull request overview

Copilot reviewed 36 out of 36 changed files in this pull request and generated 1 comment.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 85a5c9a. Configure here.}

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.