BE-615: HashQL: Introduce the concept of synthetic closures in the MIR#8894
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
PR SummaryMedium Risk Overview Intrinsics move from Supporting changes: Reviewed by Cursor Bugbot for commit d051ddb. Bugbot is set up for automated code reviews on this repo. Configure here. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module #8894 +/- ##
=========================================================================================================
- Coverage 42.67% 26.83% -15.85%
=========================================================================================================
Files 776 652 -124
Lines 68104 50479 -17625
Branches 3908 3324 -584
=========================================================================================================
- Hits 29066 13547 -15519
+ Misses 38682 36777 -1905
+ Partials 356 155 -201
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR extends the HashQL MIR infrastructure to support compiler-synthesized wrapper bodies (“synthetic closures”) and broadens administrative reduction so single-operation closures can be inlined away, enabling optimization cascades (e.g., constant folding after inlining). It also updates several downstream components (pretty-printing, execution/eval contexts, statement placement) and refreshes a large set of MIR/SQL golden outputs to reflect the new inlining/reduction behavior.
Changes:
- Introduces
Source::Synthetic(Symbol)and updates inlining/execution/pretty-printing logic to recognize the new source kind. - Adds
ReductionKind::TrivialClosureto administrative reduction and adds unit/snapshot tests for trivial-closure classification + transitive inlining. - Refactors allocator usage in
MixedBitSetPool/Pooland updates MIR reification to use allocator-aware pools; updates many UI golden files to new optimized MIR/SQL.
Reviewed changes
Copilot reviewed 48 out of 49 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| libs/@local/hashql/mir/tests/ui/pass/post_inline/showcase.stdout | Updated post-inline MIR output to reflect trivial-closure inlining/CFG simplification. |
| libs/@local/hashql/mir/tests/ui/pass/post_inline/nested-branch-elimination.stdout | Updated golden output showing cascade to constant results after inlining. |
| libs/@local/hashql/mir/tests/ui/pass/post_inline/constant-propagation-after-inline.stdout | Updated golden output showing constant propagation after inlining. |
| libs/@local/hashql/mir/tests/ui/pass/post_inline/closure-env-cleanup.stdout | Updated golden output reflecting cleanup after inlining. |
| libs/@local/hashql/mir/tests/ui/pass/post_inline/cascading-simplification.stdout | Updated golden output reflecting aggressive simplification after inlining. |
| libs/@local/hashql/mir/tests/ui/pass/lower/unreachable_only.snap | New snapshot covering degenerate “unreachable-only” body through pipeline. |
| libs/@local/hashql/mir/tests/ui/pass/lower/goto_into_unreachable.snap | New snapshot covering goto into unreachable and propagation/simplification. |
| libs/@local/hashql/mir/tests/ui/pass/inline/too-large-to-inline.stdout | Updated inlining golden output (calls replaced by primitive ops/inputs where appropriate). |
| libs/@local/hashql/mir/tests/ui/pass/inline/heuristic-inline.stdout | Updated heuristic-inlining golden output after new reductions. |
| libs/@local/hashql/mir/tests/ui/pass/inline/filter-with-ctor.stdout | Updated golden output for filter + ctor interaction after new reductions. |
| libs/@local/hashql/mir/tests/ui/pass/inline/filter-aggressive.stdout | Updated aggressive filter inlining golden output. |
| libs/@local/hashql/mir/tests/ui/pass/inline/closure-inline.stdout | Updated closure-inlining golden output (e.g., direct input loads). |
| libs/@local/hashql/mir/tests/ui/pass/administrative_reduction/non-reducible-computation.stdout | Updated AR golden output to reflect new “trivial closure” shape handling. |
| libs/@local/hashql/mir/tests/ui/pass/administrative_reduction/non-reducible-computation.jsonc | Updated test description to reflect trivial-closure distinction. |
| libs/@local/hashql/mir/tests/ui/pass/administrative_reduction/inline_trivial_closure_transitive.snap | New snapshot verifying transitive trivial-closure reduction results. |
| libs/@local/hashql/mir/tests/ui/pass/administrative_reduction/inline_trivial_closure_binary.snap | New snapshot verifying binary-op trivial-closure reduction results. |
| libs/@local/hashql/mir/src/reify/mod.rs | Updates reifier state to use allocator-aware MixedBitSetPool and recycler-in allocator construction. |
| libs/@local/hashql/mir/src/pretty/text.rs | Adds Synthetic printing and adjusts intrinsic printing to use Intrinsic { id, .. }. |
| libs/@local/hashql/mir/src/pass/transform/inline/tests.rs | Updates inline tests to accommodate new Source/intrinsic representation and adds required imports. |
| libs/@local/hashql/mir/src/pass/transform/inline/analysis.rs | Treats Source::Synthetic as InlineDirective::Always. |
| libs/@local/hashql/mir/src/pass/transform/administrative_reduction/tests.rs | Adds classification tests for trivial closures and inlining snapshot tests. |
| libs/@local/hashql/mir/src/pass/transform/administrative_reduction/mod.rs | Updates module docs to include “trivial closures” as a reduction target. |
| libs/@local/hashql/mir/src/pass/transform/administrative_reduction/kind.rs | Extends reduction classification to TrivialClosure for single-operation bodies. |
| libs/@local/hashql/mir/src/pass/tests.rs | Extracts/expands pass-level pipeline tests and snapshot helpers. |
| libs/@local/hashql/mir/src/pass/mod.rs | Moves inline tests module to pass/tests.rs and wires it in under #[cfg(test)]. |
| libs/@local/hashql/mir/src/pass/execution/statement_placement/postgres/mod.rs | Treats Synthetic like other non-filter sources for statement-placement behavior. |
| libs/@local/hashql/mir/src/pass/execution/statement_placement/interpret/mod.rs | Treats Synthetic like other non-filter sources for statement-placement behavior. |
| libs/@local/hashql/mir/src/pass/execution/statement_placement/embedding/mod.rs | Treats Synthetic like other non-filter sources for statement-placement behavior. |
| libs/@local/hashql/mir/src/pass/execution/mod.rs | Skips execution analysis for Synthetic sources similarly to ctor/closure/thunk/intrinsic. |
| libs/@local/hashql/mir/src/lib.rs | Introduces new intrinsic module. |
| libs/@local/hashql/mir/src/intrinsic.rs | Adds IntrinsicId and Intrinsic metadata struct. |
| libs/@local/hashql/mir/src/def.rs | Removes old DefId intrinsic constants, keeping placeholder. |
| libs/@local/hashql/mir/src/builder/body.rs | Updates body-builder defaults/docs and adds body! macro support for [synthetic ...]. |
| libs/@local/hashql/mir/src/body/mod.rs | Changes Source::Intrinsic payload type and adds Source::Synthetic. |
| libs/@local/hashql/eval/tests/ui/postgres/tuple-construction.aux.mir | Updated eval MIR golden output reflecting reordered temps after reductions. |
| libs/@local/hashql/eval/tests/ui/postgres/struct-construction.aux.mir | Updated eval MIR golden output reflecting reordered temps after reductions. |
| libs/@local/hashql/eval/tests/ui/postgres/opaque-passthrough.aux.mir | Updated eval MIR golden output reflecting simplified temp flow. |
| libs/@local/hashql/eval/tests/ui/postgres/mixed-sources-filter.stdout | Updated SQL golden output (JOIN shape changed) after upstream MIR changes. |
| libs/@local/hashql/eval/tests/ui/postgres/list-construction.aux.mir | Updated eval MIR golden output reflecting reordered inputs. |
| libs/@local/hashql/eval/tests/ui/postgres/entity-archived-check.stdout | Updated SQL golden output (JOIN shape changed) after upstream MIR changes. |
| libs/@local/hashql/eval/tests/ui/postgres/dict-construction.aux.mir | Updated eval MIR golden output reflecting reordered inputs. |
| libs/@local/hashql/eval/tests/ui/postgres/comparison-no-cast.aux.mir | Updated eval MIR golden output reflecting reordered inputs. |
| libs/@local/hashql/eval/src/context.rs | Skips eval codegen work for Synthetic sources similarly to other non-filter sources. |
| libs/@local/hashql/core/src/symbol/sym.rs | Adds pointer symbol. |
| libs/@local/hashql/core/src/module/std_lib/graph/entity.rs | Adjusts property signature (temporary) and updates imports/comments. |
| libs/@local/hashql/core/src/module/std_lib/core/mod.rs | Makes core::json module publicly accessible. |
| libs/@local/hashql/core/src/module/std_lib/core/json.rs | Refactors JSON-path types into reusable types submodule helpers. |
| libs/@local/hashql/core/src/collections/pool.rs | Makes Pool allocator-aware and updates MixedBitSetPool alias accordingly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Merging this PR will improve performance by 18.18%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes |
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1001 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1526 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 51 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 107 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 25 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
8efe84c to
a0c8ad1
Compare
00cf591 to
85a5c9a
Compare
85a5c9a to
ceb05c9
Compare
a0c8ad1 to
0ddf199
Compare
0ddf199 to
d051ddb
Compare
ceb05c9 to
df8bde2
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d051ddb. Configure here.
| }) | ||
| { | ||
| return Some(Self::ForwardingClosure); | ||
| return Some(kind); |
There was a problem hiding this comment.
AR ignores intrinsic inline policy
Medium Severity
TrivialClosure classification applies to any single-block body with a primitive final assignment, with no check on Source::Intrinsic or Intrinsic::optimize. Intrinsic fallbacks that are one operation can be administratively inlined at Apply sites even though the inline pass treats intrinsics as never inline and the MIR docs say intrinsics are not inlined.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit d051ddb. Configure here.



🌟 What is the purpose of this PR?
Introduces
Source::Syntheticfor compiler-generated wrapper bodies and extends administrative reduction to handle single-operation function bodies (TrivialClosure).This is groundwork for making intrinsics usable in value position (e.g. passing
+as an argument to a higher-order function). TheSyntheticsource variant will identify wrapper bodies that bridge non-first-class operations into callable closures, andTrivialClosureensures they are always inlined away.🔍 What does this change?
Source::Synthetic(Symbol)to the MIRSourceenum for compiler-synthesized operation wrappersReductionKind::TrivialClosureto administrative reduction, recognizing single-basic-block bodies where a trivial prelude leads to a single non-call operation (Binary,Unary,Aggregate,Input,Load) whose result is returnedSyntheticbodies getInlineDirective::Always(same asCtor)Sourcematches updated: execution pipeline, statement placement, eval context, pretty printer[synthetic sym::path]source to thebody!test macroThe
TrivialClosureclassification is intentionally broader than just synthetic bodies. Any function matching the shape is reducible, which unlocks optimization cascades: single-operation closures get inlined, exposing constant-foldable expressions, enabling branch elimination downstream.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
🐾 Next steps
qualified_pathin reification to generate synthetic wrapper bodies for intrinsics in value position🛡 What tests cover this?
classify_trivial_closure_binary,classify_trivial_closure_unary,classify_non_reducible_non_trivial_preludeinline_trivial_closure_binary,inline_trivial_closure_transitivenested-branch-eliminationnow demonstrates the full optimization cascade)❓ How to test this?
cargo test --package hashql-mir --lib -- administrative_reduction::tests📹 Demo
The
nested-branch-eliminationcompiletest shows the cascade effect: closures containing single comparisons (> 100,< 0) are now inlined by AR, exposing constant-foldable comparisons (50 > 100,50 < 0), which enables the entire branch structure to collapse toreturn "in range".