Skip to content

BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner#8839

Open
indietyp wants to merge 7 commits into
bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-offrom
bm/be-587-hashql-eval-interner-and-context-restructuring
Open

BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner#8839
indietyp wants to merge 7 commits into
bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-offrom
bm/be-587-hashql-eval-interner-and-context-restructuring

Conversation

@indietyp

@indietyp indietyp commented Jun 8, 2026

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

This PR refactors the HashQL evaluation pipeline to introduce a cleaner separation between the code generation phase and the code execution phase. The previously unified EvalContext is split into CodeGenerationContext (used during MIR compilation and Postgres query preparation) and CodeExecutionContext (used during orchestrated query execution). This makes the data flow and lifecycle of each phase explicit and prevents accidental misuse of context across phase boundaries.

Additionally, a dedicated eval::intern::Interner is introduced for the evaluation stage, which must be constructed from the MIR interner via From to preserve interned pointer identity across the MIR/eval boundary. Input construction for orchestrator tests is also migrated away from using the JSON decoder to directly building typed Value trees using the heap allocator, removing a dependency on the type environment at input-construction time.

The compile_query implementation in the graph API is stubbed out with unimplemented! pending the HashQL API wire-up tracked in BE-537.

🔗 Related links

🔍 What does this change?

  • Renames EvalContext to CodeGenerationContext and introduces a new CodeExecutionContext struct that is produced from CodeGenerationContext via From, carrying only the fields needed at execution time.
  • Adds eval::intern::Interner as a dedicated interner for the evaluation stage, constructed from hashql_mir::intern::Interner to preserve pointer identity.
  • Updates Orchestrator and PostgresCompiler to use the appropriate context type for their respective phases.
  • Replaces JSON-decoder-based input construction in orchestrator tests with direct Value tree construction using heap-allocated Rc and StructBuilder, removing the dependency on the post-lowering type environment for building inputs.
  • Stubs out EntityQuery::compile_query with unimplemented! and suppresses dead_code warnings pending the full HashQL API integration.
  • Updates the EvalDiagnosticCategory enum to replace the graph-feature-gated Graph variant with an Orchestrator variant backed by OrchestratorDiagnosticCategory.
  • Migrates orchestrator test helpers to pass Lowered by value (consuming it) and convert the MIR interner into the eval interner at the appropriate pipeline stage.

🛡 What tests cover this?

  • Existing orchestrator integration tests and eval_postgres compile tests exercise the refactored context types and input construction.
  • Decoder unit tests updated to use Interner::testing for the new eval-stage interner.

❓ How to test this?

  1. Run cargo test across the hashql-eval and hash-graph-api crates.
  2. Run the orchestrator integration tests to confirm query execution still produces correct results against a live Postgres instance.

indietyp added 3 commits June 8, 2026 12:32
chore: add new dependency

chore: format

feat: error module

feat: introduce hashql_eval interner

chore: checkpoint

feat: checkpoint

feat: checkpoint

chore: remove old value module

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

chore: checkpoint

feat: move entity query into its own modul

fix: query request

feat: checkpoint (it compiles!)

feat: checkpoint

feat: checkpoint

feat: checkpoint

fix: issue around cached thunking

feat: covariance for opaque inners

fix: cfgattr serde

chore: remove graph module

fix: merge fuckup
Copilot AI review requested due to automatic review settings June 8, 2026 11:41
@cursor

cursor Bot commented Jun 8, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Entity/subgraph endpoints that send a raw query body now fail with 501 until BE-537 lands; filter-based queries are unchanged. The eval context/interner split touches orchestration and Postgres compilation paths used by integration tests.

Overview
Refactors the HashQL eval pipeline so Postgres compilation and orchestrated execution use different context types, and temporarily disables raw HashQL entity queries in the graph REST API.

Eval pipeline: EvalContext becomes CodeGenerationContext (live-out analysis, diagnostics, alloc) for PostgresCompiler and related codegen. After queries are prepared, execution uses a slimmer CodeExecutionContext derived via From, which Orchestrator now holds. A dedicated eval::intern::Interner is introduced and built from the MIR interner so symbol interning stays consistent across the MIR→eval boundary; decoders and partial hydration use this instead of hashql_mir::intern::Interner. EvalDiagnosticCategory drops the graph compiler variant in favor of Orchestrator.

Graph API: The in-request HashQL parse/lowering/HIR/eval path for EntityQuery::Query is removed; those requests return 501 Not Implemented (BE-537) while Filter queries still compile through unchanged.

Tests / harness: Orchestrator integration tests build seed Inputs as heap-backed Value trees (opaque/struct wrapping) rather than JSON Decoder + type environment; execution helpers convert the MIR interner at compile time and call run_in with the heap allocator.

Reviewed by Cursor Bugbot for commit 6981475. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel

vercel Bot commented Jun 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jun 23, 2026 10:00am
petrinaut Ready Ready Preview Jun 23, 2026 10:00am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jun 23, 2026 10:00am

indietyp commented Jun 8, 2026

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8af8844. Configure here.

Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 71.87500% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.58%. Comparing base (cb9d12c) to head (6981475).

Files with missing lines Patch % Lines
libs/@local/hashql/eval/src/context.rs 52.94% 8 Missing ⚠️
libs/@local/hashql/eval/src/error.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                                          Coverage Diff                                           @@
##           bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of    #8839      +/-   ##
======================================================================================================
- Coverage                                                               59.59%   59.58%   -0.01%     
======================================================================================================
  Files                                                                    1348     1349       +1     
  Lines                                                                  131855   131870      +15     
  Branches                                                                 5945     5946       +1     
======================================================================================================
- Hits                                                                    78575    78574       -1     
- Misses                                                                  52375    52388      +13     
- Partials                                                                  905      908       +3     
Flag Coverage Δ
rust.hashql-ast 87.23% <ø> (ø)
rust.hashql-compiletest 28.40% <ø> (ø)
rust.hashql-eval 74.97% <71.87%> (-0.27%) ⬇️
rust.hashql-hir 89.06% <ø> (ø)
rust.hashql-mir 88.04% <ø> (-0.40%) ⬇️
rust.hashql-syntax-jexpr 94.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@indietyp indietyp changed the title BE-587: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner Jun 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the HashQL evaluation pipeline to explicitly separate code-generation vs code-execution concerns by splitting the former EvalContext into CodeGenerationContext and CodeExecutionContext, and introduces an eval-stage intern::Interner to preserve interned pointer identity across the MIR → eval boundary. Test harnesses and orchestrator utilities are updated to construct typed Value inputs directly (without the JSON decoder/type-env dependency) and to pass the appropriate context/interner types through the compilation/execution stages.

Changes:

  • Split evaluation context into generation/execution contexts and updated Postgres compilation + orchestrator execution call sites accordingly.
  • Added eval-stage intern::Interner (convertible from MIR interner) and migrated decoder/tests to use it.
  • Reworked orchestrator test input construction to build heap-allocated typed Value trees directly; graph API compile_query was stubbed pending BE-537.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
libs/@local/hashql/eval/tests/orchestrator/output.rs Generalized success rendering to support non-Global value allocators.
libs/@local/hashql/eval/tests/orchestrator/main.rs Updated test driver to consume lowered artifacts and pass symbol intern sets into input builder.
libs/@local/hashql/eval/tests/orchestrator/inputs.rs Replaced JSON-decoder-based input construction with direct typed Value construction using heap allocation + struct builders.
libs/@local/hashql/eval/tests/orchestrator/execution.rs Updated lowering/execution glue to use MIR interner → eval interner conversion and the new context split; switched to Orchestrator::run_in.
libs/@local/hashql/eval/src/postgres/projections.rs Added #[inline] on a conversion impl.
libs/@local/hashql/eval/src/postgres/parameters.rs Added #[inline] on a conversion impl.
libs/@local/hashql/eval/src/postgres/mod.rs Switched compiler context type to CodeGenerationContext and updated docs accordingly.
libs/@local/hashql/eval/src/postgres/filter/tests.rs Migrated tests to CodeGenerationContext and eval-stage test interner helper.
libs/@local/hashql/eval/src/postgres/filter/mod.rs Updated filter compiler to reference CodeGenerationContext.
libs/@local/hashql/eval/src/orchestrator/partial.rs Switched to eval-stage Interner in hydration pipeline.
libs/@local/hashql/eval/src/orchestrator/mod.rs Updated orchestrator to depend on CodeExecutionContext; added allocator-parametric run_in usage in call sites.
libs/@local/hashql/eval/src/orchestrator/events.rs Added #[inline] on Default impl.
libs/@local/hashql/eval/src/orchestrator/codec/decode/tests.rs Updated decoder tests to use eval-stage Interner::testing.
libs/@local/hashql/eval/src/orchestrator/codec/decode/mod.rs Switched decoder interner reference to the eval-stage interner.
libs/@local/hashql/eval/src/lib.rs Exported new intern module.
libs/@local/hashql/eval/src/intern.rs Introduced eval-stage Interner convertible from MIR interner (preserves interned identity).
libs/@local/hashql/eval/src/error.rs Replaced graph-gated diagnostic category variant with an orchestrator-backed variant.
libs/@local/hashql/eval/src/context.rs Implemented CodeGenerationContext / CodeExecutionContext split and From conversion.
libs/@local/hashql/core/src/graph/linked.rs Minor Debug impl formatting cleanup + added #[inline] on accessors.
libs/@local/hashql/compiletest/src/suite/eval_postgres.rs Updated compiletest suite to use CodeGenerationContext and eval interner conversion.
libs/@local/graph/api/src/rest/entity_query_request.rs Added lint expectation and stubbed compile_query while HashQL API wiring is pending BE-537.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs
Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs Outdated
@codspeed-hq

codspeed-hq Bot commented Jun 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 80 untouched benchmarks


Comparing bm/be-587-hashql-eval-interner-and-context-restructuring (6981475) with bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of (22e6b09)1

Open in CodSpeed

Footnotes

  1. No successful run was found on bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of (cb9d12c) during the generation of this report, so bfc20c6 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 74f8c73 to a203514 Compare June 8, 2026 11:55
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 8af8844 to 82bb954 Compare June 8, 2026 11:55
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 2d1dad4 to fc88818 Compare June 19, 2026 08:15
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 90ab940 to 7cb3ed3 Compare June 19, 2026 08:15
Copilot AI review requested due to automatic review settings June 19, 2026 15:14
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from fc88818 to 79c6226 Compare June 19, 2026 15:14
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 7cb3ed3 to 233e8eb Compare June 19, 2026 15:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Comment thread libs/@local/hashql/eval/tests/orchestrator/execution.rs
Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 79c6226 to 610b509 Compare June 22, 2026 10:51
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 233e8eb to 0f62210 Compare June 22, 2026 10:51
Copilot AI review requested due to automatic review settings June 22, 2026 12:02
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 0f62210 to e9a1230 Compare June 22, 2026 12:02
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 610b509 to 5bc1aaa Compare June 22, 2026 12:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Comment thread libs/@local/hashql/eval/tests/orchestrator/execution.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants