Skip to content

Store queries in schema template and warmup plan cache on startup#4157

Open
sergei-pustovykh wants to merge 22 commits into
FoundationDB:mainfrom
sergei-pustovykh:prepare-statements-metadata
Open

Store queries in schema template and warmup plan cache on startup#4157
sergei-pustovykh wants to merge 22 commits into
FoundationDB:mainfrom
sergei-pustovykh:prepare-statements-metadata

Conversation

@sergei-pustovykh

@sergei-pustovykh sergei-pustovykh commented May 12, 2026

Copy link
Copy Markdown
Contributor

This PR introduces a new concept: RecordLayerSchemaTemplate can declare a set of SELECT queries — "stored queries" - which are populated into the RecordMetaData and serialized to protobuf alongside the rest of the schema template.
To initialise schema template, a new clause CREATE QUERY name AS <select> is introduced, valid only inside CREATE SCHEMA TEMPLATE. The body accepts any SELECT-shaped query. DML and DDL bodies are rejected at parse time. The original SQL text is captured (whitespace and casing preserved) and stored on the template. On the wire it surfaces as a stored_queries map<string, string> field on MetaData in record_metadata.proto.
On engine startup, OfflineStoredQueriesProcessor walks the catalog, iterates every schema template, and plans each of its stored queries offline automatically inserting the resulting plans into the RelationalPlanCache.
Planner metrics from the offline path are reported through the new OfflineMetricCollector to a Codahale MetricRegistry, landing alongside online-query metrics under the same event names.

Note: the stored-query name is not consumed by anything yet at runtime. Names are persisted so that a future feature (e.g., named-plan addressing or plan pinning) can pick them up without another metadata migration.

Worth discussing: offline and online planner metrics currently land under the same event names. That makes the existing dashboards just work but conflates startup pre-warming costs with user-query latencies. We should decide whether to keep them merged, split into distinct events or tag them with a dimension so they can be filtered apart.

@sergei-pustovykh sergei-pustovykh added enhancement New feature or request relational issues related to relational FDB and removed relational issues related to relational FDB labels May 13, 2026
@sergei-pustovykh sergei-pustovykh force-pushed the prepare-statements-metadata branch from bcb6d3b to a146137 Compare June 3, 2026 15:42
@sergei-pustovykh sergei-pustovykh force-pushed the prepare-statements-metadata branch from a146137 to 18d346f Compare June 4, 2026 10:27
@sergei-pustovykh sergei-pustovykh changed the title Prepare statements to RecordLayerSchemaTemplate and RecordMetaData Stored queries in RecordLayerSchemaTemplate, startup plan cache warming Jun 15, 2026
@sergei-pustovykh sergei-pustovykh changed the title Stored queries in RecordLayerSchemaTemplate, startup plan cache warming Store queries in schema template and warmup plan cache on startup Jun 15, 2026
@sergei-pustovykh sergei-pustovykh marked this pull request as ready for review June 15, 2026 16:06
@sergei-pustovykh sergei-pustovykh requested a review from hatyo June 15, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant