diff --git a/.gitignore b/.gitignore index 567609b1..cc9fdefd 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,4 @@ build/ + +# generated e2e_test case index (see test/runner/e2e_test.cpp) +test_list diff --git a/gql/CMakeLists.txt b/gql/CMakeLists.txt index fa5a69b3..fd5f5950 100644 --- a/gql/CMakeLists.txt +++ b/gql/CMakeLists.txt @@ -2,6 +2,7 @@ include_directories( ${PROJECT_SOURCE_DIR}/src/include ${CMAKE_BINARY_DIR}/src/include ${PROJECT_SOURCE_DIR}/third_party/antlr4_runtime/src + ${PROJECT_SOURCE_DIR}/third_party/yyjson/src # For _GQL_TO_JSON/_GQL_MAX/_GQL_MIN (json_utils.h pulls in yyjson.h) src/include src/parser ${PROJECT_SOURCE_DIR}/src/include/parser/antlr_parser) # For GQL parser includes diff --git a/gql/README.md b/gql/README.md new file mode 100644 index 00000000..dd0acaf7 --- /dev/null +++ b/gql/README.md @@ -0,0 +1,455 @@ +# GQL extension — ISO GQL compatibility layer for LadybugDB + +`CALL GQL("")` translates GQL to LadybugDB's native Cypher +dialect and executes it. GQL and the equivalent Cypher produce the same result +sets (see the dual-run parity tests in `test/test_files/`). + +This is a **translation layer**, not a native GQL execution engine: LadybugDB's +query engine is Cypher-dialect; this extension maps GQL statements onto it. + +## Usage + +```sql +LOAD EXTENSION '.../libgql.lbug_extension'; +CALL GQL("INSERT (n:Person {name: 'Alice', age: 30})"); +CALL GQL("SELECT n.name AS name, count(*) AS c FROM GRAPH main MATCH (n:Person)"); +``` + +Notes: + +- `CALL GQL(...)` must be the only statement in the query batch (engine + rewrite-path constraint). +- Unmapped GQL constructs fail fast with + `GQL feature not supported: ` — there is no silent pass-through + to the Cypher parser. +- LadybugDB-specific DDL (`CREATE NODE TABLE`, `COPY ...`, ...) can be passed + through `CALL GQL(...)` as a convenience; this is a private extension, not + ISO GQL. + +## Compatibility matrix (graded) + +Format follows the GQL conformance model: mandatory features (by standard +subclause) + optional features (G-feature IDs). ✓ = implemented and covered by +parity tests; ✗ = explicitly rejected with `GQL feature not supported`. + +### Mandatory features + +| Subclause | Feature | Status | Mapping | +|---|---|---|---| +| 8 | Transaction management (START TRANSACTION / COMMIT / ROLLBACK) | ✓ | → `BEGIN TRANSACTION` / `COMMIT` / `ROLLBACK` | +| 14.4 | MATCH / OPTIONAL MATCH | ✓ | pass-through (pattern syntax shared) | +| 14.6 | FILTER | ✓ | → `WHERE` | +| 14.8 | FOR ... IN | ✓ (basic) | → `UNWIND ... AS` (WITH ORDINALITY/OFFSET ✗) | +| 14.9–14.10 | ORDER BY / SKIP / LIMIT | ✓ | OFFSET → `SKIP` | +| 14.11 | RETURN (incl. GROUP BY on RETURN) | ✓ | grouping → `WITH ... RETURN ...` | +| 14.12 | SELECT ... FROM GRAPH ... | ✓ | → `USE GRAPH` + `MATCH ... RETURN ...` | +| 16.15 | GROUP BY / HAVING (incl. implicit grouping) | ✓ | → `WITH keys, aggs WHERE ... RETURN ...` | +| 13.2 | INSERT | ✓ | → `CREATE` | +| 13.3 | SET | ✓ (approx.) | order-independent assignment emulated via `WITH` snapshot when needed; `SET n:Label` ✗ | +| 13.4 | REMOVE | △ approx. | property → `SET n.prop = NULL` (fixed-schema approximation of "property removed"); label removal ✗ | +| 13.5 | DELETE / DETACH DELETE | ✓ | → `DELETE` / `DETACH DELETE` | +| 12.4–12.5 | CREATE GRAPH / DROP GRAPH | ✓ | `ANY` → native `CREATE GRAPH g ANY` (open graph); typed (`CREATE GRAPH g t` / inline `{ ... }`) → `CREATE GRAPH` + `CREATE NODE/REL TABLE` DDL; `IF NOT EXISTS` emulated; `CREATE OR REPLACE` ✗; `LIKE ` / `AS COPY OF ` ✗ | +| 12.6–12.7 | CREATE GRAPH TYPE / DROP GRAPH TYPE | ✓ (basic) | graph type registered in extension memory (see notes); `AS { spec }` / `AS COPY OF ` ✓; `LIKE ` ✗ | +| 7 | Session management | △ partial | `SESSION SET GRAPH` → `USE GRAPH`; SCHEMA / TIME ZONE / PARAMETER ✗ | + +### Optional features (selected) + +| G-feature | Feature | Status | Notes | +|---|---|---|---| +| G035/G036/G037 | Quantified path patterns (`{m,n}`, `*`, `+`, `?`) | ✓ | quantified single edges and single-hop QPPIs `( ()-[]->() ){m,n}` → Cypher var-length `[e*m..n]`; multi-hop / bound interiors `( ()-[]->()-[]->() ){m,n}` expand by repetition (fixed `{n}` unrolls inline; ranged `{m,n}` with `1 ≤ n-m ≤ 4` re-translates the statement per count and joins with `UNION ALL`); interior element variables bind to **lists** (one entry per repetition, seam nodes shared); unbounded multi-hop, lower-bound-0 multi-hop, and node-quantified patterns ✗ (see difference 23) | +| G005/G015–G020 | Path search prefixes (ANY SHORTEST / ALL SHORTEST) | ✓ (single-hop) | → `[e*SHORTEST ...]` / `[e*ALL SHORTEST ...]`; counted `SHORTEST k`, `SHORTEST GROUP(S)`, `ALL/ANY PATHS` ✗ | +| G010–G013 | Path modes (WALK/TRAIL/ACYCLIC/SIMPLE) | ✓ | single var-length slot → `[e*TRAIL ...]` / `[e*ACYCLIC ...]`; multi-hop patterns (and every ACYCLIC/SIMPLE pattern) additionally bind a path variable and filter with `IS_TRAIL`/`IS_ACYCLIC`/`_gql_is_simple` over the whole path — exact ISO semantics (see difference 7); WALK = engine default. SIMPLE is *not* bare engine `*ACYCLIC` (that only keeps intermediate nodes distinct — an endpoint may equal an intermediate); SIMPLE = `_gql_is_simple(p)` (nodes distinct except first=last may coincide), with `*ACYCLIC` as a cheap superset prefilter | +| G074 etc. | Label expressions (`&`, `!`, `|`, `%`) | ✓ (node patterns) | `:A&B`/`:A\|B`/`:!A`/parens → WHERE predicates over `labels(v)` (graph-kind-aware, see difference 13); simple `:Label` unchanged (table pruning); INSERT label sets `:A&B` → `CREATE (n:A:B ...)` on ANY graphs; `%` wildcard → true on table graphs / `size(labels(v)) > 0` on ANY; `IS [NOT] LABELED ` predicates share the same predicate outlet (`IS NOT` negates the whole predicate); edge label expressions ✗ | +| G100 | ELEMENT_ID | ✓ | → `internal_id()` | +| G115 | PROPERTY_EXISTS | ✓ | → `(v.prop IS NOT NULL)` (fixed-schema model of "property present", consistent with difference 1) | +| GA05 | Cast specification | ✓ | `CAST` shared syntax | +| GC03 | CREATE GRAPH TYPE | ✓ (basic) | graph type → node/rel table schema bridge; multi-label node types and `LIKE ` ✗ | + +Quantifier bounds follow the GQL/Neo4j semantics — note that GQL `*` is +**zero**-or-more (`[e*0..]` in Cypher; Cypher's bare `*` is one-or-more), `+` +is one-or-more, `?` is `{0,1}`. Lower bound 0 binds start = end with an empty +edge list. `DIFFERENT EDGES` over a **single** path pattern maps to the TRAIL +machinery (whole-path edge distinctness); over multiple patterns it is +rejected (cross-pattern edge disjointness is deliberately out of scope). +`REPEATABLE ELEMENTS` is dropped (LadybugDB MATCH already allows edge +repetition). Multi-hop QPPIs with **bounded** quantifiers are expanded by +repetition (see difference 23 for the ranged-representation restrictions); +unbounded multi-hop repetition is a translator expressiveness ceiling and is +rejected loudly. + +### Function-name mapping + +Most GQL function names match LadybugDB's catalog directly +(`UPPER`/`LOWER`/`CEILING`/`LN`/`COUNT`/`SUM`/...). Divergent spellings are +mapped (table adapted from Neo4j's `GQLAliasFunctionNameRewriter`, Apache-2.0 — +see `THIRD_PARTY_NOTICES.md`): + +| GQL | LadybugDB | +|---|---| +| `COLLECT_LIST` | `COLLECT` | +| `PERCENTILE_DISC` / `PERCENTILE_CONT` | `PERCENTILEDISC` / `PERCENTILECONT` | +| `CHAR_LENGTH` / `CHARACTER_LENGTH` | `SIZE` | +| `LOCAL_DATETIME` / `ZONED_DATETIME` | `TIMESTAMP` | +| `PATH_LENGTH` | `LENGTH` (path values; quantified element bindings are lists — use `SIZE`) | +| `ELEMENT_ID` | `internal_id` | +| `LOCAL_TIME` / `ZONED_TIME` | ✗ (no TIME type in LadybugDB) | +| `STDDEV_SAMP` / `STDDEV_POP` | ✗ (no such aggregate in LadybugDB) | + +`||` (GQL string concatenation) maps to `+`. + +GQL `MAX`/`MIN` translate to the in-extension aggregates `_gql_max`/`_gql_min` +(registered by this extension), which implement the GQL total order: numbers +compare numerically across INT/DOUBLE, lists lexicographically, and cross-class +ranking follows the TCK pin `null < bool < array < string < number < object` +(see difference 20). Result type follows the input type, so display of native +columns is unchanged. Heterogeneous / nested-list `FOR` sources are wrapped +element-wise in `_gql_to_json(ANY) → JSON` so their element types survive +LadybugDB's list homogenization (see difference 17). + +GQL `SUM`/`AVG` likewise translate to the in-extension aggregates +`_gql_sum`/`_gql_avg`. Typed inputs produce exactly what native `SUM`/`AVG` +produce (INT128/UINT128/DOUBLE, display included — the parity tests compare +against the native aggregates directly); JSON inputs (ANY-graph properties, +`_gql_to_json`-wrapped values) return JSON that preserves int-ness (`sum` of +integers prints `75`, of mixed numbers `3.5`; `avg` is always real, `2.0`). +Nulls (SQL NULL and JSON `null`) are skipped; empty groups are NULL, matching +LadybugDB's native `SUM`. + +On ANY graphs the comparison operators `<` `<=` `>` `>=` `=` `<>` and `ORDER +BY` keys translate to the in-extension predicates `_gql_lt/le/gt/ge/eq/ne` +(`ANY, ANY) → BOOL`) and the sort key `_gql_sortkey(ANY) → STRING`, which +implement the GQL total order (see difference 21). Typed graphs keep the +engine's native operators byte-for-byte. + +### Known semantic differences + +1. **REMOVE property** is approximated as `SET n.prop = NULL` — on LadybugDB's + fixed schema the property column still exists (as NULL); ISO GQL "property + removed" would make `PROPERTY_EXISTS`/`properties()` behave differently. + The layer's `PROPERTY_EXISTS` mapping (G115 → `(v.prop IS NOT NULL)`) is the + same fixed-schema model, so removed reads as not present consistently + (`properties()` still lists every column). +2. **SET assignment order** — GQL evaluates all RHS values before assigning + (order-independent); Cypher `SET` is sequential. The translator snapshots + RHS values via `WITH` when items interfere; otherwise direct mapping is + equivalent. +3. **FROM GRAPH / SESSION SET GRAPH** route through `USE GRAPH`, which is + session-sticky — the session's current graph stays switched after + `CALL GQL` returns. +4. **Transaction-wrapped programs** (`START TRANSACTION COMMIT` in one + call) are rejected: the engine rewrite path would expose the COMMIT message + instead of query rows. Issue BEGIN/COMMIT as separate statements. +5. **NEXT composition** across statements is not supported (Cypher + multi-statements do not share binding scope). +6. `SELECT *` with GROUP BY/HAVING is not supported (use explicit items). +7. **Path modes are enforced exactly** (ISO GQL TRAIL = all edges distinct, + ACYCLIC = all nodes distinct). A single var-length segment maps onto the + engine's `*TRAIL`/`*ACYCLIC` recursive types — note the engine's `*ACYCLIC` + alone only distincts intermediate nodes (start/end unconstrained) — and + every ACYCLIC pattern, plus any TRAIL pattern with more than one edge, also + binds a path variable and filters with `IS_TRAIL`/`IS_ACYCLIC` over the + whole path (closed walks like `1 -> 2 -> 1` are correctly excluded). +8. **GQL has no `LENGTH()`** — ISO GQL spells it `PATH_LENGTH()` for paths + (mapped to LadybugDB `LENGTH`). A bare `length(...)` call is a GQL syntax + error at the parser level. +9. **Edge directions**: GQL's undirected / mixed-direction edge spellings + (`~[e]~`, `<-[e]~`, `~[e]->`, `<->[e]`, ...) collapse to LadybugDB's + any-direction `-[e]-` (a directed property graph has no undirected edges to + distinguish). +10. **Graph types are per-database memory**: `CREATE GRAPH TYPE` registers the + canonicalized type in extension state attached to the open database + (LadybugDB has no graph-type catalog object). Types survive across + `CALL GQL` calls and follow catalog semantics within one database (so + independent sessions/tests do not leak types into each other), but they are + not persisted to WAL — re-run `CREATE GRAPH TYPE` after a restart. Graph + *names* and their schemas are durable; only the type registry is not. +11. **Synthetic primary key**: expanding a graph type adds + `_gql_id SERIAL PRIMARY KEY` to every node table (GQL node types have no key + property; LadybugDB node tables require one). The column is auto-filled on + INSERT and is a regular hidden-ish property; `_gql_id` is rejected as a + user property name. +12. **`NOT NULL` is dropped**: GQL property types may carry `NOT NULL`; LadybugDB + DDL has no NOT NULL constraint, so it is accepted and ignored. +13. **Multi-label semantics depend on the graph kind.** Typed/tabled graphs + give a node exactly one label (= table name), so a conjunction of distinct + labels (`:A&B`) matches nothing and multi-label `INSERT` is rejected (the + engine would silently create nothing there). Open ANY graphs store a label + set (`STRING[]`) and carry real multi-label nodes: `INSERT (n:A&B ...)`, + `MATCH (n:A&B)`, `:A|B`, `:!A` all work there. Compound label expressions + translate to WHERE predicates over `labels(v)` chosen by graph kind (ANY: + `list_contains`; typed: scalar equality); the operators themselves are + graph-kind independent (`&` is AND, `|` is OR on both kinds). Multi-label + node *types* in `CREATE GRAPH TYPE` remain unsupported. Undirected edge + types are rejected too (LadybugDB rel tables are directed). + **`:A:B` colon chains are not GQL label syntax at all** — ISO GQL spells + conjunction only as `&` (`