Skip to content

Add the temporal-covering projection generator#25

Open
estebanzimanyi wants to merge 2 commits into
MobilityDB:masterfrom
estebanzimanyi:feat/covering-projection
Open

Add the temporal-covering projection generator#25
estebanzimanyi wants to merge 2 commits into
MobilityDB:masterfrom
estebanzimanyi:feat/covering-projection

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

Stacks on #24 (the temporal-covering descriptor). Until #24 merges, this PR's diff shows both commits; review/merge #24 first.

Projects the catalog's temporalCovering block (added by #24) onto the canonical, language-agnostic covering-column contract — per temporal type, the ordered covering columns with the fully-composed MEOS expression that derives each from the value. This is the reference rendering every binding generator (PyMEOS, JMEOS, MobilityDuck, MobilitySpark, …) reproduces in its own idiom, so a temporal table prunes the same way on every platform (Iceberg manifest + Parquet row-group min/max).

What it adds

  • generator/covering.pybuild_covering_projection(catalog), pure dict → dict. Composes each covering column's expression: a box column → accessor(box_from(VALUE)), a value column → accessor(VALUE). VALUE is the temporal column the binding substitutes.
  • generate_covering.py — CLI mirroring generate_movfeat_openapi.py; reads output/meos-idl.json, writes output/meos-covering-projection.json.
  • tests/test_covering_projection.py — unit + live gate.

Mirrors the existing projection idiom (generator/movfeat.py + root CLI + test).

Projected output (sample)

tgeompoint (STBOX)
  xmin double      stbox_xmin(tspatial_to_stbox(VALUE))
  ...  zmin/zmax [hasZ]
  tmin timestamptz stbox_tmin(tspatial_to_stbox(VALUE))
  srid int         tspatial_srid(VALUE)
tfloat (TBOX)
  vmin double      tbox_xmin(tnumber_to_tbox(VALUE))
  tmin timestamptz tbox_tmin(tnumber_to_tbox(VALUE))

Verification

  • 5 projection tests green; live gate against the real catalog: all 11 covered types project to well-formed, value-composed expressions.
  • Python-only (no compiled artifact); the build/test is the test suite, green this turn.

Fold meta/temporal-covering.json into meos-idl.json as temporalCovering, the
single codegen source of truth for projecting a MEOS temporal column into
Parquet/Iceberg covering columns (RFC #870 TemporalParquet / #913 Temporal
Data Lake). Every binding generates the identical covering schema, so a
temporal table prunes the same way on every platform (Iceberg manifest +
Parquet row-group min/max) with no spatial-aware engine.

The descriptor is keyed by temporal-type class, not by type: the spatial
class derives an STBOX covering (xmin..zmax, tmin/tmax, srid) via
tspatial_to_stbox; the number class derives a TBOX covering (vmin/vmax,
tmin/tmax) via tnumber_to_tbox. parser/covering.py attaches it with a
byType index and the set of referenced C symbols; tools/covering_parity.py
audits that every symbol is exported by the catalog and every covered type
is a real MeosType, reporting any miss as a worklist rather than a pass.

Mirrors the portable-aliases infrastructure (descriptor + schema + parser
hook + parity audit + tests + doc).
Project the catalog's temporalCovering block onto the canonical,
language-agnostic covering-column contract: per temporal type, the ordered
covering columns with the fully-composed MEOS expression that derives each
from the value (e.g. tgeompoint xmin = stbox_xmin(tspatial_to_stbox(VALUE)),
srid = tspatial_srid(VALUE); tfloat vmin = tbox_xmin(tnumber_to_tbox(VALUE))).

Every binding generator renders this same contract in its own idiom — a
DuckDB GENERATED column, a Spark UDF projection, a PyMEOS writer — so a
temporal table prunes the same way on every platform (Iceberg manifest +
Parquet row-group min/max). The VALUE placeholder is the temporal column
reference the binding substitutes.

generator/covering.py is the pure dict->dict projection; generate_covering.py
is the CLI writing output/meos-covering-projection.json. Mirrors the
generate_movfeat_openapi.py / generator/movfeat.py projection idiom.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant