Skip to content

ci: run SQL suite over subprocess + http + unix transports#1

Merged
rustyconover merged 1 commit into
mainfrom
ci/multi-transport-sql
Jun 24, 2026
Merged

ci: run SQL suite over subprocess + http + unix transports#1
rustyconover merged 1 commit into
mainfrom
ci/multi-transport-sql

Conversation

@rustyconover

Copy link
Copy Markdown
Contributor

Adds multi-transport (subprocess + http + unix) coverage for the haybarn SQL
suite, with the full suite GREEN over all three transports.

What changed

  • ci/run-integration.sh is now parameterized by TRANSPORT
    (subprocess default, plus http and unix). The offline mocking
    (VGI_GOOGLE_MOCK=1 + a temp VGI_GOOGLE_DISCOVERY_DIR built from
    tests.mock_google) is exported by the script and inherited by the worker
    for all three transports
    — no separate mock server. For http it injects
    INSTALL httpfs FROM core; LOAD httpfs; into the staged tests and boots the
    worker with --http --port 0 --port-file <f>; for unix it boots with
    --unix <sock>. One cleanup() trap captures $? first, then removes the
    discovery dir and kills the worker. A silent-skip guard (tee +
    PIPESTATUS) fails the leg unless the runner reports All tests passed (N>0 assertions …) and nothing was skipped.
  • pyproject.toml: new http extra (vgi-python[http], waitress).
  • google_worker.py: PEP 723 header depends on vgi-python[http].
  • uv.lock: relocked with the http extra (uv sync --frozen --extra http clean).
  • .github/workflows/ci.yml: the integration job is now a
    transport: [subprocess, http, unix] × os: [ubuntu, macos] matrix; sets
    WORKER_CMD, runs uv sync --frozen --extra http. Test + lint jobs unchanged.
  • ci/README.md: documents the transport matrix, offline mocking across all
    transports, httpfs-on-http, the silent-skip guard, and the externalized
    pagination cursor.

Worker

No worker code change needed: the worker's externalized _ScanState cursor
(Google's nextPageToken carried in an ArrowSerializableDataclass) already
round-trips correctly over the stateless HTTP transport, so the centerpiece
pageToken page-boundary round-trip (google_drive(count := 5, page_size := 1)
→ five distinct rows) passes over http unchanged.

Local validation

  • subprocess: GREEN — 31 assertions
  • http: GREEN — 33 assertions (31 + injected httpfs INSTALL/LOAD)
  • unix: GREEN — 31 assertions

🤖 Generated with Claude Code

Parameterize ci/run-integration.sh by a TRANSPORT env (subprocess default,
plus http and unix), and turn the integration job into a
transport x os matrix so the haybarn SQL suite runs over all three VGI
transports in CI.

- run-integration.sh: keep the offline mocking (VGI_GOOGLE_MOCK=1 + a temp
  VGI_GOOGLE_DISCOVERY_DIR built from tests.mock_google) and export both so
  the booted worker inherits them for ALL transports. For http, inject
  `INSTALL httpfs FROM core; LOAD httpfs;` into the staged tests and boot the
  worker with `--http --port 0 --port-file`; for unix, boot with `--unix`.
  One cleanup() trap captures $? first, then removes the discovery dir AND
  kills the worker. Add a silent-skip guard (tee + PIPESTATUS) that fails the
  leg unless `All tests passed (N>0 assertions` and nothing was skipped.
- pyproject.toml: add an `http` extra (vgi-python[http]) for waitress.
- google_worker.py: PEP 723 header now depends on vgi-python[http].
- uv.lock: relocked with the http extra.
- ci.yml: integration job is now transport: [subprocess, http, unix] x
  os: [ubuntu, macos], sets WORKER_CMD, `uv sync --frozen --extra http`.
- ci/README.md: document the transport matrix, offline mocking across all
  transports, httpfs-on-http, the silent-skip guard, and the externalized
  pagination cursor (works over HTTP unchanged).

The worker's externalized _ScanState cursor already round-trips over HTTP, so
the full suite (including the pageToken page-boundary round-trip) is GREEN over
all three transports with no worker changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@rustyconover rustyconover merged commit 035bbce into main Jun 24, 2026
10 checks passed
@rustyconover rustyconover deleted the ci/multi-transport-sql branch June 24, 2026 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant