Skip to content

Commit b145a8f

Browse files
rustyconoverclaude
rustyconover
and
claude
committed
ci: run SQL E2E via standalone haybarn + signed vgi community extension
Replace the e2e-sql job (which installed haybarn-unittest via uv tool and ran `make test-sql` on a clean runner, where `LOAD vgi;` failed because the vgi extension was never provisioned) with the proven vgi-scikit-learn pattern: - resolve-haybarn job resolves the latest Query-farm-haybarn/haybarn release. - integration job (ubuntu + macos) builds target/release/units-worker, downloads the prebuilt standalone haybarn_unittest asset, and runs ci/run-integration.sh. - ci/run-integration.sh stages the suite through ci/preprocess-require.awk, warms the extension cache (`INSTALL vgi FROM community;` — what makes the tests' explicit `LOAD vgi;` succeed), then runs the whole suite in one invocation. build-test (fmt/clippy/build/unit) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent bb03c59 commit b145a8f

4 files changed

Lines changed: 164 additions & 12 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 50 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ concurrency:
1010
group: ci-${{ github.ref }}
1111
cancel-in-progress: true
1212

13+
permissions:
14+
contents: read
15+
1316
env:
1417
CARGO_TERM_COLOR: always
1518
RUST_BACKTRACE: 1
@@ -40,9 +43,31 @@ jobs:
4043
- name: Build worker (release)
4144
run: cargo build --release --bin units-worker
4245

43-
e2e-sql:
44-
name: SQL end-to-end (DuckDB sqllogictest)
46+
# Resolve the latest published haybarn release once, so the whole matrix tests
47+
# the same version (and we never hardcode/pin it).
48+
resolve-haybarn:
4549
runs-on: ubuntu-latest
50+
outputs:
51+
release: ${{ steps.r.outputs.release }}
52+
steps:
53+
- id: r
54+
run: |
55+
REL=$(gh release view --repo Query-farm-haybarn/haybarn --json tagName --jq .tagName)
56+
echo "release=$REL" >> "$GITHUB_OUTPUT"
57+
echo "Latest haybarn release: $REL"
58+
env:
59+
GH_TOKEN: ${{ github.token }}
60+
61+
integration:
62+
name: SQL end-to-end (DuckDB sqllogictest)
63+
needs: resolve-haybarn
64+
strategy:
65+
fail-fast: false
66+
matrix:
67+
include:
68+
- { os: ubuntu-latest, asset: haybarn_unittest-linux-amd64.zip }
69+
- { os: macos-latest, asset: haybarn_unittest-osx-arm64.zip }
70+
runs-on: ${{ matrix.os }}
4671
steps:
4772
- uses: actions/checkout@v7
4873

@@ -51,15 +76,28 @@ jobs:
5176

5277
- uses: Swatinem/rust-cache@v2
5378

54-
- name: Install uv
55-
uses: astral-sh/setup-uv@v7
56-
57-
- name: Install haybarn-unittest
58-
run: uv tool install haybarn-unittest
79+
- name: Build worker (release)
80+
run: cargo build --release --bin units-worker
5981

60-
- name: SQL E2E (build release worker + run sqllogictest suite)
61-
# `make test-sql` builds target/release/units-worker first, then points
62-
# VGI_UNITS_WORKER at it and runs the test/sql/*.test suite.
82+
- name: Download haybarn-unittest
83+
run: |
84+
gh release download "$HAYBARN_RELEASE" \
85+
--repo Query-farm-haybarn/haybarn \
86+
--pattern '${{ matrix.asset }}' \
87+
--output haybarn-unittest.zip --clobber
88+
mkdir -p hb && unzip -o -q haybarn-unittest.zip -d hb
89+
env:
90+
GH_TOKEN: ${{ github.token }}
91+
HAYBARN_RELEASE: ${{ needs.resolve-haybarn.outputs.release }}
92+
93+
- name: Resolve runner + worker paths
6394
run: |
64-
export PATH="$HOME/.local/bin:$PATH"
65-
make test-sql
95+
# Absolute paths: run-integration.sh cd's into a staging dir before
96+
# invoking the runner, so relative paths would not resolve.
97+
UNITTEST="$PWD/$(find hb -name 'haybarn-unittest' -type f | head -1)"
98+
chmod +x "$UNITTEST"
99+
echo "HAYBARN_UNITTEST=$UNITTEST" >> "$GITHUB_ENV"
100+
echo "VGI_UNITS_WORKER=$PWD/target/release/units-worker" >> "$GITHUB_ENV"
101+
102+
- name: Run extension integration suite
103+
run: ci/run-integration.sh

‎ci/README.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# CI: the vgi-units worker integration suite
2+
3+
[`.github/workflows/ci.yml`](../.github/workflows/ci.yml) runs fmt/clippy/build,
4+
the Rust unit + integration tests, and this repo's sqllogictest suite
5+
(`test/sql/*.test`) against the vgi-units VGI worker through the **real DuckDB
6+
`vgi` extension** on every push / PR.
7+
8+
## How it works (no C++ build)
9+
10+
Rather than building the vgi DuckDB extension from source, the integration job
11+
drives a **prebuilt** standalone `haybarn-unittest` (the DuckDB/Haybarn
12+
sqllogictest runner, published in Haybarn's releases) and installs the
13+
**signed** `vgi` extension from the Haybarn community channel:
14+
15+
1. **Build the worker** — `cargo build --release --bin units-worker`. The
16+
compiled `target/release/units-worker` is a self-contained stdio worker the
17+
extension spawns (the `.test` files `ATTACH` it via `${VGI_UNITS_WORKER}`).
18+
2. **Download the runner** — the matching `haybarn_unittest-*` asset per
19+
platform from the latest Haybarn release.
20+
3. **Preprocess** — the standalone runner links none of the extensions the
21+
tests gate on, so [`preprocess-require.awk`](preprocess-require.awk) rewrites
22+
each `require <ext>` into an explicit signed `INSTALL <ext> FROM
23+
{community,core}; LOAD <ext>;`. `require-env` and everything else pass
24+
through untouched. (The vgi-units `.test` files already `LOAD vgi;`
25+
explicitly and use `require-env VGI_UNITS_WORKER`, so the rewrite is a
26+
no-op here, but the awk is kept verbatim for parity with the other workers.)
27+
4. **Run** — [`run-integration.sh`](run-integration.sh) stages the preprocessed
28+
tree, points `VGI_UNITS_WORKER` at the release binary, warms the extension
29+
cache once (`INSTALL vgi FROM community;` — this is what makes the tests'
30+
explicit `LOAD vgi;` succeed), then runs the suite in a single
31+
`haybarn-unittest` invocation. Any failed assertion exits non-zero and fails
32+
the job.
33+
34+
## Run it locally
35+
36+
```bash
37+
cargo build --release --bin units-worker
38+
# point HAYBARN_UNITTEST at a haybarn-unittest binary (or a local DuckDB
39+
# `unittest` built with the vgi extension), and the worker at the release binary:
40+
HAYBARN_UNITTEST=/path/to/haybarn-unittest \
41+
VGI_UNITS_WORKER="$PWD/target/release/units-worker" \
42+
ci/run-integration.sh
43+
```
44+
45+
Or use the Makefile target `make test-sql`, which builds the release worker and
46+
runs the suite against a `haybarn-unittest` on `PATH` (`uv tool install
47+
haybarn-unittest`).

‎ci/preprocess-require.awk‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Copyright 2026 Query Farm LLC - https://query.farm
2+
#
3+
# Rewrite each `require <ext>` gate in this repo's sqllogictest files into an
4+
# explicit signed INSTALL+LOAD, so the prebuilt standalone `haybarn-unittest`
5+
# (which links none of these extensions) can run the suite. The vgi extension
6+
# comes from the signed community channel; httpfs/json/parquet/spatial from the
7+
# signed core channel. `require-env` and every other directive pass through
8+
# untouched. See ci/README.md.
9+
/^require[ \t]+vgi[ \t]*$/ {
10+
print "statement ok"; print "INSTALL vgi FROM community;"; print "";
11+
print "statement ok"; print "LOAD vgi;"; next
12+
}
13+
/^require[ \t]+(httpfs|json|parquet|spatial)[ \t]*$/ {
14+
ext = $2
15+
print "statement ok"; print "INSTALL " ext " FROM core;"; print "";
16+
print "statement ok"; print "LOAD " ext ";"; next
17+
}
18+
{ print }

‎ci/run-integration.sh‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/usr/bin/env bash
2+
# Copyright 2026 Query Farm LLC - https://query.farm
3+
#
4+
# Run this repo's sqllogictest suite (test/sql/*.test) against the vgi-units
5+
# VGI worker, using a prebuilt standalone `haybarn-unittest` and the signed
6+
# community `vgi` extension — no C++ build from source. See ci/README.md.
7+
#
8+
# Required environment:
9+
# HAYBARN_UNITTEST path to the haybarn-unittest binary
10+
# VGI_UNITS_WORKER worker LOCATION the .test files attach (a stdio command
11+
# such as the compiled units-worker binary, or an http:// URL)
12+
# Optional:
13+
# STAGE scratch dir for the preprocessed test tree (default: mktemp)
14+
set -euo pipefail
15+
16+
: "${HAYBARN_UNITTEST:?path to the haybarn-unittest binary}"
17+
: "${VGI_UNITS_WORKER:?worker LOCATION (stdio command or http:// URL)}"
18+
19+
HERE="$(cd "$(dirname "$0")" && pwd)"
20+
REPO="$(cd "$HERE/.." && pwd)"
21+
STAGE="${STAGE:-$(mktemp -d)}"
22+
23+
echo "Staging preprocessed tests into $STAGE ..."
24+
mkdir -p "$STAGE/test/sql"
25+
for f in "$REPO"/test/sql/*.test; do
26+
awk -f "$HERE/preprocess-require.awk" "$f" > "$STAGE/test/sql/$(basename "$f")"
27+
done
28+
29+
cd "$STAGE"
30+
31+
# Warm the extension cache once: vgi from the signed community channel. A miss
32+
# here is only a warning — the per-test LOAD vgi; (the .test files load it
33+
# explicitly) is what actually gates each file, and that LOAD only succeeds once
34+
# vgi has been INSTALLed from community.
35+
echo "Warming the extension cache (vgi from community) ..."
36+
mkdir -p "$STAGE/test"
37+
cat > "$STAGE/test/_warm.test" <<'EOF'
38+
# name: test/_warm.test
39+
# group: [warm]
40+
statement ok
41+
INSTALL vgi FROM community;
42+
EOF
43+
"$HAYBARN_UNITTEST" "test/_warm.test" >/dev/null 2>&1 || echo "::warning::extension warm step did not fully succeed"
44+
rm -f "$STAGE/test/_warm.test"
45+
46+
# Run the whole suite in one invocation, streaming the runner's native
47+
# sqllogictest report. Any failed assertion exits non-zero and fails the job.
48+
echo "Running suite (worker: $VGI_UNITS_WORKER) ..."
49+
"$HAYBARN_UNITTEST" "test/sql/*"

0 commit comments

Comments
 (0)