test: verify gojq parity for all 27 queries#188
Merged
Conversation
Verifies every runnable query produces semantically equivalent output under gojq and system jq, against the curated fixtures the integration test already exercises. 54 invocations (26 queries × text + jsonl modes); all pass with text-mode byte-identical and JSONL set-equal after canonical-form normalization (gojq alphabetizes JSON keys; jq preserves insertion order — same data, different serialization). Closes the verification step from ADR-0005's "Consequences": gojq is safe as the binary's default engine; no queries need `#! engine: jq`. The script becomes a permanent CI target so future query changes can't silently break parity. CI gains a gojq install step (Linux release tarball, pinned to 0.12.19) and a new shellcheck entry for the parity script.
…che gojq
- Drop the `env_prefix_str` 5th param of run_parity (always `""` at all 28 call sites) along with its word-splitting and SC2206 disable. Trims 28 lines of dead `""` from the table.
- Replace the per-line awk-pipes-to-jq-cmd canonicalization with a single stream-level `jq -cS .` switched on `$format`. Fixes a latent foot-gun (text-mode lines starting with `{` would have been incorrectly canonicalized) and cuts runtime ~30% (2.4s → 1.7s) by collapsing dozens of subprocess forks.
- Unify the `-n` vs path branch with an `input_args=()` array — one invocation per engine instead of two parallel branches.
- Drop the push-only `DIVERGENCES` array + footer summary; inline failure prints already carry the diff and are richer.
- CI: wrap the gojq install in actions/cache@v4 keyed on version, matching the SwiftPM cache pattern. Saves the ~3 MB download per run after first.
- Verify both cross-catalog split files are non-empty before continuing; a missing or malformed TYPES_FIXTURE used to slip through and surface as "outputs differ" later. - Compute expected invocations from the queries directory and assert PASS+FAIL hits it before declaring success — catches "added a query, forgot a both_modes row." - One-line comment justifying the `set -e` drop vs peer scripts.
jakebromberg
changed the title
This was referenced May 30, 2026
jakebromberg
added a commit
that referenced
this pull request
May 30, 2026
Follow-up to PR 3a (#193). Implements the gojq evaluator and the `audit query` handler. internal/engine adds a gojq wrapper supporting -L, --arg, --argjson, --slurpfile, -n, -r, $ENV.NAME, plus a `#! engine: jq` shell-out fallback per ADR-0005. Library resolution works against both filesystem-rooted and embed.FS-rooted query directories. internal/cli/query.go parses each query's front-matter to validate --arg / --argjson and wires the catalog declarations into positional input + slurpfile bindings. Covers all seven distinct catalog combinations produced by PR 2's front-matter (including the two-of-same-kind cross-catalog-name-collisions case). cmd/audit/main.go re-enables the `query` case that PR 3a stubbed out, and cmd/audit/audit_test.go ships an integration test exercising extract → query → status end-to-end on a synthetic file-hashes fixture. go.mod pins itchyny/gojq v0.12.19 — the version verified parity-clean in #188.
jakebromberg
added a commit
that referenced
this pull request
May 30, 2026
Follow-up to PR 3a (#193). Implements the gojq evaluator and the `audit query` handler. internal/engine adds a gojq wrapper supporting -L, --arg, --argjson, --slurpfile, -n, -r, $ENV.NAME, plus a `#! engine: jq` shell-out fallback per ADR-0005. Library resolution works against both filesystem-rooted and embed.FS-rooted query directories. internal/cli/query.go parses each query's front-matter to validate --arg / --argjson and wires the catalog declarations into positional input + slurpfile bindings. Covers all seven distinct catalog combinations produced by PR 2's front-matter (including the two-of-same-kind cross-catalog-name-collisions case). cmd/audit/main.go re-enables the `query` case that PR 3a stubbed out, and cmd/audit/audit_test.go ships an integration test exercising extract → query → status end-to-end on a synthetic file-hashes fixture. go.mod pins itchyny/gojq v0.12.19 — the version verified parity-clean in #188.
jakebromberg
added a commit
that referenced
this pull request
May 30, 2026
Follow-up to PR 3a (#193). Implements the gojq evaluator and the `audit query` handler. internal/engine adds a gojq wrapper supporting -L, --arg, --argjson, --slurpfile, -n, -r, $ENV.NAME, plus a `#! engine: jq` shell-out fallback per ADR-0005. Library resolution works against both filesystem-rooted and embed.FS-rooted query directories. internal/cli/query.go parses each query's front-matter to validate --arg / --argjson and wires the catalog declarations into positional input + slurpfile bindings. Covers all seven distinct catalog combinations produced by PR 2's front-matter (including the two-of-same-kind cross-catalog-name-collisions case). cmd/audit/main.go re-enables the `query` case that PR 3a stubbed out, and cmd/audit/audit_test.go ships an integration test exercising extract → query → status end-to-end on a synthetic file-hashes fixture. go.mod pins itchyny/gojq v0.12.19 — the version verified parity-clean in #188.
jakebromberg
added a commit
that referenced
this pull request
May 30, 2026
Follow-up to PR 3a (#193). Implements the gojq evaluator and the `audit query` handler. internal/engine adds a gojq wrapper supporting -L, --arg, --argjson, --slurpfile, -n, -r, $ENV.NAME, plus a `#! engine: jq` shell-out fallback per ADR-0005. Library resolution works against both filesystem-rooted and embed.FS-rooted query directories. internal/cli/query.go parses each query's front-matter to validate --arg / --argjson and wires the catalog declarations into positional input + slurpfile bindings. Covers all seven distinct catalog combinations produced by PR 2's front-matter (including the two-of-same-kind cross-catalog-name-collisions case). cmd/audit/main.go re-enables the `query` case that PR 3a stubbed out, and cmd/audit/audit_test.go ships an integration test exercising extract → query → status end-to-end on a synthetic file-hashes fixture. go.mod pins itchyny/gojq v0.12.19 — the version verified parity-clean in #188.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pipeline/queries/_tests/test_gojq_parity.sh, a parity test that runs every runnable query under both gojq and system jq against the curated fixtures and verifies semantic equivalence (canonical-form JSONL + byte-identical text).#! engine: jqannotation, gojq is safe as the binary's default engine.Unblocks PR 3 (binary skeleton) in the audit-binary tracker (#177).
Test plan
pipeline/queries/_tests/test_gojq_parity.sh→ PASS: 54 / FAIL: 0pipeline/queries/_tests/test_canonical.sh→ 26/26 pass (no regression)pipeline/queries/_tests/test_queries_integration.sh→ 124/124 pass (no regression)Closes #187