PR 3a — audit binary substrate + extract/status subcommands

[jakebromberg]

Summary

First slice of the audit binary per tracker #177's PR 3 plan. Ships the substrate (front-matter / manifest / catalog-envelope parsers), .audit/ cache, discovery, extractor subprocess runner, and the extract + status subcommands. The query subcommand stubs out with a pointer to the follow-up PR.

What this PR delivers

• audit extract <name> — runs every [[command]] block in extractors/<name>/manifest.toml as a subprocess; materializes catalogs into .audit/catalogs/<file>; updates .audit/meta.json; auto-bootstraps .gitignore.
• audit status — prints .audit/ state, resolved queries/extractors source paths, source-mtime staleness, and root-vs-cwd warnings.
• audit query <name> — stub pointing at the follow-up PR.
• New Go module at the repo root (github.com/jakebromberg/code-audit-pipeline). Pinned BurntSushi/toml v1.4.0. gojq lands in the follow-up.
• go generate ./... copies pipeline/queries/*.jq into cmd/audit/queries/ so the //go:embed directive ships the canonical query corpus alongside the binary.
• CI gains a go job (vet + test + build).

Architecture (per the ADRs)

• ADR-0001 (amended by 0007): .audit/meta.json schema with audit_version, last_touched_at, root, catalogs[<kind>] (path, source_sha, envelope_summary, cli_args).
• ADR-0002: two parsers — #! key: value front-matter on .jq files; TOML manifest.toml on extractor dirs.
• ADR-0004: binary as router. Extractors stay external and are invoked via os/exec.
• ADR-0006: lookup-order discovery for both queries and extractors (--flag → $AUDIT_HOME → cwd-relative → embedded queries / ~/.config/audit/extractors).
• ADR-0007: meta.json.envelope_summary is a derived cache; catalog file is authoritative; refreshed on every status/query if the file's sha256 differs.

Diff size note

~2.8k lines including ~490 lines of plan doc and ~800 lines of Go tests. Production Go is ~1100 lines across 12 files. Over the 1000-line soft limit, but coherent as one reviewable unit: every package is exercised by extract + status. The follow-up PR (engine + query) is sized at ~800 lines.

Test plan

• go vet ./... passes
• go test ./... passes (with -race)
• go build ./cmd/audit produces a binary
• Smoke test: audit extract file-hashes --root <tmp> --audit-root <tmp> produces .audit/catalogs/file-hashes.json and a populated meta.json
• Smoke test: audit status --root <tmp> reports cached catalogs; when invoked from the audit-pipeline checkout, it resolves queries+extractors via cwd-relative discovery and exits 0
• Smoke test: audit query <anything> exits 2 with the follow-up-PR message
• Existing CI (shellcheck, pipeline, swift) untouched

Closes #191

[jakebromberg]

Code review

Found 4 issues:

1. Discovery lookup order inverted vs ADR-0006. docs/adr/0006-bundling-and-discovery.md prescribes "explicit flags, then cwd-relative paths, then $AUDIT_HOME, then a fallback." The implementation appends $AUDIT_HOME before cwd-relative. A contributor with AUDIT_HOME set (which audit init plants in shell profiles per ADR-0006's "Consequences") would never pick up local edits in ./pipeline/queries/ — the cwd-relative contributor path is dead in their environment.

code-audit-pipeline/internal/discovery/discovery.go

Lines 49 to 63 in 4d385f0

	candidates := []struct{ path, label string }{}
	if opts.Flag != "" {
	candidates = append(candidates, struct{ path, label string }{opts.Flag, "--queries-dir " + opts.Flag})
	}
	if opts.AuditHome != "" {
	candidates = append(candidates, struct{ path, label string }{
	filepath.Join(opts.AuditHome, "pipeline", "queries"),
	"$AUDIT_HOME (" + opts.AuditHome + ")",
	})
	}
	candidates = append(candidates, struct{ path, label string }{
	filepath.Join(cwd, "pipeline", "queries"),
	"cwd-relative " + filepath.Join(cwd, "pipeline", "queries"),
	})

2. audit status exits 0 when a registered catalog file is missing on disk. The print loop sets marker = "MISSING" but does not set staleAny = true. The exit-code logic only flips to 1 on rootMismatch, query/extractor resolution failure, or staleAny. The function's own doc comment says "0 if all catalogs are fresh and the cwd matches meta.json.root; 1 otherwise" — MISSING violates "fresh," so the path should exit 1.

code-audit-pipeline/internal/cli/status.go

Lines 84 to 92 in 4d385f0

	staleAny := false
	for _, r := range rows {
	marker := "ok"
	if !r.Exists {
	marker = "MISSING"
	} else if r.StaleSourceCount > 0 {
	marker = fmt.Sprintf("%d files newer than catalog", r.StaleSourceCount)
	staleAny = true
	}

3. audit extract leaves meta.json inconsistent with disk on partial failure. The typescript manifest has two [[command]] blocks. If the first succeeds (catalog file written, cache.PutCatalog updates the in-memory map) and the second fails, the function returns 1 without calling cache.Save(). The first catalog file remains on disk but meta.json never records it. Next audit status shows nothing because it reads the stale on-disk meta.json.

code-audit-pipeline/internal/cli/extract.go

Lines 106 to 126 in 4d385f0

	results, err := extractor.Run(ctx, extractorDir, cmd, args, catalogsDir)
	if err != nil {
	fmt.Fprintf(out, "audit: extract %s/%s: %v\n", name, cmd.Catalog, err)
	return 1
	}
	for _, r := range results {
	outBase := filepath.Base(r.OutputPath)
	if err := cache.PutCatalog(r.Catalog, outBase, r.CLIArgs); err != nil {
	fmt.Fprintf(out, "audit: cache put %s: %v\n", r.Catalog, err)
	return 1
	}
	}
	ran++
	}
	if ran == 0 {
	fmt.Fprintf(out, "audit: no [[command]] blocks matched (manifest has %d, filter selected 0)\n", len(m.Commands))
	return 2
	}
	if err := cache.Save(); err != nil {
	fmt.Fprintf(out, "audit: save cache: %v\n", err)
	return 1

4. ErrNoCommand is exported but unused (~/.claude/CLAUDE.md says "Don't add features, refactor, or introduce abstractions beyond what the task requires"). The comment claims it "distinguishes 'extractor invocation failed' from 'extractor was filtered out by --catalog flags'" but cli/extract.go checks ran == 0 directly and emits its own message — the sentinel has no consumer.

https://github.com/jakebromberg/code-audit-pipeline/blob/4d385f0ac525402930ff3b00519ffc27c23c2651/internal/extractor/runner.go#L171-L175