Skip to content

[OPIK-6366] [INFRA] feat: add zizmor security scan for GitHub Actions workflows#7546

Open
JetoPistola wants to merge 4 commits into
mainfrom
danield/OPIK-6366-add-zizmor-security-scan
Open

[OPIK-6366] [INFRA] feat: add zizmor security scan for GitHub Actions workflows#7546
JetoPistola wants to merge 4 commits into
mainfrom
danield/OPIK-6366-add-zizmor-security-scan

Conversation

@JetoPistola

@JetoPistola JetoPistola commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Details

image

Adds zizmor (Trail of Bits) as a security scanner for GitHub Actions workflows, complementing the syntax/shell linting actionlint already provides. actionlint asks "will this YAML run?"; zizmor asks "if it runs, can it be exploited?" (template injection, dangerous triggers, excessive GITHUB_TOKEN permissions).

CI wiring — zizmor runs as a pre-commit hook in the unified 🐙 Code Quality workflow, exactly like actionlint and hadolint, driven by .pre-commit-config.yaml as SSOT. This diverges from the ticket's original standalone-zizmor.yml/SARIF-upload proposal: the repo standardized on pre-commit-as-SSOT for workflow linters after the ticket was filed, so a standalone workflow would fight that architecture. (SARIF → Security-tab upload is therefore intentionally not included; can be revisited as a scheduled scan later.)

  • Hook pinned to zizmorcore/zizmor-pre-commit@v1.27.0 (→ zizmor==1.27.0), run --offline --min-severity high --persona regular (no GitHub API calls → no rate limits; blocks only high-confidence, high-severity findings).
  • Rule tuning in auto-discovered .github/zizmor.yml. unpinned-uses (SHA-pin every action) is disabled there pending a follow-up migration — same split discipline as actionlint→SC2086 cleanup.

Findings fixed — all 63 high-severity findings on main (zizmor --min-severity high now reports 0):

Class Count Fix
template-injection 44 Hoist ${{ }} out of run: bodies into env:, reference quoted $VAR
excessive-permissions 18 Scope write grants to the jobs that use them; workflow defaults → contents: read
dangerous-triggers 1 Documented inline # zizmor: ignore on labeler.yml's pull_request_target (safe — never executes PR code)

Permission fixes preserve the caller→reusable-workflow packages: write chain (a called workflow's permissions can only be reduced from the caller's, never elevated), so GHCR image pushes keep working.

Timing-table hardening — the new hook initially showed a blank description in the Code Quality per-hook timing comment (no entry in precommit-hook-descriptions.tsv). Added the missing description and a coverage test that reads every hook name from .pre-commit-config.yaml and asserts each resolves to a non-empty description, so a future hook added without a TSV entry fails CI instead of silently rendering blank.

Review follow-ups — corrected a pre-existing GithubGitHub branding typo in the commit-author name of three sync workflows (Baz). A separate pre-existing resumeUrl-logging leak Baz flagged in the release-n-deploy wrapper is tracked as OPIK-7414 rather than folded in here (out of this ticket's scope).

Change checklist

  • New pre-commit hook + .github/zizmor.yml config
  • 63 high-severity findings fixed across 26 workflow files
  • Timing-table description added for zizmor + coverage test guarding all hooks
  • GithubGitHub commit-author typo fixed in three workflows
  • CONTRIBUTING.md documents local zizmor usage, severity floor, suppression policy
  • No silent suppressions — the one exemption is inline and justified

Issues

OPIK-6366. Follow-ups: OPIK-7413 (deferred 257 unpinned-uses SHA-pinning migration) and OPIK-7414 (pre-existing resumeUrl log leak in the release-n-deploy wrapper) — both filed and linked.

Testing

  • zizmor --offline --min-severity high --persona regular .github/workflows/No findings.
  • pre-commit run actionlint + pre-commit run zizmor on all changed workflows → Passed (shellcheck included; newly-visible SC2086 in docker-vulnerability-scan.yaml fixed by quoting).
  • zizmor hook runs green in every commit's pre-commit gate — verifies the CI wiring end-to-end.
  • scripts/test_precommit_wrappers.shAll passed, including the new hook-description coverage test. Verified it fails (exit 1, naming the offending hook) when a description is removed, and passes when restored.

Documentation

CONTRIBUTING.md → "GitHub Actions workflows" section gains a zizmor subsection (local usage, severity floor, explicit-suppression policy, template-injection fix pattern).

… workflows

Add zizmor (Trail of Bits) as a security scanner for GitHub Actions
workflows, complementing actionlint's syntax/shell linting. Runs as a
pre-commit hook in the unified Code Quality workflow at high severity,
regular persona, offline — matching how actionlint and hadolint are wired.

Rule tuning lives in .github/zizmor.yml (auto-discovered); unpinned-uses
is disabled there pending a separate SHA-pinning migration.

Fixes all 63 high-severity findings on main:
- 44 template-injection: hoist ${{ }} out of run: bodies into env:
- 18 excessive-permissions: scope write grants down to the jobs that
  use them; workflow defaults drop to contents: read (packages: write
  passed down to reusable-workflow-call jobs to preserve GHCR pushes)
- 1 dangerous-triggers: documented inline ignore on labeler.yml's
  pull_request_target (safe — never executes PR-controlled code)

Document local zizmor usage in CONTRIBUTING.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation Infrastructure 🟠 size/L and removed baz: pending labels Jul 21, 2026
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

⏱️ pre-commit per-hook timing

Hook Description Result Duration
⚙️ actionlint — github workflows Lint GitHub Actions workflows 1.22s
🧪 pre-commit wrapper smoke tests Self-test the wrapper scripts 0.66s
🌈 zizmor — github workflows security Security-scan GitHub Actions workflows 0.13s
Total (3 ran) 2.01s
⏭️ 39 skipped (no matching files changed)
Hook Description Result
🐍 trim trailing whitespace — python sdk Strip trailing whitespace ⏭️
🐍 fix end of files — python sdk Ensure files end in a newline ⏭️
🐍 ruff — python sdk Lint + autofix Python (ruff) ⏭️
🐍 ruff-format — python sdk Format Python code (ruff) ⏭️
🐍 mypy — python sdk Static type check ⏭️
🤖 trim trailing whitespace — optimizer Strip trailing whitespace ⏭️
🤖 fix end of files — optimizer Ensure files end in a newline ⏭️
🤖 check yaml — optimizer Validate YAML syntax ⏭️
🤖 check json — optimizer Validate JSON syntax ⏭️
🤖 check toml — optimizer Validate TOML syntax ⏭️
🤖 check for added large files — optimizer Block large files (>1MB) ⏭️
🔐 detect private key — optimizer Block committed private keys ⏭️
🤖 check for merge conflicts — optimizer Block merge-conflict markers ⏭️
🤖 check for case conflicts — optimizer Block case-only name clashes ⏭️
🤖 pyupgrade — optimizer Modernize Python syntax ⏭️
🤖 ruff — optimizer Lint + autofix Python (ruff) ⏭️
🤖 ruff-format — optimizer Format Python code (ruff) ⏭️
🤖 mypy — optimizer Static type check ⏭️
📓 nbstripout — optimizer notebooks Strip notebook output ⏭️
📝 markdownlint — optimizer Lint Markdown ⏭️
🔤 codespell — optimizer Fix common misspellings ⏭️
📊 radon cc — optimizer Cyclomatic-complexity gate ⏭️
📊 radon raw — optimizer Raw size metrics gate ⏭️
📊 xenon — optimizer Fail on complexity thresholds ⏭️
📊 lizard — optimizer Cyclomatic-complexity gate ⏭️
🧹 vulture — optimizer Find dead code ⏭️
🛡️ trim trailing whitespace — guardrails Strip trailing whitespace ⏭️
🛡️ fix end of files — guardrails Ensure files end in a newline ⏭️
🛡️ ruff — guardrails Lint + autofix Python (ruff) ⏭️
🛡️ ruff-format — guardrails Format Python code (ruff) ⏭️
🛡️ mypy — guardrails Static type check ⏭️
⚓ helm-docs Regenerate Helm chart README ⏭️
block non-public FE plugins Block non-public FE plugins ⏭️
☕ spotless — java backend Format Java code ⏭️
🌐 eslint — frontend Lint + autofix JS/TS ⏭️
🌐 typecheck — frontend Whole-project tsc type check ⏭️
📘 eslint — typescript sdk Lint + autofix JS/TS ⏭️
📘 typecheck — typescript sdk Whole-project tsc type check ⏭️
🐳 hadolint — dockerfiles Lint Dockerfiles ⏭️

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

🌿 Preview your docs: https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik

No broken links found

Unverified links (timeout / rate-limited / server error — not failing the check)

https://aistudio.google.com/welcome (timeout)
↳ on page: /docs/opik/integrations/gemini
https://beeai.dev/ (timeout)
↳ on page: /docs/opik/integrations/beeai-typescript
https://cloud.google.com/vertex-ai?hl=en (timeout)
↳ on page: /docs/opik/integrations/gemini
https://console.mistral.ai/api-keys/ (timeout)
↳ on page: /docs/opik/integrations/mistral
https://console.x.ai/ (timeout)
↳ on page: /docs/opik/integrations/xai-grok
https://docs.konghq.com/gateway/latest/ (timeout)
↳ on page: /docs/opik/integrations/kong-ai-gateway
https://docs.konghq.com/gateway/latest/ai-gateway/ (timeout)
↳ on page: /docs/opik/integrations/kong-ai-gateway
https://docs.predibase.com/integrations/comet (403)
↳ on page: /docs/opik/integrations/predibase
https://docs.x.ai/ (timeout)
↳ on page: /docs/opik/integrations/xai-grok
https://js.langchain.com/ (timeout)
↳ on page: /docs/opik/integrations/langchainjs
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/alerts/update-alert/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/alerts/update-alert
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/automation-rule-evaluators/delete-automation-rule-evaluator-batch/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/automation-rule-evaluators/delete-automation-rule-evaluator-batch
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/automation-rule-evaluators/get-evaluator-by-id/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/automation-rule-evaluators/get-evaluator-by-id
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/llm-provider-key/store-llm-provider-api-key/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/llm-provider-key/store-llm-provider-api-key
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/projects/get-recent-activity (500)
↳ on page: /docs/opik/v1/reference/typescript-sdk/opik-query-language
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/projects/update-project/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/projects/update-project
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/prompts/update-prompt-versions/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/prompts/update-prompt-versions
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/retention-rules/deactivate-retention-rule/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/retention-rules/deactivate-retention-rule
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/retention-rules/get-retention-rule-by-id/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/retention-rules/get-retention-rule-by-id
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/rest-api/system-analytics-queries/execute-analytics-query/ (timeout)
↳ on page: /docs/opik/v1/reference/rest-api/system-analytics-queries/execute-analytics-query
https://opik-preview-019f8626-0577-7148-a4f6-b367a555016b.docs.buildwithfern.com/docs/opik/v1/reference/typescript-sdk/evaluation/overview/ (timeout)
↳ on page: /docs/opik/v1/reference/typescript-sdk/evaluation/overview
https://platform.openai.com/docs/guides/agents (timeout)
↳ on page: /docs/opik/v1/integrations/openai_agents
https://portal.azure.com/ (403)
↳ on page: /docs/opik/administration/workspace-settings/ai_providers
https://x.ai/ (timeout)
↳ on page: /docs/opik/integrations/xai-grok


📌 Results for commit 6c480ff

Comment thread .github/workflows/release-wrapper-release-n-deploy.yml
Comment thread .github/workflows/sdks_generate_openapi_spec_and_fern_code.yml Outdated
Comment thread .github/zizmor.yml
Comment thread .pre-commit-config.yaml
JetoPistola and others added 2 commits July 21, 2026 09:42
The zizmor hook showed a blank description in the Code Quality per-hook
timing comment because it had no entry in precommit-hook-descriptions.tsv.

- Add the missing zizmor description.
- Add a coverage test in test_precommit_wrappers.sh that reads every hook
  name from .pre-commit-config.yaml and asserts each resolves to a
  non-empty description via precommit-hook-desc.py — so a hook added
  without a TSV entry now fails CI instead of silently rendering blank.
- Widen the wrapper-tests hook's files: trigger to include the TSV, the
  desc resolver, and .pre-commit-config.yaml so the check reruns when a
  hook is added or renamed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
git config user.name was "Github Actions" in three sync/commit workflows;
fixed to "GitHub Actions". Flagged by Baz on PR #7546.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JetoPistola
JetoPistola marked this pull request as ready for review July 21, 2026 07:08
@JetoPistola
JetoPistola requested review from a team as code owners July 21, 2026 07:08
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Python SDK E2E Tests Results (Python 3.13)

286 tests   283 ✅  4m 6s ⏱️
  1 suites    3 💤
  1 files      0 ❌

Results for commit 9edea8f.

♻️ This comment has been updated with latest results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

baz: pending documentation Improvements or additions to documentation Infrastructure 🟠 size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant