Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ devkit is a Claude Code plugin: deterministic YAML workflow engine, dispatcher a

| Path | What | Grep here for |
|---|---|---|
| `skills/` | 38 SKILL.md skills (engine dispatchers + capability tools) | Skill descriptions; what each skill triggers (workflow or direct exec) |
| `skills/` | 39 SKILL.md skills (engine dispatchers + capability tools) | Skill descriptions; what each skill triggers (workflow or direct exec) |
| `skills/_principles.yml` | Shared cross-cutting principle config | Rules applied to every skill |
| `skills/creating-workflows/` | Workflow YAML schema reference | Step types, `parallel:`/`branch:`/`loop:`/`expect:` semantics |
| `workflows/` | 22 YAML workflow definitions | What each skill actually runs |
| `src/engine/engine.go` | Go workflow executor | How `parallel: [ids]` skips-then-fans-out (`parallelChildren`), branch eval, loop gates |
| `src/engine/workflow.go` | Workflow struct + YAML parsing | Step field definitions |
| `src/engine/engine.go` | Go workflow executor | How `parallel: [ids]` fans out (`runParallel`), branch eval, loop gates, `end:` |
| `src/engine/workflow.go` | Workflow struct + YAML parsing + validation | Step field definitions; `ParallelChildren`/`NextIndex` (shared sequential walk) |
| `src/cmd/guard.go` | Hook gatekeeper | What Bash/Edit/etc. is allowed per step type + enforce level |
| `src/cmd/workflow.go` | `devkit workflow` CLI command | Workflow invocation entry point |
| `src/cmd/mcp.go` | `devkit mcp` CLI command | MCP server bootstrap |
Expand Down Expand Up @@ -42,7 +42,7 @@ devkit is a Claude Code plugin: deterministic YAML workflow engine, dispatcher a

- **Workflows are deterministic.** The engine controls step sequencing, loops, branches, and gates — not the model. YAML is parsed once, the graph is fixed, steps run with state persistence and hook enforcement.
- **Skills come in two flavors.** *Engine-shim skills* (most) are thin dispatchers that call `devkit_start` then loop on `devkit_advance`, delegating to the workflow engine. *Capability skills* (e.g., `skills/scrape/`) are runtime-interpreted prompt templates where the model is the executor — it picks a backend from what's installed, generates shell/script via heredoc, judges output, falls through on failure. Use the capability form when work needs on-the-spot adaptation that pre-compiled YAML can't express (anti-bot arms races, heterogeneous backend outputs, content-judgment branching).
- **Parallel step pattern.** Steps listed in another step's `parallel: [ids]` are skipped during the sequential walk (see `parallelChildren` in `src/engine/engine.go`) and dispatched concurrently via `runParallel`. Never expect a step that appears in a parallel list to also execute in its sequential position.
- **Parallel step pattern.** Steps listed in another step's `parallel: [ids]` are skipped during the sequential walk (`Workflow.NextIndex` in `src/engine/workflow.go`, used by both the CLI engine and the MCP server) and run only inside their dispatcher — `runParallel` in the CLI, per-child `outputs` on `devkit_advance` over MCP. Never expect a step that appears in a parallel list to also execute in its sequential position.
- **Enforce levels gate hook permissions.** `enforce: soft` lets gather/setup steps run `git diff` and other shell work during dispatch; `enforce: hard` blocks them. The gatekeeper logic lives in `src/cmd/guard.go`; see `guard_test.go` for the allow/deny matrix.
- **Engine binary is separate from the CLI wrapper.** `bin/devkit` is the committed shell wrapper; it execs `devkit-engine` (compiled Go, gitignored). The wrapper handles install/version checks; the engine handles workflow execution.
- **Skills are the future; `commands/` is legacy.** PR #77 migrated commands to skills. Both auto-discover, but skills support paths, disable-model-invocation, and supporting files.
Expand Down
15 changes: 8 additions & 7 deletions skills/creating-workflows/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ steps:
max: number # Maximum iterations
until: string # Stop condition (string match in output)
gate: string # Shell command run after each iteration — exit 0 keeps, non-zero reverts
branch: # Optional: conditional execution
if: string # Condition expression
then: string # Step id to jump to if true
else: string # Step id to jump to if false
branch: # Optional: first matching rule wins
- when: string # Sentinel word to look for in this step's output
goto: string # Step id to jump to
end: bool # Optional: workflow completes after this step (unless a branch matched)
```

## Step Fields
Expand All @@ -38,10 +38,11 @@ steps:
- **prompt** — The instruction. Use `{{step-id}}` to reference earlier outputs. Use `{{input}}` for workflow input. Mutually exclusive with `command`.
- **command** — Shell command run directly (no LLM). Output is captured and available via `{{step-id}}`. Costs $0. Mutually exclusive with `prompt`.
- **expect** — Only for `command` steps. Values: `success` (step fails if exit code is non-zero) or `failure` (step fails if exit code is 0). Omit for the default behavior where all exit codes are informational. Enables bugfix reproduction gates: repro with `expect: failure` must fail before fix, verify with `expect: success` must pass after.
- **parallel** — Lists step IDs to run concurrently. Results collected before next sequential step.
- **loop** — Repeats with `max` iterations. Exits early if output contains `until` string. Optional `gate` command enforces quality after each iteration.
- **parallel** — Lists step IDs to run concurrently. Listed steps are skipped in their sequential position and run only inside the dispatcher. Children must be plain prompt steps (no command, loop, branch, or end), and a branch may not `goto` a child. Over MCP, the agent reports results with `devkit_advance` `outputs: {"<child-id>": "<result>", ...}` — one entry per child — so later steps can use `{{child-id}}`.
- **loop** — Repeats with `max` iterations. Completes when the `gate` (if set) passes **and** the `until` sentinel (if set) appears in the iteration's output — a failing gate is never overridden by the sentinel. Over MCP, if `max` is reached with the gate still failing the workflow halts (`loop_exhausted`); if only the sentinel is missing it continues with a `LOOP EXHAUSTED` warning.
- **loop.gate** — Shell command run after each loop iteration. Exit 0 = keep changes and commit. Non-zero = revert changes via `git checkout`. 3 consecutive gate failures trigger stuck detection and stop the loop.
- **branch** — Routes execution conditionally. Both `then` and `else` reference step `id`s.
- **branch** — Routes execution conditionally: a list of `when`/`goto` rules evaluated against the step's output; the first match jumps to `goto`, no match falls through.
- **end** — Marks a terminal step. After it runs (and no branch rule matched) the workflow completes instead of falling through. Use it to stop a main path before an alternative path placed later in the file (e.g. a TINY `quick-fix` reached only by branch).

## Minimal Example

Expand Down
8 changes: 6 additions & 2 deletions src/cmd/guard.go
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,11 @@ func runPreToolGuard() {
guardExit(2)
return

case "prompt":
case "prompt", "parallel":
// A parallel dispatcher is a prompt step whose work runs in
// subagents; it carries the same fence (Agent/Task allowed,
// Write/Edit/Bash blocked under hard). MCP parks on the
// dispatcher now that children are skipped in the walk.
if enforce == lib.EnforceHard {
if isDevkitMCPTool(tool) {
guardExit(0)
Expand Down Expand Up @@ -506,7 +510,7 @@ func runPreToolGuard() {
return

default:
// parallel or unknown — engine is dispatching, allow through.
// unknown step type — allow through rather than wedge the session.
guardExit(0)
}
}
Expand Down
40 changes: 27 additions & 13 deletions src/cmd/guard_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,32 @@ func TestGuardPreToolUse(t *testing.T) {
wantExit: 2,
wantStderrSubstr: "cannot read session state",
},
{
// MCP now parks on the parallel dispatcher (children are
// skipped), so the dispatcher must carry the prompt fence:
// subagents may run, but Write/Edit stay blocked under hard.
name: "parallel+hard+Write → block",
dataDir: true,
hasSession: true,
session: lib.SessionState{
Status: "running", StepType: "parallel", StepEnforce: lib.EnforceHard, CurrentStep: "dispatch-reviews",
Workflow: "tri-review", TotalSteps: 6,
},
stdin: `{"tool_name":"Write"}`,
wantExit: 2,
wantStderrSubstr: "gather evidence with Read/Grep/Glob",
},
{
name: "parallel+hard+Agent → allow (dispatch subagents)",
dataDir: true,
hasSession: true,
session: lib.SessionState{
Status: "running", StepType: "parallel", StepEnforce: lib.EnforceHard, CurrentStep: "dispatch-reviews",
Workflow: "tri-review", TotalSteps: 6,
},
stdin: `{"tool_name":"Agent"}`,
wantExit: 0,
},
{
// Empty step_type is non-command and non-prompt, so the
// default branch (parallel/unknown) allows through. This
Expand Down Expand Up @@ -444,19 +470,7 @@ func TestGuardPreToolUse(t *testing.T) {
wantExit: 2,
},
{
name: "parallel+hard+Write → allow",
dataDir: true,
hasSession: true,
session: lib.SessionState{
Status: "running", StepType: "parallel", StepEnforce: lib.EnforceHard, CurrentStep: "fanout",
},
stdin: `{"tool_name":"Write"}`,
wantExit: 0,
},
{
// parallel+soft should allow exactly the same as parallel+hard.
// Explicit row so a future refactor that adds an enforce
// check to the parallel branch cannot sneak through.
// parallel+soft behaves like prompt+soft: allow with a nudge.
name: "parallel+soft+Bash → allow",
dataDir: true,
hasSession: true,
Expand Down
17 changes: 9 additions & 8 deletions src/engine/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,14 +108,9 @@ func (e *Engine) RunWorkflow(ctx context.Context, wf *Workflow, cfg RunConfig) (
// addCost updates the running total (used by loops to keep overBudget accurate)
addCost := func(c float64) { totalUSD += c }

// Build set of step IDs that are dispatched by parallel steps,
// so we skip them during sequential walk (they run inside runParallel).
parallelChildren := make(map[string]bool)
for _, s := range wf.Steps {
for _, pid := range s.Parallel {
parallelChildren[pid] = true
}
}
// Steps dispatched by a parallel step are skipped during the
// sequential walk (they run inside runParallel).
parallelChildren := wf.ParallelChildren()

// Walk steps sequentially, with branch jumps
failed := false
Expand Down Expand Up @@ -173,6 +168,9 @@ func (e *Engine) RunWorkflow(ctx context.Context, wf *Workflow, cfg RunConfig) (
break
}
totalUSD += cost
if step.End {
break
}
i++
continue
}
Expand Down Expand Up @@ -216,6 +214,9 @@ func (e *Engine) RunWorkflow(ctx context.Context, wf *Workflow, cfg RunConfig) (
i = jump
continue
}
if step.End {
break
}

i++
}
Expand Down
158 changes: 158 additions & 0 deletions src/engine/engine_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1713,3 +1713,161 @@ func TestInterpolateDeterministic(t *testing.T) {
}
}
}

func TestRunWorkflowEndStopsBeforeTrailingSteps(t *testing.T) {
db := tempDB(t)
dir, git := initGitRepo(t)
runner := newMockRunner([]runners.RunResult{
result("BIG change"),
result("full path done"),
}, nil)
eng := mustEngine(t, db, git, runner, dir)
wf := &Workflow{
Name: "test",
Steps: []WfStep{
{ID: "triage", Prompt: "Classify {{input}}", Branch: []Branch{{When: "TINY", Goto: "quick"}}},
{ID: "full", Prompt: "Full", End: true},
{ID: "quick", Prompt: "Quick"},
},
}
res, err := eng.RunWorkflow(context.Background(), wf, RunConfig{Input: "x"})
if err != nil {
t.Fatalf("RunWorkflow: %v", err)
}
if runner.callIdx != 2 {
t.Errorf("runner called %d times, want 2 (quick must not run after end)", runner.callIdx)
}
if _, ok := res.Outputs["quick"]; ok {
t.Error("quick ran after an end: true step")
}
}

func TestValidateRejectsUnsupportedParallelChildren(t *testing.T) {
base := func(child WfStep) *Workflow {
return &Workflow{Name: "w", Steps: []WfStep{
{ID: "fan", Parallel: []string{"c"}},
child,
{ID: "after", Prompt: "after"},
}}
}
cases := map[string]WfStep{
"command child": {ID: "c", Command: "true"},
"loop child": {ID: "c", Prompt: "p", Loop: &Loop{Max: 2}},
"branch child": {ID: "c", Prompt: "p", Branch: []Branch{{When: "X", Goto: "after"}}},
"end child": {ID: "c", Prompt: "p", End: true},
}
for name, child := range cases {
if err := base(child).Validate(); err == nil {
t.Errorf("%s: expected validation error", name)
}
}
// goto into a parallel child would be silently skipped by the walk.
wf := &Workflow{Name: "w", Steps: []WfStep{
{ID: "t", Prompt: "t", Branch: []Branch{{When: "X", Goto: "c"}}},
{ID: "c", Prompt: "c"},
{ID: "fan", Parallel: []string{"c"}},
}}
if err := wf.Validate(); err == nil {
t.Error("branch goto targeting a parallel child should be rejected")
}
}

func TestNextIndexSkipsParallelChildren(t *testing.T) {
wf := &Workflow{Name: "w", Steps: []WfStep{
{ID: "a", Prompt: "a"},
{ID: "b", Prompt: "b"},
{ID: "fan", Parallel: []string{"a", "b"}},
{ID: "z", Prompt: "z"},
}}
if got := wf.NextIndex(0); got != 2 {
t.Errorf("NextIndex(0) = %d, want 2", got)
}
if got := wf.NextIndex(3); got != 3 {
t.Errorf("NextIndex(3) = %d, want 3", got)
}
if got := wf.NextIndex(4); got != 4 {
t.Errorf("NextIndex(4) = %d, want 4 (past end)", got)
}
}

// The full paths of feature and bugfix must stop at their report step
// instead of falling through into the TINY quick-fix step placed last.
func TestShippedWorkflowsFullPathEndsBeforeQuickFix(t *testing.T) {
for wfFile, terminal := range map[string]string{
"feature.yml": "final-report",
"bugfix.yml": "summary",
} {
wf, err := ParseFile(filepath.Join("..", "..", "workflows", wfFile))
if err != nil {
t.Fatalf("%s: %v", wfFile, err)
}
var found bool
for i, s := range wf.Steps {
if s.ID != terminal {
continue
}
found = true
if !s.End {
t.Errorf("%s: step %q must set end: true", wfFile, terminal)
}
if i+1 >= len(wf.Steps) || wf.Steps[i+1].ID != "quick-fix" {
t.Errorf("%s: expected quick-fix right after %q", wfFile, terminal)
}
}
if !found {
t.Errorf("%s: step %q not found", wfFile, terminal)
}
}
}

func TestRunWorkflowEndOnParallelDispatcher(t *testing.T) {
db := tempDB(t)
dir, git := initGitRepo(t)
runner := newMockRunner([]runners.RunResult{
result("a out"), result("b out"), result("must not run"),
}, nil)
eng := mustEngine(t, db, git, runner, dir)
wf := &Workflow{Name: "test", Steps: []WfStep{
{ID: "a", Prompt: "A"},
{ID: "b", Prompt: "B"},
{ID: "fan", Parallel: []string{"a", "b"}, End: true},
{ID: "trailing", Prompt: "T"},
}}
res, err := eng.RunWorkflow(context.Background(), wf, RunConfig{Input: "x"})
if err != nil {
t.Fatalf("RunWorkflow: %v", err)
}
if _, ok := res.Outputs["trailing"]; ok {
t.Error("trailing step ran after an end: true dispatcher")
}
}

func TestEffectiveEnforceParallelDispatcherInheritsSoftChildren(t *testing.T) {
wf := Workflow{Name: "w", Enforce: EnforceHard, Steps: []WfStep{
{ID: "a", Prompt: "a", Enforce: EnforceSoft},
{ID: "b", Prompt: "b"},
{ID: "fan", Parallel: []string{"a", "b"}},
{ID: "fan-hard", Parallel: []string{"b"}},
{ID: "fan-pinned", Parallel: []string{"a"}, Enforce: EnforceHard},
}}
if got := EffectiveEnforce(wf, wf.Steps[2]); got != EnforceSoft {
t.Errorf("dispatcher with a soft child = %q, want soft (children need their tools)", got)
}
if got := EffectiveEnforce(wf, wf.Steps[3]); got != EnforceHard {
t.Errorf("dispatcher with only hard children = %q, want hard", got)
}
if got := EffectiveEnforce(wf, wf.Steps[4]); got != EnforceHard {
t.Errorf("explicit enforce on dispatcher = %q, want hard", got)
}
}

func TestValidateRejectsBranchOnParallelDispatcher(t *testing.T) {
wf := &Workflow{Name: "w", Steps: []WfStep{
{ID: "fan", Parallel: []string{"c"}, Branch: []Branch{{When: "X", Goto: "after"}}},
{ID: "c", Prompt: "c"},
{ID: "after", Prompt: "after"},
}}
if err := wf.Validate(); err == nil {
t.Error("branch on a parallel dispatcher should be rejected (CLI never evaluates it)")
}
}
Loading
Loading