|
1 | | -# Agent Context for Drover |
| 1 | +# Agent Context for Drover Orchestrator (`drover`) |
2 | 2 |
|
3 | 3 | Welcome, AI Agent. This document provides context, architecture, and instructions for working within the `drover` repository. |
4 | 4 |
|
5 | 5 | ## Ecosystem Role |
6 | 6 |
|
7 | | -> **Part of the Drover Ecosystem**: `drover` is **Drover Orchestrator**—parallel **task** dispatch in Git worktrees via DBOS. It invokes **Drover Code** (and other agents) locally per **worktree run**. Milestone A hosted path: **Drover Cloud** runs single **agent jobs** on workers; Orchestrator is self-hosted and out of scope until a future platform integration. Glossary: [`CONTEXT.md`](CONTEXT.md), [`../CONTEXT-MAP.md`](../CONTEXT-MAP.md). |
| 7 | +> **Part of the Drover Ecosystem**: `drover` is **Drover Orchestrator**—parallel **task** dispatch in Git worktrees via DBOS. It invokes **Drover Code** (and other agents) locally per **worktree run**. Milestone A hosted path: **Drover Cloud** runs single **agent jobs** on workers; Orchestrator is self-hosted and out of scope until a future platform integration. |
| 8 | +
|
| 9 | +**Glossary:** [`CONTEXT.md`](CONTEXT.md). **Org index:** [`../AGENTS.md`](../AGENTS.md). |
8 | 10 |
|
9 | 11 | ## Project Overview |
10 | 12 |
|
11 | | -Drover is a durable workflow orchestrator designed to parallelize AI coding tasks. Instead of running a single agent (like Claude Code or Cursor) linearly, Drover breaks down a large epic into dependencies, respects blockers, and executes tasks using multiple agents concurrently inside isolated Git worktrees. |
| 13 | +Drover is a durable workflow orchestrator designed to parallelize AI coding tasks. Instead of running a single agent linearly, Drover breaks down a large epic into dependencies, respects blockers, and executes tasks using multiple agents concurrently inside isolated Git worktrees. |
12 | 14 |
|
13 | 15 | ### Core Technologies |
| 16 | + |
14 | 17 | - **Language**: Go 1.26.3 |
15 | | -- **Workflow Engine**: DBOS v0.14.0 (Durable Operating System) — using `dbos.Go` for concurrent steps, `dbos.Sleep` for durable delays |
16 | | -- **Database**: PostgreSQL (Production) / SQLite (Local Development) |
17 | | -- **CLI Framework**: Cobra v1.10 |
18 | | -- **Observability**: OpenTelemetry v1.43.0 |
19 | | -- **Agent Interface**: Pluggable — supports Claude Code, Codex, Amp, OpenCode, Drover Code |
| 18 | +- **Workflow Engine**: DBOS — `dbos.Go` for concurrent steps, `dbos.Sleep` for durable delays |
| 19 | +- **Database**: PostgreSQL (production) / SQLite (local) |
| 20 | +- **CLI**: Cobra |
| 21 | +- **Observability**: OpenTelemetry |
| 22 | +- **Agent interface**: Pluggable — Claude Code, Codex, Amp, OpenCode, Drover Code |
20 | 23 |
|
21 | 24 | ## Architecture |
22 | 25 |
|
23 | 26 | ### Package Map |
24 | 27 |
|
25 | | -| Package | Purpose | Coverage | |
26 | | -|---------|---------|----------| |
27 | | -| `internal/workflow` | Core orchestration (Orchestrator, DBOSOrchestrator, step functions) | 44.1% | |
28 | | -| `internal/db` | SQLite/PostgreSQL persistence layer | 38.0% | |
29 | | -| `internal/executor` | Pluggable agent interface (Claude, Codex, Amp, OpenCode, Drover Code) | 47.8% | |
30 | | -| `internal/git` | Git worktree lifecycle (create, merge, prune, cleanup) | 69.5% | |
31 | | -| `internal/config` | Configuration loading (env vars, flags, defaults) | 4.8% | |
32 | | -| `internal/analytics` | Task execution analytics and reporting | 91.4% | |
33 | | -| `internal/backpressure` | Adaptive concurrency control (AIMD algorithm) | 77.8% | |
34 | | -| `internal/beads` | Hierarchical task ID parser (e.g., `task-123.1.2`) | 72.8% | |
35 | | -| `internal/clock` | Deterministic time injection for testing | — | |
36 | | -| `internal/context` | Context window management for LLM prompts | — | |
37 | | -| `internal/taskcontext` | Recent-task context carrying between agents | 98.6% | |
38 | | -| `internal/llmproxy` | LLM proxy server for rate limiting | 100% | |
39 | | -| `internal/modes` | Execution mode selection (parallel, sequential, queue) | 90.8% | |
40 | | -| `internal/search` | Task search and filtering | 65.7% | |
41 | | -| `internal/template` | Prompt template engine | 100% | |
42 | | -| `internal/webhooks` | Webhook notification system | 66.1% | |
43 | | -| `internal/dashboard` | Real-time WebSocket dashboard (requires DB) | 39.2% | |
44 | | -| `internal/memory` | Process memory tracking (platform-specific: Linux `/proc`, macOS `ps`/`sysctl`) | 80.6% | |
45 | | -| `internal/callbacks` | Task lifecycle callbacks | 28.4% | |
46 | | -| `internal/flags` | CLI flag parsing | 63.3% | |
47 | | -| `pkg/telemetry` | OpenTelemetry traces, metrics, and spans | 66.5% | |
48 | | -| `pkg/types` | Shared types (Task, TaskStatus, Epic) | — | |
| 28 | +| Package | Purpose | |
| 29 | +|---------|---------| |
| 30 | +| `internal/workflow` | Core orchestration (Orchestrator, DBOSOrchestrator, step functions) | |
| 31 | +| `internal/db` | SQLite/PostgreSQL persistence | |
| 32 | +| `internal/executor` | Pluggable agent interface | |
| 33 | +| `internal/git` | Git worktree lifecycle | |
| 34 | +| `internal/config` | Configuration loading | |
| 35 | +| `internal/analytics` | Task execution analytics | |
| 36 | +| `internal/backpressure` | Adaptive concurrency (AIMD) | |
| 37 | +| `internal/beads` | Hierarchical task ID parser | |
| 38 | +| `internal/modes` | Execution modes (parallel, sequential, queue) | |
| 39 | +| `pkg/telemetry` | OpenTelemetry | |
49 | 40 |
|
50 | 41 | ### Key Interfaces |
51 | 42 |
|
52 | | -- **`workflow.GitManager`** — Defined in `internal/workflow/git_interface.go`. Decouples orchestration from `*git.WorktreeManager`. All worktree operations go through this interface. |
53 | | -- **`executor.Agent`** — Defined in `internal/executor/agent.go`. Pluggable agent execution with `ExecuteWithContext`, `SetProjectGuidelines`, `SetContextManager`, `SetTaskContext`. |
54 | | -- **`clock.Clock`** — Defined in `internal/clock/clock.go`. Injectable time source for deterministic testing. |
| 43 | +- **`workflow.GitManager`** — worktree operations; do not use `*git.WorktreeManager` directly in orchestrator |
| 44 | +- **`executor.Agent`** — pluggable agent execution |
| 45 | +- **`clock.Clock`** — injectable time for tests |
55 | 46 |
|
56 | 47 | ## Key Concepts |
57 | | -- **Durable Workflows**: Every action is checkpointed by DBOS. If Drover crashes or gets interrupted, it can resume from exactly where it left off. |
58 | | -- **Git Worktrees**: To allow multiple agents to modify the same repository simultaneously without conflict, each worker operates in an isolated Git worktree. |
59 | | -- **Hierarchical Tasks**: Tasks can have sub-tasks (using the `task-123.1` format inspired by Beads). Sub-tasks run first, the parent runs last. |
60 | | -- **Deterministic Clock**: The `clock.Clock` interface replaces raw `time.Now()` calls, enabling fully deterministic time-based unit tests via `clock.MockClock`. |
61 | | -- **Workflow ID Generation**: Uses `sync/atomic.Int64` counter for thread-safe, unique workflow IDs. |
62 | | -- **Concurrent Side-Effects**: Dashboard broadcasts, webhooks, analytics, and event recording use `dbos.Go` for fire-and-forget concurrent execution, keeping them off the critical task-execution path. |
63 | | -- **Durable Delays**: Queue polling uses `dbos.Sleep` instead of `time.Sleep`, ensuring delays survive process crashes and DBOS recovery. |
64 | | -- **Durable Timeouts**: Agent execution uses `dbos.Select` to race the task step against a configurable timeout step. If Drover crashes and recovers, the timeout race replays from the DBOS checkpoint. |
65 | | -- **Dependency Cascade**: `OnTaskComplete` resolves blocker dependencies via `allBlockersComplete`, only enqueuing dependent tasks once ALL their blockers are marked complete. Uses `dbos.RunWorkflow` with `WithDelay(500ms)` for durable, staggered cascade scheduling. |
66 | | - |
67 | | -## Development Workflow |
68 | | - |
69 | | -When making modifications to `drover`: |
70 | | - |
71 | | -1. Ensure changes to core loops or workflows respect DBOS checkpointing rules (e.g., only interact with the database via the DBOS transaction methods). |
72 | | -2. Never introduce non-deterministic operations inside a DBOS workflow function without wrapping it in a DBOS step/activity. |
73 | | -3. Use `dbos.Go` for fire-and-forget side effects (dashboard, webhooks, analytics). Use `dbos.RunAsStep` for operations whose results are needed. |
74 | | -4. Use `dbos.Sleep` instead of `time.Sleep` inside workflows for durable delays that survive recovery. |
75 | | -5. Use `dbos.Select` with `dbos.Go` channels for durable timeout races on long-running steps (e.g., agent execution). |
76 | | -6. Use the `clock.Clock` interface for any time-dependent logic; never use `time.Now()` directly in testable code. |
77 | | -7. When modifying the `GitManager` interface, update `mock_git_manager_test.go` to match. |
78 | | -8. When modifying the `executor.Agent` interface, update `MockAgent` in `step_functions_test.go` to match. |
79 | | - |
80 | | -### Build and Test |
| 48 | + |
| 49 | +- **Durable workflows**: checkpointed by DBOS; resume after crash |
| 50 | +- **Git worktrees**: isolated parallel agent workspaces |
| 51 | +- **Hierarchical tasks**: `task-123.1` sub-tasks (Beads-style) |
| 52 | +- Use `dbos.Go` for fire-and-forget side effects; `dbos.RunAsStep` when results are needed |
| 53 | +- Use `dbos.Sleep` not `time.Sleep` inside workflows |
| 54 | +- Use `clock.Clock` not raw `time.Now()` in testable workflow paths |
| 55 | + |
| 56 | +## Build and Test |
81 | 57 |
|
82 | 58 | ```bash |
83 | | -# Build the CLI |
84 | 59 | go build -o drover ./cmd/drover |
85 | | - |
86 | | -# Run all tests |
87 | 60 | go test ./... |
88 | | - |
89 | | -# Run with race detector |
90 | 61 | go test -race ./... |
91 | | - |
92 | | -# Run specific package tests with coverage |
93 | | -go test ./internal/workflow/... -coverprofile=coverage.out |
94 | | - |
95 | | -# View coverage report |
96 | | -go tool cover -html=coverage.out |
97 | 62 | ``` |
98 | 63 |
|
99 | | -### Environment-Dependent Tests |
100 | | - |
101 | | -All 24 packages pass in the current environment. The following tests require specific infrastructure: |
102 | | - |
103 | | -- `internal/workflow` — DBOS integration tests (`TestDBOSOrchestrator_*`) skip without `DBOS_SYSTEM_DATABASE_URL` |
104 | | -- `internal/memory` — Platform-specific: uses build tags (`tracker_linux.go`, `tracker_darwin.go`) |
| 64 | +DBOS integration tests require `DBOS_SYSTEM_DATABASE_URL`. |
105 | 65 |
|
106 | 66 | ## Known Guidelines |
107 | | -- Never introduce non-deterministic operations inside a DBOS workflow function without wrapping it in a DBOS step/activity. |
108 | | -- Use `dbos.Go` for fire-and-forget side effects (dashboard, webhooks, analytics). Use `dbos.RunAsStep` for operations whose results are needed by the workflow. |
109 | | -- Use `dbos.Sleep` instead of `time.Sleep` inside workflows for durable, crash-safe delays. |
110 | | -- Respect the OpenTelemetry spans; when adding new operations, ensure they are properly traced. |
111 | | -- Use `clock.Clock` for time, `sync/atomic` for counters — no raw `time.Now()` or non-atomic state in workflow paths. |
112 | | -- The `GitManager` interface is the standard for worktree operations — never reference `*git.WorktreeManager` directly in the orchestrator. |
| 67 | + |
| 68 | +- No non-deterministic work inside workflow functions without a DBOS step |
| 69 | +- Respect OpenTelemetry spans on new operations |
| 70 | +- **Provisioning work** never runs through Orchestrator — that is **Drover Cloud** only |
0 commit comments