feat(run): enforce output schemas with bounded corrective retries (aictrl#5958)

EVENTS.md

@@ -530,3 +530,90 @@ Emitted when a permission request resolves to `allow` (either by matching an `al
   "input": { "command": "ls" }
 }
 ```
+
+## Structured final results
+
+These events are additive in schema version `"1"` and emitted only when
+`--output-schema <file>` is supplied. Other runs retain their event types and order.
+The schema is validated before any model request, with Ajv defaults and
+`allErrors: true` (draft-07, or Ajv2020 for a declared draft 2020-12). Validation
+does not coerce types, insert defaults or remove extra properties.
+
+`--output-schema-retries <n>` permits N additional corrective attempts (default 2).
+Zero permits exactly one attempt. `--output-result <file>` writes only accepted
+JSON using two-space indentation and a trailing newline, with a temporary file
+in the destination directory followed by an atomic rename. Failures preserve the
+file. Both flags require `--output-schema`; the destination directory must exist and be writable at configuration time.
+Without `--output-result`, formatted mode prints final JSON; NDJSON includes the
+value in the terminal event below.
+
+### `structured_output_rejected`
+
+Emitted once per rejected StructuredOutput attempt, including unparseable JSON
+and a model turn that finishes in prose without a final tool call:
+
+```json
+{
+  "type": "structured_output_rejected",
+  "attempt": 1,
+  "maxAttempts": 3,
+  "errors": [{ "path": "/count", "keyword": "type", "message": "must be integer" }]
+}
+```
+
+The usual event envelope fields also apply. `attempt` is one-based;
+`maxAttempts` equals `retryCount + 1`. Diagnostics include at most ten errors,
+with bounded JSON-pointer paths, keywords and messages. Their encoded payload
+is under 2 KB and never includes submitted values or raw JSON arguments. A `path`
+can contain property names submitted by the model (for example, in map or
+`patternProperties` schemas), never their values. Each path segment is clipped
+to 64 characters; paths also have an overall length bound.
+Validation feedback goes to the model in the same session through the `invalid` tool.
+A prose-only finish emits the fixed diagnostic
+`{ "path": "", "keyword": "missing", "message": "call StructuredOutput with the final result" }`
+and receives an ephemeral reminder before the next turn while budget remains.
+Other tools remain usable until a valid final result; their outputs are not final
+results. Provider-compatible tool schemas may be transformed, while local
+validation always uses the canonical schema.
+
+### `structured_output`
+
+Exactly one terminal event per schema-enabled session, before `session_complete`:
+
+```json
+{ "type": "structured_output", "status": "accepted", "attempts": 2, "value": { "result": "ok" } }
+```
+
+or:
+
+```json
+{ "type": "structured_output", "status": "failed", "reason": "exhausted", "attempts": 3 }
+```
+
+`attempts` counts StructuredOutput calls, including rejected/unparseable ones,
+and prose-only finishes without a final call. Other tool calls do not count.
+Compaction turns do not consume a missing-result attempt. Discarded provider
+streams restore the budget and captured result; their rejection events remain
+as telemetry. A valid call in a parallel step wins regardless of call order if
+that step started with budget remaining, and its attempt count stays within the
+configured maximum.
+When the budget is spent, the last rejection determines the reason: `missing`
+for a prose-only finish, otherwise `exhausted`. Failure reasons are:
+
+- `exhausted`: invalid or unparseable calls spent the corrective budget.
+- `missing`: prose finishes spent the budget, or the session ended normally
+  without an accepted result (including headless permission rejection).
+- `step_limit`: the agent reached its configured step cap without a result.
+- `aborted`: the invocation was cancelled, including SIGINT/SIGTERM.
+- `error`: a provider or session error, including stream idle timeout or an
+  invalid accepted result received from a remote session.
+
+Failed events have no `value`. No prose fallback is accepted.
+Configuration failures happen before a session starts and use the existing
+`invocation_error`/`invocation_complete` envelope, with code `OUTPUT_SCHEMA_CONFIG` and the specific cause clipped to 300 characters.
+
+Exit codes are **0** on accepted results, **2** for schema configuration errors,
+**3** for exhausted attempts, missing output and step limits. Provider errors and
+timeouts keep **1**; SIGINT and SIGTERM keep **130** and **143**.
+`StructuredOutputError.data.retries` records the number of corrective
+attempts used (initial attempt excluded), including prose-only corrective turns.

README.md

@@ -49,6 +49,16 @@ For programmatic consumption, use `--format json` to get raw events.
 aictrl run --format json "review this PR" | jq '.type'
 ```
 
+### Validated Final Results
+
+`aictrl run --output-schema <file>` validates the final JSON result.
+Use `--output-schema-retries <n>` for additional corrective attempts (default 2)
+and `--output-result <file>` to atomically write the accepted value.
+Exit codes: **0** accepted, **2** configuration error, **3** missing or invalid result;
+provider failures and stream timeouts use **1**, signals use **130**/**143**.
+See the [full contract](packages/cli/README.md#validated-final-results)
+and [JSON event reference](EVENTS.md).
+
 ### Non-Interactive Execution
 
 In headless mode, Aictrl automatically rejects all interactive permission requests (like `question` or `plan_enter`), ensuring your pipelines never hang.

packages/cli/README.md

@@ -28,6 +28,43 @@ For programmatic pipes:
 aictrl run --format json "scan for secrets" | jq '.properties.part.text'
 ```
 
+### Validated Final Results
+
+`aictrl run --output-schema <file>` enforces a JSON Schema on the final result
+while the agent can continue using tools. The schema root must be `type: "object"`.
+Schemas use Ajv defaults (strict validation, draft-07); a declared draft 2020-12
+schema uses Ajv2020. Invalid configuration exits before any model request.
+
+```bash
+aictrl run --output-schema result.schema.json --output-schema-retries 2 \
+  --output-result result.json "review this change"
+```
+
+- `--output-schema <file>`: JSON Schema file, forwarded in local and `--attach` runs.
+- `--output-schema-retries <n>`: additional corrective attempts, integer >= 0,
+  default 2. Zero permits one attempt; N permits at most N+1 attempts.
+- `--output-result <file>`: atomically write only the validated JSON value,
+  indented with two spaces and a trailing newline. Failures preserve an existing
+  file and leave an absent file absent. The destination directory must exist.
+
+Ajv strict mode rejects unknown schema keywords and unregistered `format`
+values such as `date-time` and `uri` as configuration errors (exit **2**).
+The result parent directory must exist and be writable at configuration time.
+
+The retries and result flags require `--output-schema`. Without a result file,
+formatted mode prints validated JSON at the end; `--format json` emits it in a
+terminal `structured_output` event. Rejections emit `structured_output_rejected`
+with bounded JSON-pointer diagnostics. See [EVENTS.md](../../EVENTS.md) for the contract.
+No valid result means failure, including missing output and the agent step limit;
+there is no prose fallback. A prose-only finish consumes one attempt and receives
+a reminder to call StructuredOutput while retries remain. Earlier tools are not
+replayed for corrective attempts.
+
+Exit codes: **0** for accepted results, **2** for schema configuration errors,
+**3** for exhausted attempts, missing output or a step limit without a result.
+Provider failures and stream timeouts retain **1**; SIGINT/SIGTERM retain
+**130**/**143**. Invocations without `--output-schema` keep their existing output.
+
 ### Auto-Reject Permissions
 
 In headless mode, Aictrl automatically rejects permissions that would otherwise prompt a user.

packages/cli/package.json

@@ -102,6 +102,7 @@
     "@standard-schema/spec": "1.0.0",
     "@zip.js/zip.js": "2.7.62",
     "ai": "catalog:",
+    "ajv": "8.18.0",
     "decimal.js": "10.5.0",
     "diff": "catalog:",
     "drizzle-orm": "1.0.0-beta.12-a5629fb",

packages/cli/src/cli/cmd/generate.ts

@@ -1,6 +1,7 @@
 import type { CommandModule } from "yargs"
 import z from "zod"
 import { MessageV2 } from "@/session/message-v2"
+import { Session } from "@/session"
 import { ProviderTermination } from "@/provider/termination"
 
 export const GenerateCommand = {
@@ -10,9 +11,26 @@ export const GenerateCommand = {
     const registry = z.registry<{ id: string }>()
     registry.add(MessageV2.StepFinishPart, { id: "StepFinishPart" })
     registry.add(ProviderTermination.Info, { id: "ProviderTermination" })
+    registry.add(MessageV2.Format, { id: "OutputFormat" })
+    registry.add(MessageV2.Assistant, { id: "AssistantMessage" })
+    registry.add(
+      z.object({
+        type: z.literal(Session.Event.StructuredOutput.type),
+        properties: Session.Event.StructuredOutput.properties,
+      }),
+      { id: "EventSessionStructuredOutput" },
+    )
+    registry.add(
+      z.object({
+        type: z.literal(Session.Event.StructuredOutputRejected.type),
+        properties: Session.Event.StructuredOutputRejected.properties,
+      }),
+      { id: "EventSessionStructuredOutputRejected" },
+    )
     const components = z.toJSONSchema(registry, {
       metadata: z.registry(),
       target: "draft-2020-12",
+      io: "input",
       uri: (id) => `#/components/schemas/${id}`,
     })
     // Component JSON pointers are references, not standalone schema resource IDs.

packages/cli/src/cli/cmd/run.invocation.ts

@@ -1,5 +1,6 @@
 import { Stdout } from "../stdout"
 import { SCHEMA_VERSION } from "./run.errors"
+import { OUTPUT_SCHEMA_CONFIG } from "./run.output"
 
 export type RunInvocationPhase = "validation" | "stdin" | "bootstrap" | "session"
 
@@ -27,14 +28,17 @@ export function createRunInvocation(enabled: boolean) {
 
   emit("invocation_start")
 
-  function error(_error: unknown, code = `INVOCATION_${phase.toUpperCase()}_FAILED`) {
+  function error(cause: unknown, code = `INVOCATION_${phase.toUpperCase()}_FAILED`) {
     if (!id || failed || completed) return
     failed = true
     if (sessionID) return
     emit("invocation_error", {
       phase,
       code,
-      message: `Invocation failed during ${phase}`,
+      message:
+        code === OUTPUT_SCHEMA_CONFIG
+          ? (cause instanceof Error ? cause.message : String(cause)).slice(0, 300)
+          : `Invocation failed during ${phase}`,
     })
   }
 

packages/cli/src/cli/cmd/run.output.ts

@@ -0,0 +1,63 @@
+import path from "path"
+import fs from "fs/promises"
+import { OutputSchema } from "@/session/output-schema"
+
+export const OUTPUT_SCHEMA_CONFIG = "OUTPUT_SCHEMA_CONFIG"
+export const OUTPUT_CONFIG_EXIT = 2
+export const OUTPUT_FAILED_EXIT = 3
+
+export async function outputSchema(input: { schema?: string; retries?: number; result?: string }) {
+  if (!input.schema) {
+    if (input.retries !== undefined || input.result !== undefined)
+      throw new Error("--output-schema-retries and --output-result require --output-schema")
+    return
+  }
+  if (input.retries !== undefined && (!Number.isSafeInteger(input.retries) || input.retries < 0))
+    throw new Error("--output-schema-retries must be an integer >= 0")
+  const text = await Bun.file(input.schema)
+    .text()
+    .catch(() => {
+      throw new Error(`${input.schema}: cannot read schema file`)
+    })
+  const schema: unknown = (() => {
+    try {
+      return JSON.parse(text)
+    } catch {
+      throw new Error(`${input.schema}: invalid JSON`)
+    }
+  })()
+  if (!schema || typeof schema !== "object" || Array.isArray(schema))
+    throw new Error(`${input.schema}: schema root must have type "object"`)
+  const canonical = schema as Record<string, unknown>
+  const validate = (() => {
+    try {
+      return OutputSchema.compile(canonical)
+    } catch (error) {
+      throw new Error(`${input.schema}: ${error instanceof Error ? error.message : "cannot compile schema"}`)
+    }
+  })()
+  if (input.result) {
+    const directory = path.dirname(input.result)
+    const writable = await fs
+      .stat(directory)
+      .then(async (stat) => {
+        if (!stat.isDirectory()) return false
+        await fs.access(directory, fs.constants.W_OK | fs.constants.X_OK)
+        return true
+      })
+      .catch(() => false)
+    if (!writable) throw new Error(`${input.result}: output-result parent directory must exist and be writable`)
+  }
+  return { format: { type: "json_schema" as const, schema: canonical, retryCount: input.retries ?? 2 }, validate }
+}
+
+export async function outputResult(file: string, value: unknown, aborted = () => false) {
+  const temp = path.join(path.dirname(file), `.${path.basename(file)}.${crypto.randomUUID()}.tmp`)
+  try {
+    await Bun.write(temp, JSON.stringify(value, null, 2) + "\n", { createPath: false })
+    if (aborted()) throw new Error("Structured output cancelled")
+    await fs.rename(temp, file)
+  } finally {
+    await fs.rm(temp, { force: true })
+  }
+}