Skip to content

Validate fallback output-schema compatibility (#338 follow-up) #457

Description

@dgenio

Summary

Follow-up to #338 (PR #455). #455 validates a fallback tool's input schema at compile time and runtime, but explicitly leaves output-schema compatibility out of scope. A fallback whose output_schema differs from the primary's can pass input validation, run successfully, and then merge an unexpected output shape into the accumulated context — breaking downstream steps or the flow output_schema in a way no current check catches.

Why this matters

The fallback path is the recovery path. A fallback that "succeeds" but emits the wrong output shape converts a clean failure into a confusing downstream error far from the cause — the same class of problem #338 fixed on the input side, mirrored on the output side.

Proposed implementation

  1. At compile time, check the fallback tool's output_schema against the primary step's output_schema and/or the downstream context consumers, emitting a fallback_output_* CompilationError (or warning) on mismatch.
  2. Decide the policy: hard error vs. warning. Input incompatibility is a blocking error in fix: validate fallback inputs against fallback schema #455 because the fallback would deterministically fail; output mismatches may warrant a warning when downstream consumers are tolerant.
  3. Document the chosen semantics in docs/concepts/tools-and-flows.md alongside the existing fallback section.

Acceptance criteria

  • A fallback whose output shape is incompatible with the primary's declared outputs / downstream consumers is reported by compile_flow().
  • Documented fallback semantics state which output contract applies and whether mismatch is an error or a warning.

Test plan

  • Compiler tests for compatible and incompatible fallback output schemas (extend TestFallbackInputCompatibility in tests/test_compiler.py).

Relates to #338.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions