Skip to content

Add regression coverage for integration adapters and the chainweaver.testing harness #439

Description

@dgenio

Summary

Add focused regression tests for the integration adapters (export/, integrations/) and for the public test harness chainweaver.testing (FlowTestRunner, assert_result_matches, capture_steps, fakes/replay), which underpin downstream users' own test suites and are themselves a supported public surface.

Why this matters

chainweaver.testing is shipped for users to test their flows (FlowTestRunner, assertions, fakes, replay), and the export/integration adapters are how flows reach other frameworks — yet these surfaces are easy to under-test relative to the executor core. A regression for the harness protects every downstream user's test suite from silent breakage on refactor, and adapter tests protect the ecosystem bridges. Extends open #342 (adapter + safety-classification coverage) to explicitly include the testing harness.

Current evidence

External context

Not required for this issue.

Proposed implementation

  1. Inventory current coverage of chainweaver.testing and the adapters (use --cov-report=term-missing scoped to those paths).
  2. Add tests: FlowTestRunner fake/passthrough tools, calls_to/inputs_to accounting, middleware capture, assert_result_matches pass/fail paths, replay behavior, fakes determinism.
  3. For adapters, add round-trip/shape tests (export payload structure; integration wrap/unwrap) guarded by import-skips for optional extras.
  4. Coordinate with Add regression coverage for integration adapters and trace safety-classification heuristics #342 so adapter coverage isn't double-built; this issue owns the harness coverage.

AI-agent execution notes

Inspect chainweaver/testing/runner.py, assertions.py, fakes.py, replay.py, chainweaver/export/*, chainweaver/integrations/*, existing tests, open #342. Respect the design trap that testing helpers stay out of top-level __all__. Guard optional-extra adapter tests with import-skips. Keep --cov-fail-under=80 satisfied or improved. Don't change harness behavior — add coverage; if a test reveals a real bug, file/fix it explicitly.

Acceptance criteria

  • Coverage for chainweaver.testing and the export adapters measurably increases, with both success and failure paths exercised.
  • Optional-extra integration adapters have import-skip-guarded round-trip tests.
  • Coverage gate stays green; the harness/adapter behavior is unchanged.

Test plan

The new tests; scoped coverage report before/after; CI matrix run; import-skip behavior verified when extras are absent.

Documentation plan

docs/agent-context/workflows.md testing note if conventions change; CHANGELOG (test-infra note); coordinate with #342.

Migration and compatibility notes

Not expected to require migration.

Risks and tradeoffs

Possible overlap with #342 on adapter scope — coordinate (this issue owns harness coverage). Tests must encode behavior, not just chase coverage numbers.

Suggested labels

testing, reliability, contributor-experience

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions