Daily Test Coverage ImproverResearch and Plan

[github-actions[bot]]

Summary of Findings

Repository Overview

Agoric SDK is a JavaScript/TypeScript monorepo with 57 packages in packages/, managed by Yarn workspaces and Lerna Lite. It implements Agoric's smart contract platform, including ERTP, Zoe, SwingSet, orchestration, fast-usdc, inter-protocol, governance, and more. There are also packages in services/, a3p-integration/, and multichain-testing/.

Testing Framework & Tools

Tool	Usage
AVA (patched, ^6.4.1)	Primary test runner — async, concurrent
c8 (^10.1.3)	V8-native code coverage collection
@endo/ses-ava	SES-hardened test harness (used in ~15+ packages)
@fast-check/ava	Property-based testing (used in ~5 packages)

Coverage Infrastructure

• Coverage is collected via NODE_V8_COVERAGE environment variable and c8
• Full cross-repo coverage report generated by scripts/ci/generate-test-coverage-report.sh:

  export NODE_V8_COVERAGE="$PWD/coverage/tmp"
  export C8_OPTIONS="--clean=false"
  yarn lerna run test:c8 --no-bail
  yarn c8 report --all --include 'packages/*/{src,tools}' --reporter=html-spa --reports-dir=coverage/html

• Reports published to Netlify: (agoricsdkcoverage.netlify.app/redacted)
• Codecov integration configured in codecov.yaml (currently in informational mode — not blocking PRs)
• Important caveat (documented in COVERAGE.md): Lines from bundled code cannot be measured until source maps are implemented in @endo/bundle-source, @agoric/transform-metering, and @agoric/static-module-record. This means coverage numbers undercount actual code coverage significantly.

Key Commands

# Install dependencies (pinned Yarn version)
corepack enable && yarn install

# Build all packages
yarn build

# Run all tests
yarn test

# Run tests with coverage for a single package
cd packages/(name) && yarn test:c8

# Generate full-repo HTML coverage report
NODE_MEMORY_SIZE=8192 ./scripts/ci/generate-test-coverage-report.sh

# Generate per-package HTML report
cd packages/(name)
yarn c8 -a ava
yarn c8 report --reporter=html-spa
open coverage/html/index.html

How Tests Are Organized

• Each package has a test/ directory
• AVA config in each package.json under the "ava" key (typically "files": ["test/**/*.test.*"])
• Most test files begin with import '@endo/init/debug.js' for SES initialization
• Most packages have:
  • "test": runs ava directly
  • "test:c8": runs c8 --all ava for coverage
• Some packages have additional scripts: test:xs (XS worker tests), test:unit, test:swingset

Packages with No Real Tests (test script is exit 0)

These packages have source code but their test script does nothing:

Package	Notes
sparse-ints	Utility for sparse integer sets — highly testable pure functions
stat-logger	Statistics/telemetry logging utilities
benchmark	Benchmark infrastructure
deployment	Deployment scripts
eslint-config	ESLint configuration (testing lint rules would add value)

Coverage Gap Analysis (estimated from source/test file ratios)

Package	Source files (approx)	Coverage risk
client-utils	~346	🔴 Very high — large package, 12 test files
cosmic-proto	~255	🔴 High — mostly generated protobuf code
telemetry	~20	🔴 High — only 3 test files
swingset-runner	~21	🔴 High — only 1 test file
vats	~68	🟡 Medium
orchestration	~102	🟡 Medium — active development area
inter-protocol	~46	🟡 Medium
agoric-cli	~36	🟡 Medium

Improvement Plan

Strategy

1. Quick wins — enable tests for untested packages: Add meaningful unit tests for sparse-ints and stat-logger which have pure functions that are easy to test
2. Expand edge case coverage in utility packages: internal, vow, base-zone, store — these support everything else
3. Improve coverage of key business logic: governance, inter-protocol, fast-usdc — these handle real economic decisions
4. Orchestration improvements: Add tests for error paths and edge cases in the orchestration package

Phase-by-Phase Approach

Immediate wins (Phase 3 — early iterations):

• Add unit tests for sparse-ints (pure functions, ~100% coverage achievable)
• Add tests for telemetry utilities (pure utility functions)
• Add edge case tests for internal package helpers
• Add coverage for untested branches in governance parameter handling

Medium-term:

• Expand orchestration error path coverage
• Add tests for client-utils utilities
• Add integration tests for inter-protocol fee management paths

Longer-term:

• Improve SwingSet and Zoe coverage for complex kernel paths
• Address bundle coverage limitation with source map support

Questions/Clarifications Needed

1. Bundle coverage: Are there plans to improve source map support in @endo/bundle-source? This would dramatically increase measured coverage for contract and vat code.
2. client-utils file count: The high source count may include generated code — should those be excluded?
3. XS worker tests: Should coverage include XS tests as well?
4. Slow test exclusions: Some tests are intentionally slow/skipped in CI — are these excluded from coverage?

How to Control this Workflow

You can add comments to this discussion to provide feedback, adjustments to the plan, or direction on which packages to prioritize.

Use these commands to control the workflow:

gh aw disable daily-test-improver --repo Agoric/agoric-sdk
gh aw enable daily-test-improver --repo Agoric/agoric-sdk
gh aw run daily-test-improver --repo Agoric/agoric-sdk --repeat (number-of-repeats)
gh aw logs daily-test-improver --repo Agoric/agoric-sdk

What Happens Next

• Next run (Phase 2): The workflow will analyze the codebase to create .github/actions/daily-test-improver/coverage-steps/action.yml containing CI steps to build the project, run tests, and produce a combined coverage report
• Phase 3 (after Phase 2 merges): The workflow will begin implementing actual test coverage improvements by writing new tests and submitting draft pull requests
• If running in repeat mode, the workflow will automatically advance through phases without manual intervention
• Humans can review this research and add comments to influence the direction before the workflow continues

AI generated by Daily Test Coverage Improver

• expires on Mar 13, 2026, 8:25 PM UTC

[github-actions[bot]]

Phase 2 complete: created .github/actions/daily-test-improver/coverage-steps/action.yml with steps to build, run tests with V8 coverage collection (yarn lerna run test:c8 --no-bail), generate a combined HTML report via c8, and upload as the coverage artifact. A draft PR titled "Daily Test Coverage Improver - Updates to complete configuration" has been submitted for review — please review and merge to proceed to Phase 3.

Generated by Daily Test Coverage Improver

To install this workflow, run gh aw add githubnext/agentics/workflows/daily-test-improver.md@42d5a50bca89dfba883981429036469ab90a4758. View source at https://github.com/githubnext/agentics/tree/42d5a50bca89dfba883981429036469ab90a4758/workflows/daily-test-improver.md.

[github-actions[bot]]

Phase 2 complete — I've created the coverage steps configuration at .github/actions/daily-test-improver/coverage-steps/action.yml and submitted a draft PR: Daily Test Coverage Improver - Updates to complete configuration.

The composite action sets up Node.js 22, installs dependencies with Yarn 4, builds all packages, runs yarn lerna run test:c8 --no-bail with NODE_V8_COVERAGE to collect V8 coverage across all packages, generates a combined HTML coverage report via c8 report --all, and uploads it as the coverage artifact. Note: in this agent's sandboxed environment the network is firewalled so yarn install couldn't be fully verified locally, but the steps match exactly what scripts/ci/generate-test-coverage-report.sh does and what the existing CI workflows use.

Generated by Daily Test Coverage Improver

To install this workflow, run gh aw add githubnext/agentics/workflows/daily-test-improver.md@42d5a50bca89dfba883981429036469ab90a4758. View source at https://github.com/githubnext/agentics/tree/42d5a50bca89dfba883981429036469ab90a4758/workflows/daily-test-improver.md.

[github-actions[bot]]

Phase 2 complete. I've created the coverage steps configuration in .github/actions/daily-test-improver/coverage-steps/action.yml and opened a pull request: Daily Test Coverage Improver - Updates to complete configuration.

The action builds all packages, runs yarn lerna run test:c8 --no-bail across all 38 packages that have a test:c8 script, generates a combined HTML coverage report via yarn c8 report --all, and uploads it as a coverage artifact. Once the PR is merged, Phase 3 will begin implementing actual test coverage improvements.

Generated by Daily Test Coverage Improver

To install this workflow, run gh aw add githubnext/agentics/workflows/daily-test-improver.md@42d5a50bca89dfba883981429036469ab90a4758. View source at https://github.com/githubnext/agentics/tree/42d5a50bca89dfba883981429036469ab90a4758/workflows/daily-test-improver.md.