Define, execute, and validate behavioral contracts for MCP tools. JSON suites run against a JOONS MCP Mock Server started by the runner. Part of the JOONS ecosystem: Probe diagnoses a real server, Diff compares snapshots and makes snapshot files, Policy Gate checks snapshots against policies, and Mock Server serves a mock from a snapshot and fixtures.
Check expected tool behavior consistently with readable assertions, capture chains, ordered reports, and CI-friendly exit codes.
Version 0.1.0 runs only against a JOONS MCP Mock Server. Before any tool call, the runner checks the server's name, title, and instructions. Real-server execution is not in 0.1.0; it is planned with explicit opt-in and an exact per-tool allowlist. Input is data, never executed. The child starts without a shell, in an empty temporary directory, with a minimal environment; it is cleaned up on all paths. See SECURITY.md for the library exception and limitations.
Install Node.js 20 or newer, then install the runner and Mock Server separately:
npm install -g github:dbdefault/joons-mcp-contract-runner#v0.1.0
npm install -g github:dbdefault/joons-mcp-mock-server#v0.1.0Alternatively pass the Mock Server's .js entry point with --mock-server PATH. Create a small suite such as suite.json:
{
"format": "joons-mcp-contract-test/suite",
"suiteVersion": 1,
"tests": [
{"id":"search", "tool":"search_products", "arguments":{"query":"boots"},
"expect":{"outcome":"result", "isError":false, "text":{"contains":"boots"}}}
]
}Using the included snapshot and fixtures from a checkout:
joons-mcp-contract-test validate suite.json --snapshot examples/snapshot.json
joons-mcp-contract-test run suite.json --snapshot examples/snapshot.json --fixtures examples/fixtures.jsonIf the globally installed Mock Server is not resolved from the working directory or runner installation, pass its .js path using --mock-server. More examples and the outage scenario are in examples/README.md.
A suite has format, suiteVersion, and one to 1000 ordered tests; optional name, defaults, and tools configure it. Each test has a unique id, a tool, and an expect, with optional arguments, timeout, skip and captures. Tool inclusion checks produce a synthetic #tools result. Unknown schema keys are errors. See suite reference.
| Assertion | Checks |
|---|---|
outcome |
result or JSON-RPC error |
isError, contentCount |
result error flag and number of content blocks |
text |
equals, contains, or startsWith on joined text blocks |
structuredContent |
exact equality or recursive subset match |
fields |
pointer existence, equality, captured equality, type or length bounds |
maxDurationMs |
elapsed time limit |
errorCode, errorMessageContains |
JSON-RPC error code and message |
Missing assertion inputs do not pass by default. See assertion examples.
Capture named values from a passing test's structured content using JSON pointers. Refer to an earlier capture in arguments with {"$capture":"name"}. Captures are copied, isolated between runs and omitted from reports. If a value is unavailable, dependent tests skip.
Concurrency defaults to 1 and ranges from 1 to 8. Independent tests may run in parallel; tests using or producing captures wait for earlier tests and run in order. Results remain in suite order. Per-test timeout defaults to 5000 ms. The total timeout defaults to 600000 ms and skips remaining tests when exceeded. A missing listed tool fails a result-expecting test without a call.
Output formats are terminal, JSON, and Markdown; --json-out and --md-out write files atomically. --stable makes durations zero and omits timestamps. Arguments and captures are omitted, sensitive keys are masked, and untrusted terminal/Markdown text is sanitized. Skips alone do not fail a run.
| Exit | Meaning |
|---|---|
| 0 | PASS, or --fail-on none; validation valid |
| 1 | FAIL verdict |
| 2 | invalid input, mock unavailable, startup or identity failure |
| 3 | usage error |
| 4 | output write error |
For a checkout with development dependencies, a GitHub Actions step can run:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- run: npm ci
- run: node scripts/test.js
- run: node bin/joons-mcp-contract-test.js validate examples/shop.suite.json --snapshot examples/snapshot.json
- run: node bin/joons-mcp-contract-test.js run examples/shop.suite.json --snapshot examples/snapshot.json --fixtures examples/fixtures.json --mock-server node_modules/joons-mcp-mock-server/bin/joons-mcp-mock-server.js --stableSee CI guidance and the workflow.
Suites are limited to 8 MiB, snapshots to 64 MiB, fixtures to 16 MiB; JSON complexity and test counts are bounded. See suite reference for details. Mock behavior reflects supplied snapshot and fixtures, not a real server's current behavior.
The CLI cannot run against real servers. The planned real-server design requires explicit opt-in and an exact per-tool allowlist. The library has an explicit allowNonMock escape hatch; it is not exposed by the CLI.
Preview, not on npm. Install from the GitHub tag shown above. Licensed under Apache-2.0.