mcp-assert
ActionsAbout
Deterministic correctness testing for MCP servers in CI. One line to add assertion testing to any MCP server repo.
- uses: blackwell-systems/mcp-assert-action@v1
with:
suite: evals/
server: "npx @modelcontextprotocol/server-filesystem ."
fixture: test/fixtures/| Input | Required | Default | Description |
|---|---|---|---|
suite |
Yes | — | Directory containing assertion YAML files |
server |
No | — | Override server command for all assertions |
fixture |
No | — | Fixture directory ({{fixture}} substitution) |
threshold |
No | 100 |
Minimum pass percentage |
timeout |
No | 30s |
Per-assertion timeout |
version |
No | latest |
mcp-assert version to install |
baseline |
No | — | Baseline JSON for regression detection |
fail-on-regression |
No | false |
Fail if previously-passing assertions regress |
| Output | Description |
|---|---|
passed |
Number of assertions that passed |
failed |
Number of assertions that failed |
total |
Total number of assertions |
name: Test MCP Server
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install server
run: npm install -g @modelcontextprotocol/server-filesystem
- uses: blackwell-systems/mcp-assert-action@v1
with:
suite: evals/
fixture: test/fixtures/- uses: blackwell-systems/mcp-assert-action@v1
with:
suite: evals/
server: "python -m my_mcp_server"
threshold: 95
timeout: 60s- uses: blackwell-systems/mcp-assert-action@v1
with:
suite: evals/
baseline: baseline.json
fail-on-regression: true- Downloads the
mcp-assertbinary from GitHub Releases - Runs
mcp-assert ciwith your configuration - Uploads JUnit XML results as an artifact
- Uploads shields.io badge JSON as an artifact
- Writes GitHub Step Summary automatically
- Fails the step if pass rate is below threshold
Once your assertions pass, add the badge to your server's README:
[](https://github.com/blackwell-systems/mcp-assert)Three variants are available:
| Preview | Variant | Markdown |
|---|---|---|
 |
Passing | [](https://github.com/blackwell-systems/mcp-assert) |
 |
Score | Download SVG, edit the count, host in your repo |
 |
Failing | [](https://github.com/blackwell-systems/mcp-assert) |
For dynamic badges that reflect your CI pass rate, see the Badge guide.
Each assertion is a YAML file that calls one MCP tool and checks the result:
# evals/read_file.yaml
name: read_file returns file contents
server:
command: npx
args: ["@modelcontextprotocol/server-filesystem", "{{fixture}}"]
assert:
tool: read_file
args:
path: "{{fixture}}/hello.txt"
expect:
not_error: true
contains: ["Hello, world!"]
timeout: 15sserver— how to start your MCP server (binary + args)assert.tool— which MCP tool to callassert.args— arguments passed to the toolassert.expect— deterministic checks on the response (contains,not_error,equals,is_error,json_path,min_results, etc.){{fixture}}— replaced with thefixtureinput path at runtimesetup— optional list of tool calls to run before the assertion (for stateful tests)
See mcp-assert documentation for the full assertion format reference and example suites.
MIT
mcp-assert is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.
About
mcp-assert is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.