mirror of
https://github.com/nesquena/hermes-webui.git
synced 2026-07-20 14:40:35 +00:00
nesquena-hermes
f9acf464e9
Completes the test-sharding half of #3197 (Docker-cache half shipped v0.51.177). Adds pytest-shard 3-way split to tests.yml (3 shards x 3 Python = 9 jobs, fail-fast: false). pytest-shard is 0-indexed so the matrix uses [0,1,2] — the original #3197 used [1,2,3] which would have crashed the out-of-range job and silently skipped shard 0's tests. Made the suite shard-safe by fixing 4 cross-test state-pollution bugs that passed sequentially but failed when sharded: - test_onboarding_mvp: reset onboarding_completed flag (settings.json) in the autouse fixture; the config-cleanup only cleared config.yaml/.env. - test_issue693_system_health_panel: invalidate the process-wide password-hash cache before/after so a prior test's "no password" cache doesn't defeat the auth-gate assertion. - test_auth_session_persistence: assert against auth._SESSIONS_FILE (where auth actually writes) instead of a local _TEST_STATE path that only matched under a lucky import order. - test_profile_env_isolation (root cause of the worst leak): stop deleting + re-importing api.profiles under a temp HERMES_BASE_HOME — that swapped the module object and poisoned the cached _DEFAULT_HERMES_HOME for every later test (broke test_title_aux_routing's load_config). Now points the cached path via monkeypatch.setattr (auto-restored, no module swap). - conftest: autouse fixture restores HERMES_HOME/HERMES_BASE_HOME after each test as defense-in-depth against future switch_profile leaks. Verified: all 3 shards green (6912 passed, 0 failed); full sequential run still green (6957 passed, 0 failed). Slowest shard ~70s vs ~180s sequential.
54 lines
2.1 KiB
YAML
54 lines
2.1 KiB
YAML
name: Tests
|
|
|
|
on:
|
|
pull_request:
|
|
branches: [master]
|
|
push:
|
|
branches: [master]
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
strategy:
|
|
# Don't cancel the other shards/versions when one fails — we want the full
|
|
# failure picture across the matrix in a single run.
|
|
fail-fast: false
|
|
matrix:
|
|
python-version: ['3.11', '3.12', '3.13']
|
|
# Split the suite across 3 parallel shards per Python version. pytest-shard
|
|
# partitions tests deterministically by test-id hash; the suite was made
|
|
# shard-safe (no cross-test state leakage) so every shard passes
|
|
# independently. See docs/agent-memory note on test-suite shard-safety.
|
|
# NOTE: pytest-shard is 0-indexed — shard ids must be 0..num_shards-1.
|
|
# Using 1-based ids would crash the out-of-range job AND silently skip
|
|
# shard 0's tests.
|
|
shard: [0, 1, 2]
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Set up Python ${{ matrix.python-version }}
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
cache: 'pip'
|
|
cache-dependency-path: |
|
|
**/setup.cfg
|
|
**/requirements*.txt
|
|
**/pyproject.toml
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install "pyyaml>=6.0" pytest pytest-timeout pytest-asyncio pytest-shard
|
|
# Install the `mcp` package so tests/test_mcp_server.py runs in CI.
|
|
# The package is an optional runtime dep of mcp_server.py — users
|
|
# who run the MCP integration install it themselves; CI installs
|
|
# it so test coverage exists. If mcp install fails (Python 3.13
|
|
# wheel not yet available, etc.), tests/test_mcp_server.py uses
|
|
# importorskip and the matrix stays green.
|
|
pip install mcp || echo "mcp install failed — test_mcp_server.py will importorskip"
|
|
|
|
- name: Run tests (shard ${{ matrix.shard }} of 3)
|
|
run: pytest tests/ -v --timeout=60 --shard-id=${{ matrix.shard }} --num-shards=3
|