test(tui): interactive terminal smoke suite + CI workflow#121
Merged
Conversation
Drive the real coven-code binary through a tmux pseudo-terminal, send keystrokes, capture the rendered pane, and assert on output. Offline / headless-first: no live model call, network, or credentials required. Covers headless CLI surface (version/help/unknown-flag/auth-status/models/ dump-system-prompt), TUI startup chrome, slash command palette + filtering, help overlay, prompt input editing, and quit. Cases poll with wait_for/ wait_absent to avoid render races; a dedicated tmux socket isolates the harness from the user's own sessions. CI (.github/workflows/tui-tests.yml) builds the debug binary, seeds offline settings so a credential-less runner reaches the main screen, runs the suite with REQUIRE_TMUX=1, and uploads pane captures as artifacts on failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
Adds a new interactive-terminal (TUI) smoke test harness that drives the real coven-code binary via a dedicated tmux server, plus a GitHub Actions workflow to run the suite on relevant pushes/PRs. This strengthens release confidence by asserting on actual rendered TUI output while staying offline/headless-first.
Changes:
- Introduces a Bash-based test harness (
run.sh+lib.sh) and six initial smoke cases (headless CLI + key TUI interactions). - Adds documentation for running/extending the suite and its environment/config knobs.
- Adds a CI workflow to build the binary, seed offline settings, run the suite, and upload failure pane captures as artifacts.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/tui-tests/run.sh | Entry point for discovering/building the binary, loading cases, running the suite, and printing a summary. |
| scripts/tui-tests/lib.sh | Core harness utilities: binary discovery, assertions, tmux session lifecycle, capture/poll helpers. |
| scripts/tui-tests/README.md | Suite documentation: usage, CI behavior, configuration, and how to author new cases. |
| scripts/tui-tests/cases/01_headless.sh | Offline CLI surface smoke checks (--version, --help, auth status, models, etc.). |
| scripts/tui-tests/cases/02_startup.sh | Validates initial TUI chrome (banner, footer hints, no panic strings). |
| scripts/tui-tests/cases/03_command_palette.sh | Exercises command palette open/filter and Ctrl+K entry point. |
| scripts/tui-tests/cases/04_help_overlay.sh | Exercises help overlay open/close and content presence. |
| scripts/tui-tests/cases/05_input_editing.sh | Exercises typed input echo and Ctrl+U clear behavior. |
| scripts/tui-tests/cases/06_quit.sh | Exercises Ctrl+C twice quit behavior and shell return. |
| .github/workflows/tui-tests.yml | CI workflow to install deps, build, seed offline settings, run suite, and upload failure captures. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+171
to
+183
| # wait_for <substring> [timeout-seconds] -> 0 found / 1 timeout | ||
| wait_for() { | ||
| local needle="$1" timeout="${2:-$TUI_WAIT_TIMEOUT}" | ||
| local waited=0 | ||
| # bash arithmetic is integer; scale by 10 to honor sub-second poll interval | ||
| local step_ms=400 budget_ms=$((timeout * 1000)) elapsed_ms=0 | ||
| while [ "$elapsed_ms" -lt "$budget_ms" ]; do | ||
| if tui_capture | grep -qF -- "$needle"; then return 0; fi | ||
| sleep "$TUI_POLL_INTERVAL" | ||
| elapsed_ms=$((elapsed_ms + step_ms)) | ||
| done | ||
| return 1 | ||
| } |
Comment on lines
+185
to
+196
| # wait_absent <substring> [timeout-seconds] -> 0 once gone / 1 still present | ||
| # Mirror of wait_for for disappearance (e.g. an overlay closing). | ||
| wait_absent() { | ||
| local needle="$1" timeout="${2:-$TUI_WAIT_TIMEOUT}" | ||
| local step_ms=400 budget_ms=$((timeout * 1000)) elapsed_ms=0 | ||
| while [ "$elapsed_ms" -lt "$budget_ms" ]; do | ||
| if ! tui_capture | grep -qF -- "$needle"; then return 0; fi | ||
| sleep "$TUI_POLL_INTERVAL" | ||
| elapsed_ms=$((elapsed_ms + step_ms)) | ||
| done | ||
| return 1 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an interactive-terminal (TUI) smoke test suite plus a CI workflow that runs it. The suite drives the real
coven-codebinary through atmuxpseudo-terminal, sends keystrokes, captures the rendered pane, and asserts on the output — automating the manual tmux workflow documented inAGENTS.md.Offline / headless-first: no test makes a live model call, hits the network, or needs credentials. Deterministic and runnable on any machine with the binary and
tmux.What it covers
01_headless--version,--help, unknown-flag exit code,auth status,models,--dump-system-prompt02_startup03_command_palette/opens palette, live filtering,Ctrl+Kentry point04_help_overlay?opens keybinding + command reference,Esccloses05_input_editingCtrl+Uclears06_quitCtrl+Ctwice exits cleanlyRobustness
wait_for/wait_absentinstead of fixed sleeps, eliminating render races (verified 5x back-to-back green).-L) so the harness never touches the user's own tmux sessions.TUI_LOG_DIRand uploaded as CI artifacts.CI
.github/workflows/tui-tests.ymlruns onmainpushes and PRs touchingsrc-rust/**or the suite (plus manual dispatch). It buildscargo build --locked --package claurst, seeds offline settings so a credential-less runner reaches the main screen, runs the suite withREQUIRE_TMUX=1, and uploads failure captures.Verification
env -i, no creds): 38/38 passed.--lockedbuild, and YAML all validated locally.Run it
🤖 Generated with Claude Code