Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .claude/skills/e2e-testing/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
description: How to author Playwright .NET (NUnit) end-to-end tests for the XnaFiddle Blazor WASM app — the publish→in-process-Kestrel→headless-Chromium harness, the SwiftShader GL args, the observable JS/DOM signals a test can synchronize on, and the null-and-wait per-run sync pattern. Load when adding or debugging an e2e test in XnaFiddle.E2E.Tests (e.g. issue #95 sample-switch guard), touching the SmokeTest fixture / StaticSiteHost / PublishOutput, or the e2e.yml CI job.
---

# E2E Testing

Test project: `XnaFiddle.E2E.Tests` (NUnit + `Microsoft.Playwright.NUnit`). `Nullable` is **enabled** here (unlike BlazorGL/Core — don't copy their nullable-off assumptions). `dotnet test` auto-discovers every `[Test]`, so **adding a test needs no CI edit**. Runtime domain facts (leaks, lifecycle, `UseReferenceDevice`) live in the **game-lifecycle** skill; this skill is only about authoring tests.

## The harness flow

1. `dotnet publish XnaFiddle.BlazorGL -c Release -o <dir>` produces the standalone WASM app.
2. `PublishOutput.ResolveWebRoot()` finds the served web root: honors env var `E2E_PUBLISH_DIR`, else falls back to `<repoRoot>/artifacts/e2e-publish`. Accepts either the publish root (has a `wwwroot`) or the `wwwroot` itself; throws if no `index.html`.
3. `StaticSiteHost.StartAsync(webRoot)` serves it in-process via Kestrel on an OS-assigned port (`http://127.0.0.1:0`). Uses `UseBlazorFrameworkFiles` so `_framework` Webcil assemblies get `application/wasm` (a plain static server serves octet-stream, which the browser refuses). `MapFallbackToFile("index.html")` for deep links.
4. Playwright drives headless Chromium against `host.BaseUrl`.

`OneTimeSetUp`/`OneTimeTearDown` in `SmokeTest.cs` own host + browser + page lifetime; individual tests just navigate and assert.

**Locally:** if `artifacts/e2e-publish/wwwroot/index.html` already exists, skip publish — just `dotnet test`. CI (`.github/workflows/e2e.yml`) does publish → build test proj → `playwright.ps1 install chromium` → `dotnet test --no-build`.

## Windows-only publish (CI gotcha)

The e2e CI job runs on **windows-latest**, not Linux. Reason (summarized from `e2e.yml`'s top comment): a vendored KNI csproj references `Xna.Framework.Media.csproj` but the file on disk is `XNA.Framework.Media.csproj` (uppercase). Case-insensitive Windows resolves it; case-sensitive Linux fails the publish with CS0234/CS0246. Windows is also the app's real deploy target.

## SwiftShader Chromium args (headless software GL)

CI has no GPU, so launch Chromium with ANGLE-over-SwiftShader:

```
--use-gl=angle --use-angle=swiftshader --enable-unsafe-swiftshader
```

`--enable-unsafe-swiftshader` is **required** on recent Chromium to permit SwiftShader for WebGL (without it, WebGL context creation is blocked). These are set in `LaunchAsync` `Args`.

## Observable signals — what to synchronize on

Prefer these existing signals; they need no app-code change. What each proves:

| Signal | Proves |
|---|---|
| `#theCanvas` present **and** `window.theInstance != null` | App booted, render bridge wired (`theInstance` set by `initRenderJS`). Boot gate. |
| `[data-testid="run-button"]` clickable | The Run/Restart button. **Same testid for both labels** (label toggles on `_game != null`). While `_isCompiling` it's replaced by a **testid-less Stop button**, so Playwright's `ClickAsync` actionability auto-wait naturally waits out the compile. |
| `window._canvasContextType === 'webgl'` (Reach) or `'webgl2'` (HiDef) | **THE success signal**: a game ran and a WebGL context initialized (Roslyn compiled + KNI ran + GL came up). Written at the **end of every successful run** in `LaunchGameFromTypeAsync` (`Index.razor.cs`); reset to `null` **only on a profile switch**. |
| `#blazor-error-ui` **not** visible | No unhandled error killed the circuit (`display:none` via CSS unless an error surfaces). Assert it stays hidden. |

## The null-and-wait per-run sync pattern (keystone)

Because `_canvasContextType` **stays set across same-profile restarts**, its mere presence can't prove that a *specific* restart finished. To gate on one run: null it before the click, then wait for the app to re-set it.

```csharp
await page.EvaluateAsync("() => { window._canvasContextType = null; }");
await ClickRunAsync(); // auto-waits out the compile
await WaitForWebGlContextAsync(); // passes only when THIS run re-sets it
```

Deterministic, no app-code change. The restart loop in `RepeatedRestart_UnchangedSource_StaysAliveAndKeepsWebGl` is the reference example.

## When to add an app-code hook

Rule of thumb: use the existing observable signals above; add a new app debug hook **only** for input/game-state assertions the DOM can't express. Example: issue #95's planned `GetInputDebugState` (exposing `Mouse.GetState()` position / touch count via JS interop) for a deterministic input assertion instead of a flaky pixel-read. Don't add hooks for anything already observable.

## Choosing N for restart/stress loops

Chrome caps live WebGL contexts at ~16. A loop guarding the per-run context-leak fix (`UseReferenceDevice` — see **game-lifecycle**) must **exceed** that so a regression trips context loss around run ~16 and the test catches it. The existing restart test uses **20**.

## Timeouts

Existing consts are **60s** (`BootTimeoutMs`, `RunTimeoutMs`) — the WASM payload is ~15-20MB and in-browser Roslyn is slow. If flaky, **tune these up**, don't weaken the assertion.

## Related

- **game-lifecycle** — the runtime facts these tests guard (leak, restart rebuild, profile switch).
- **issue-workflow** — branch/PR process when a test lands with an issue.
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ XnaFiddle is a standalone KNI game runner with an in-browser C# editor. It is a

On **any** task — research, maintenance, new features, bug fixes, refactors, docs, reviews — first check whether an available skill applies, and load it before proceeding. Skills (e.g. `file-loading`, `intellisense`, `verify`, `run`, `code-review`) carry condensed, repo-specific knowledge that prevents rediscovering subsystem details from scratch. When a skill matches the work, invoke it; if none fit, say so briefly and continue. This check is in addition to, not a replacement for, the Agent Workflow below.

**Proactively suggest a new or updated skill** when — and only when — you finish work having re-derived reusable subsystem knowledge that no skill covers, or having found an existing skill stale/wrong. Don't wait to be asked; surface it as a suggestion and let the user decide (don't create one unprompted). But the bar is **high**: every skill line costs tokens on every relevant load, so err toward *not* proposing. Only suggest when you're confident the knowledge will be re-hit by a concrete upcoming task, not for one-off facts the code already records or that this conversation alone needed. When unsure, stay silent.

## Agent Workflow

For every task, invoke the appropriate agent from `.claude/agents/` before proceeding. The agent's instructions provide guidelines for how the task should be performed. Before doing any work, announce which agent you are using such as "Invoking coder agent for this task..."
Expand Down
69 changes: 60 additions & 9 deletions XnaFiddle.E2E.Tests/SmokeTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
namespace XnaFiddle.E2E.Tests;

/// <summary>
/// Phase 1 e2e smoke test (issue #97): boot the published Blazor WASM app under headless
/// Phase 1 e2e smoke tests (issue #97): boot the published Blazor WASM app under headless
/// Chromium with software GL (SwiftShader) and prove the default sample compiles (Roslyn),
/// runs (KNI), and initializes a WebGL context. This is the class of regression (#90/#95)
/// that unit tests cannot reach because the browser-only KNI/WebGL code no-ops under net8.0.
Expand All @@ -17,6 +17,12 @@ public sealed class SmokeTest
private const int BootTimeoutMs = 60_000;
private const int RunTimeoutMs = 60_000;

// Restart the game this many times with unchanged source (issue #90 guard). Each Restart
// rebuilds the whole Game/GraphicsDevice, so this must exceed Chrome's ~16 live-WebGL-context
// cap: if the UseReferenceDevice leak fix (see game-lifecycle skill) ever regresses, the
// per-run OffscreenCanvas probe leak trips context loss around run ~16, which this would catch.
private const int RestartCount = 20;

private StaticSiteHost _host = null!;
private IPlaywright _playwright = null!;
private IBrowser _browser = null!;
Expand Down Expand Up @@ -61,6 +67,45 @@ public async Task OneTimeTearDown()

[Test]
public async Task DefaultSample_BootsRunsAndInitializesWebGl()
{
await BootAsync();

await ClickRunAsync();
await WaitForWebGlContextAsync();

await AssertNoBlazorErrorAsync("after first run");
}

/// <summary>
/// Issue #90 restart guard: clicking Restart repeatedly with unchanged source must not kill
/// the Blazor circuit (error banner) or lose the WebGL context. Each iteration nulls
/// window._canvasContextType and waits for the running game to re-set it — a deterministic,
/// app-code-free signal that this specific restart cycled a fresh game + GL context to
/// completion (DoCompileAndRun sets it at the end of every successful run).
/// </summary>
[Test]
public async Task RepeatedRestart_UnchangedSource_StaysAliveAndKeepsWebGl()
{
await BootAsync();

// First click compiles + runs; every later click is a cache-hit Restart (source unchanged).
await ClickRunAsync();
await WaitForWebGlContextAsync();

for (int i = 1; i <= RestartCount; i++)
{
// Clear the success signal so the wait below can only pass once THIS restart re-sets it,
// rather than observing the previous run's still-set value.
await _page.EvaluateAsync("() => { window._canvasContextType = null; }");

await ClickRunAsync();
await WaitForWebGlContextAsync();

await AssertNoBlazorErrorAsync($"on restart {i} of {RestartCount}");
}
}

private async Task BootAsync()
{
await _page.GotoAsync(_host.BaseUrl, new PageGotoOptions
{
Expand All @@ -77,23 +122,29 @@ await _page.WaitForFunctionAsync(
"() => window.theInstance !== undefined && window.theInstance !== null",
null,
new PageWaitForFunctionOptions { Timeout = BootTimeoutMs, PollingInterval = 250 });
}

// Run the default sample.
await _page.ClickAsync("[data-testid=\"run-button\"]", new PageClickOptions
// ClickAsync auto-waits for actionability, so it naturally waits out the transient "Stop"
// (compiling) button and clicks the Run/Restart button once it is back and enabled.
private Task ClickRunAsync() =>
_page.ClickAsync("[data-testid=\"run-button\"]", new PageClickOptions
{
Timeout = BootTimeoutMs,
});

// Deterministic success signal: .NET sets window._canvasContextType once a game runs and
// a WebGL context initializes ('webgl' = Reach, 'webgl2' = HiDef). Either proves that
// Roslyn compiled + KNI ran + a GL context came up under SwiftShader.
await _page.WaitForFunctionAsync(
// Deterministic success signal: .NET sets window._canvasContextType once a game runs and a
// WebGL context initializes ('webgl' = Reach, 'webgl2' = HiDef). Either proves that Roslyn
// compiled + KNI ran + a GL context came up under SwiftShader.
private Task WaitForWebGlContextAsync() =>
_page.WaitForFunctionAsync(
"() => window._canvasContextType === 'webgl' || window._canvasContextType === 'webgl2'",
null,
new PageWaitForFunctionOptions { Timeout = RunTimeoutMs, PollingInterval = 500 });

// #blazor-error-ui is display:none via CSS unless an unhandled error surfaces.
// #blazor-error-ui is display:none via CSS unless an unhandled error surfaces.
private async Task AssertNoBlazorErrorAsync(string when)
{
bool errorVisible = await _page.Locator("#blazor-error-ui").IsVisibleAsync();
Assert.That(errorVisible, Is.False, "Blazor error banner should not be displayed.");
Assert.That(errorVisible, Is.False, $"Blazor error banner should not be displayed ({when}).");
}
}