Skip to content

Commit 735789a

Browse files
committed
Add test runner registry implementation for v2
This commit introduces a new source file `test_runner_registry.c` that implements the test runner registry for version 2. The file includes definitions for various test runner modules and profiles, along with functions to append sources for different components such as arena, lexer, parser, evaluator, and build model. Additionally, it defines watch roots for each module to facilitate automatic watching of relevant source files during testing. The new structure enhances the organization and maintainability of the test runner system.
1 parent afdfb40 commit 735789a

28 files changed

Lines changed: 6724 additions & 2274 deletions

docs/architecture/code_regions_map.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,9 @@ Region responsibilities map to the same product boundary:
5555
- `src_v2/transpiler/`
5656
- `src_v2/build_model/`
5757
- `src_v2/codegen/`
58-
- `src_v2/build/nob_test.c`
58+
- `src_v2/build/nob.c`
59+
- `src_v2/build/nob_testd.c`
60+
- `src_v2/build/test_runner_*`
5961
- `test_v2/`
6062

6163
## Failure Ownership

docs/architecture/pipeline_overview.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,10 @@ No layer may bypass the next contract boundary for feature delivery.
5858
- `src_v2/transpiler/event_ir.*`
5959
- `src_v2/build_model/*`
6060
- `src_v2/codegen/*`
61-
- `test_v2/*` and `src_v2/build/nob_test.c`
61+
- `src_v2/build/nob.c`
62+
- `src_v2/build/nob_testd.c`
63+
- `src_v2/build/test_runner_*`
64+
- `test_v2/*`
6265

6366
## Failure Ownership
6467

docs/evaluator/evaluator_coverage_matrix.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ divergences.
3636

3737
## Differential Program
3838

39-
- Runner: `./build/nob_test test-evaluator-diff`
39+
- Runner: `./build/nob test evaluator-diff`
4040
- CMake resolution: `CMK2NOB_TEST_CMAKE_BIN`, then `PATH`
4141
- Version gate: only `cmake 3.28.x` participates; otherwise the suite skips
4242
- Optional report output: `NOB_DIFF_STATUS_OUT=/abs/path/report.md`
@@ -58,7 +58,7 @@ divergences.
5858
- Harness details, DSL, snapshot format, and failure artifacts live in
5959
[test_v2/evaluator_diff/README.md](../../test_v2/evaluator_diff/README.md)
6060
- Dedicated CI coverage lives in `.github/workflows/evaluator-diff.yml` and
61-
stays separate from the default `test-v2` smoke aggregate
61+
stays separate from the default smoke aggregate
6262

6363
## Audit Matrix
6464

docs/tests/README.md

Lines changed: 36 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,13 @@
55
This directory is the canonical map for test architecture, suite ownership,
66
and active test-infrastructure roadmaps.
77

8-
Current baseline architecture boundary:
9-
10-
`nob_test runner -> test framework -> shared support -> suites`
11-
12-
Active daemon-program target boundary:
13-
148
`nob front door -> daemon client/supervisor -> reactor daemon -> runner core -> suites`
159

1610
## Overview
1711

1812
The test stack separates:
1913

20-
- aggregate-safe suites (`test-v2`) used as the default smoke baseline
14+
- aggregate-safe suites used as the default smoke baseline
2115
- explicit-only host-sensitive suites used for heavy parity/closure proof
2216

2317
This documentation defines test ownership and execution policy. It does not
@@ -35,26 +29,51 @@ redefine evaluator, Event IR, build-model, or codegen product contracts.
3529

3630
## Execution Policy Summary
3731

38-
Current baseline commands still run through `./build/nob_test` today:
32+
Current supported human-facing baseline commands now run through `./build/nob test`:
3933

4034
- default smoke aggregate:
41-
`./build/nob_test test-v2`
35+
`./build/nob test`
36+
`./build/nob test smoke`
37+
- front-doored utility commands:
38+
`./build/nob test clean`
39+
`./build/nob test tidy all`
40+
`./build/nob test tidy <module>`
41+
- watch mode:
42+
`./build/nob test watch <module>`
43+
`./build/nob test watch auto`
44+
- daemon lifecycle:
45+
`./build/nob test daemon start`
46+
`./build/nob test daemon stop`
47+
`./build/nob test daemon status`
4248
- explicit artifact parity suite:
43-
`./build/nob_test test-artifact-parity`
49+
`./build/nob test artifact-parity`
4450
- explicit closure harness suite:
45-
`./build/nob_test test-evaluator-codegen-diff`
51+
`./build/nob test evaluator-codegen-diff`
4652

47-
The active daemon roadmap treats those commands as transitional and targets:
53+
The active daemon roadmap now lands its T6 surface on:
4854

49-
- `./build/nob test test-v2`
50-
- `./build/nob test artifact-parity`
51-
- `./build/nob test evaluator-codegen-diff`
55+
- `./build/nob test`
56+
- `./build/nob test smoke`
57+
- `./build/nob test clean`
58+
- `./build/nob test tidy all`
59+
- `./build/nob test tidy <module>`
5260
- `./build/nob test watch <module>`
61+
- `./build/nob test watch auto`
5362
- `./build/nob test daemon start|stop|status`
63+
- `./build/nob test artifact-parity`
64+
- `./build/nob test evaluator-codegen-diff`
65+
- daemon front-door profile flags:
66+
`--verbose`, `--asan`, `--ubsan`, `--msan`, `--san`, `--cov`
5467

5568
That daemon target is intentionally Linux-first and now freezes watch routing,
56-
cancellation, and fast local feedback as part of the same program, not as
57-
separate later add-ons.
69+
cancellation, fast local feedback, and compact failure-first watch ergonomics
70+
as part of the same program, not as separate later add-ons.
71+
72+
Current T6 limitation:
73+
- watch sessions are foreground and attached to the active client; there is no
74+
persistent detached watch session surface yet.
75+
- default watch mode is compact and failure-first; use `--verbose` to print
76+
roots, full routed path/module sets, and per-rerun fast-path detail.
5877

5978
`artifact-parity` and `evaluator-codegen-diff` remain outside default smoke
6079
while they stay heavier and host-sensitive.

docs/tests/evaluator_codegen_diff.md

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ As of April 6, 2026, `test-evaluator-codegen-diff` is the canonical
66
explicit-only closure harness for the remaining
77
`evaluator -> Event IR -> build_model -> codegen` gap.
88

9-
It is intentionally heavier than `test-v2`, remains outside the default smoke
9+
It is intentionally heavier than the default smoke aggregate, remains outside the default smoke
1010
aggregate, and is organized around explicit multi-wave closure work rather than
1111
one undifferentiated backend backlog.
1212

@@ -200,18 +200,12 @@ Reasons:
200200
not the default smoke tier
201201

202202
It therefore does not participate in the current baseline
203-
`./build/nob_test test-v2` smoke aggregate.
204-
205-
Today the suite is exposed through:
206-
207-
`./build/nob_test test-evaluator-codegen-diff`
208-
209-
The active daemon roadmap treats that command as transitional and targets:
203+
`./build/nob test` / `./build/nob test smoke` smoke aggregate.
210204

211205
`./build/nob test evaluator-codegen-diff`
212206

213207
Like the other explicit-only host suites, it inherits runner-owned workspace,
214-
locking, logging, and tool-resolution behavior from `src_v2/build/nob_test.c`.
208+
locking, logging, and tool-resolution behavior from `src_v2/build/test_runner_*`.
215209

216210
## Reporting
217211

docs/tests/test_daemon_roadmap.md

Lines changed: 32 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,27 +5,35 @@ rewrite whose sole goal is to reach a fast persistent test daemon as quickly
55
as possible.
66

77
This document supersedes the old preserved-entrypoint constraint around
8-
`./build/nob_test` for this program. The current baseline runner still exists
9-
today, but this roadmap explicitly authorizes replacing it with a new
10-
`./build/nob`-owned client/daemon architecture even if that breaks the old
11-
test command surface between waves.
8+
`./build/nob_test` for this program. The legacy shim is now gone; this roadmap
9+
explicitly authorized replacing it with a new `./build/nob`-owned
10+
client/daemon architecture even when that broke the old test command surface
11+
between waves.
1212

1313
This roadmap does not redefine evaluator, Event IR, build-model, codegen, or
1414
`nobify` product contracts. It only defines how local and CI-facing test
1515
infrastructure should evolve to maximize development speed.
1616

17-
## 1. Status
17+
Historical wave descriptions are preserved below for implementation history.
18+
Any remaining mentions of `./build/nob_test` or `src_v2/build/nob_test.c`
19+
inside those earlier-wave sections are historical only, not current guidance.
1820

19-
As of April 8, 2026:
21+
## 1. Status
2022

21-
- the current baseline test entrypoint is still `./build/nob_test`
22-
- `src_v2/build/nob.c` does not yet own test dispatch
23-
- `src_v2/build/nob_test.c` still owns module selection, profiles, incremental
24-
compilation, workspaces, logs, and preflight behavior
25-
- the project now wants a faster path to a persistent daemon than the old
26-
compatibility-first test architecture allowed
23+
As of April 9, 2026:
24+
25+
- `./build/nob test ...` is the official human-facing test entrypoint
26+
- `build/nob_testd` already owns the daemon-backed run path
27+
- watch mode and impact routing already exist behind `./build/nob test watch ...`
28+
- fast profile, daemon-side preflight caching, launcher detection, and parallel
29+
object compilation have already landed
30+
- T5 throughput observability and compact watch ergonomics have already landed
31+
- T6 legacy surface removal has already landed:
32+
- `build/nob_test` has been removed
33+
- `clean` and `tidy` are fronted through `./build/nob test ...`
34+
- `smoke` is the only supported public aggregate label
2735
- only the `nobify` product needs portability; test infrastructure does not
28-
- inter-wave breakage is acceptable if it shortens the path to the daemon
36+
- inter-wave breakage remains acceptable if it shortens time-to-daemon
2937

3038
The target end state of this roadmap is:
3139

@@ -64,12 +72,13 @@ The following decisions are frozen for this roadmap:
6472

6573
- only `nobify` product portability matters
6674
- test infrastructure may stay Linux/POSIX-only indefinitely
67-
- `./build/nob_test` is transitional and may be removed later
75+
- `./build/nob_test` has been removed; any remaining mentions below are
76+
historical wave context only
6877
- `./build/nob` becomes the official human entrypoint for tests
6978
- `src_v2/build/nob.c` adopts `NOB_GO_REBUILD_URSELF_PLUS(...)` and owns test
7079
dispatch
71-
- `src_v2/build/nob_test.c` may remain temporarily, but only as a wrapper or
72-
compatibility shim once runner-core extraction begins
80+
- `src_v2/build/nob_test.c` has been deleted; earlier wave text may still
81+
mention it as historical context
7382
- the new daemon binary is `build/nob_testd`
7483
- the transport is a pathname `AF_UNIX` socket under `Temp_tests/daemon/`
7584
- the socket type is `SOCK_SEQPACKET`, not `SOCK_STREAM`
@@ -169,6 +178,10 @@ as a contract that must survive intact.
169178

170179
## 6. Wave Plan
171180

181+
The wave descriptions below intentionally preserve the state and assumptions
182+
that were true when each wave was defined. T6 completion supersedes any older
183+
wave text that still refers to `./build/nob_test` as if it were current.
184+
172185
### T0 Contract Reset And Program Start
173186

174187
Goal:
@@ -318,6 +331,8 @@ Deliverables:
318331
- better invalidation/reporting for fast-profile cache hits versus misses
319332
- explicit fallback routing policy remains visible in watch output when a path
320333
is unknown or too broad
334+
- default watch output becomes compact/failure-first, with `--verbose` as the
335+
inspection path for roots, routed sets, and per-rerun detail
321336

322337
Non-goals:
323338
- no full include-graph intelligence
@@ -334,7 +349,7 @@ Goal:
334349
- remove transitional clutter once the daemon path is stable
335350

336351
Deliverables:
337-
- `build/nob_test` removed or replaced by a tiny documented compatibility shim
352+
- `build/nob_test` removed
338353
- docs stop presenting `./build/nob_test` as primary
339354
- architecture docs describe `nob` as client/supervisor and `nob_testd` as the
340355
reactor execution owner

0 commit comments

Comments
 (0)