Skip to content

ci(ui-merge): Test UI runs only the ui tests a pull request or merge group affects — the root ci.yml wires ui/scripts/select-affected-tests.mjs - #25882

Merged
os-zhuang merged 3 commits into
mainfrom
claude/project-thread-n0iri6
Oct 26, 2026
Merged

os-zhuang merged 3 commits into
mainfrom
claude/project-thread-n0iri6

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Tesla · project thread

Fixes #25868
Part of #25677

Clause-②: no
Metadata-protocol: none

维护者速读(草稿)

  • 改了什么: 根 ci.yml 的 Test UI(16 个分片)在 pull request 和 merge queue 上不再每次跑全量 ui 测试,而是先用 objectui 原有的 ui/scripts/select-affected-tests.mjs 选出本次改动影响到的测试文件,只跑这些。push 到 main、每小时的 schedule 和手动触发仍然跑全量。
  • 为什么改: 合并后每个 ui PR 都跑 16 个全量分片(每片 7–14 分钟,约 190 runner-min)。在 objectui 仓库时,小 PR 只跑受影响的测试,CI 3–8 分钟。
  • 风险与代价(含回滚): 选择器漏掉的测试,要等 main 上的全量运行(每小时一次)才会发现,这是 objectui 原来就接受的取舍。选择器出错、拿不到 diff 基点时一律回退到全量,不会跳过。ui/ 之外影响所有 ui 测试的改动(根目录的 lockfile、workspace、turbo.json 等非文档文件,patches/,以及 ui 测试读取其构建产物的 7 个 @objectstack 包)一律全量。回滚:revert 本 PR 即恢复每次全量。
  • 席位意见:
  • 你要做的: 这是 Tier H(改了 .github/workflows/ci.yml),需要你批准后由席位落地。

What changed

  1. .github/workflows/ci.yml, job test-ui — objectui's two-step wiring, ported:
    • Select the tests this change affects (id select): pull_request and merge_group run the selector; every other event writes mode=full. Fails open: no base, a non-zero exit or no verdict line means mode=full, never a skip. The exit status is captured beside the output, not through a command substitution.
    • The PR diff base is merge-base(origin/BASE_REF, HEAD), the same base Test Core uses, not the payload's frozen base.sha. With the frozen sha, every commit main gained while the PR was open (a lockfile bump among them) would be charged to the PR and turn it into a full run. A merge group diffs from merge_group.base_sha.
    • Run the ui tests (shard N/16) reads the output's VALUE (affected / full). On affected it runs pnpm --dir ui exec vitest run --shard=N/16 ... FILTERS with the filters relative to ui/ (the vitest root), as positionals with no separator, which is the only form ui/scripts/vitest-invocation-guard.mjs accepts. On full it runs the command it ran before.
    • The checkout takes fetch-depth: 0 so the base's history is there (Test Core and Type Check UI do the same).
  2. ui/scripts/select-affected-tests.mjs — the git paths already went through --relative (tooling(objectui-readiness 4): objectui scripts address git paths relative to the repo root — git show ref:packages/…, diff --name-only, ls-files, ls-tree break one level down in ui/; prefix via nestedPrefix() (parent #25677, 2d/6) #25728). Its own diff dropped every changed file outside ui/, though, so a lockfile bump, a root tsconfig.json or a packages/spec change selected only the always-on set (measured below). New: changedOutsideTree() lists the monorepo's own changes (git at the workspace root, --relative again, the ui/ prefix dropped), and outsideTreeReasons() turns them into full-run reasons:
    • a workspace-root file that is not documentation, or anything under patches/ (the rule the selector already applied to objectui's own root, now applied to the root this tree installs from);
    • a file in a monorepo package that a ui package depends on (any field), or that such a package depends on at run time (dependencies / peerDependencies / optionalDependencies). On this tree that set is exactly the seven packages the ui_consumers filter names: spec, client, core, types, lint, formula, sdui-parser.
    • Everything else outside ui/ (other packages, docs, .changeset/, .github/, scripts/) reaches ui tests only through the always-on set, as before.
  3. ui/scripts/__tests__/select-affected-tests-wiring.test.ts now reads the root ci.yml (located through workspaceRoot()) and runs both steps there at the monorepo root. A git stand-in answers merge-base as HEAD and records the call, so the legs assert the step asks for refs/remotes/origin/main and HEAD. A new leg covers a failing merge-base: full run. The controls are unchanged.
  4. ui/scripts/__tests__/select-affected-tests.test.ts: the ci.yml wiring block now reads the root job (including fetch-depth: 0 and the 16-wide matrix). A new nested-layout fixture (ui/ under a parent workspace) covers each outside-tree rule in both directions. One case runs real git in a scratch repo to show that changedFiles and changedOutsideTree split one diff between them.

The turbo parity case (agrees with turbo on who depends on whom) was already green in the merged layout on main. It is unchanged and still green.

ui/.github/workflows/ci.yml (objectui's copy, which runs nowhere) is left as it is; the governance-workflow audit owns it.

Measured

Selector verdicts on this tree (c475b45b + this diff), from scratch commits in a detached worktree:

change verdict selected test files
nothing under ui/ (this PR's own shape: workflow + ui/scripts) affected 686 of 5810
one leaf, ui/packages/plugin-gantt/src/index.ts affected 1036 of 5810
ui/packages/plugin-kanban/src/KanbanColumnSummary.ts (reaches app-shell) affected 2647 of 5810
pnpm-lock.yaml + packages/spec/src/index.ts, before this diff affected 686 of 5810 (the hole)
the same, after full —

Weighted by ui/scripts/test-shard-timings.json (the sequencer's per-file dataset): always-on alone is 8.8% of the summed file time, plugin-gantt 20.2%, plugin-kanban 54.2%, ui/packages/fields 88.9%. Each shard still pays its fixed cost (checkout, install, building the seven packages). The before/after shard wall times are read from this PR's own Test UI run, which is the always-on shape, and posted on the card.

Verification

  • pnpm exec vitest run scripts/__tests__/select-affected-tests.test.ts scripts/__tests__/select-affected-tests-wiring.test.ts (in ui/): 2 files, 49 tests, all pass.
  • Whole ui/scripts/__tests__ suite (pnpm exec vitest run scripts/__tests__ in ui/, head bc79c8aa): 190 files, 5608 passed, 2 skipped, 1 failed. The failure is check-pre-install-import-graph.test.ts > "finds no pre-install gate reaching a package" (unresolved: scripts/objectstack-pin.mjs, which the cutover deleted and ui/.github/workflows/* still name). It fails the same way on c475b45b with this diff's four files restored (37 tests, 1 failed), so it is red on main already and not this PR's. It is reported to the dispatcher. That file is in the always-on set, so it keeps running on every affected Test UI run.
  • node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran: 63 derived, 59 run with exit 0, 4 NOT MEASURED (exit 3, built dists absent): check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure, check:sourcemap-no-sources-content. These are Build Core's guards, and Build Core runs them on this PR (ci.yml is in the core filter).

Acceptance notes


Generated by Claude Code

…up affects

Wire ui/scripts/select-affected-tests.mjs into the root test-ui job and
teach the selector the changes outside ui/ that every ui test runs
through (a non-documentation workspace-root file, patches/, one of the
monorepo packages the ui tests read built).

Claude-Session: https://claude.ai/code/session_015oJuSyTy7Y3cm91KJMEW2m
Co-Authored-By: Claude <noreply@anthropic.com>
… nested-layout selector cases

Claude-Session: https://claude.ai/code/session_015oJuSyTy7Y3cm91KJMEW2m
Co-Authored-By: Claude <noreply@anthropic.com>
…st-ui job through helpers/root-ci

Claude-Session: https://claude.ai/code/session_015oJuSyTy7Y3cm91KJMEW2m
Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l and removed size/m labels Oct 26, 2026
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

CI on bc79c8aa: everything green except Test UI (14/16), which is not this PR's.

  • The failing test is ui/scripts/__tests__/check-pre-install-import-graph.test.ts > "finds no pre-install gate reaching a package": unresolved: scripts/objectstack-pin.mjs. The cutover deleted that script, and ui/.github/workflows/* still name it. This PR changes none of those inputs.
  • It reproduces locally on c475b45b (this branch's base) with this PR's four files restored: 37 tests, 1 failed. So it is already red on main. That file is outside every ui package, so the selector puts it in the always-on set, and it runs (and fails) on every affected run too.
  • No fix exists yet. It is reported to the dispatcher as an out-of-scope finding. Test UI is not a required context.

Reading from this run, the always-on shape (no ui package changed): the 16 Test UI shards took 2.1–3.7 min each, 46 runner-min in total. Before this PR, the same jobs ran the full suite: 6.6–12.0 min per shard and 184 runner-min in run 40251065567 (a ci.yml-only change), and 8.5–14.3 min per shard in run 40248884869.

Head is now d775186d (a merge of main after #25879; the selector's ci.yml pins read the root test-ui job through helpers/root-ci.ts).


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 26, 2026 03:13
@os-zhuang
os-zhuang added this pull request to the merge queue Oct 26, 2026
@os-tesla
os-tesla requested a review from os-zhuang October 26, 2026 03:16
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 40263268635 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

分类:failure —— 按下面的日志分诊。

失败的 job(日志抽取,best effort):

  • Test UI (13/16) — 失败步骤: Run the ui tests (shard 13/16)

    × finds no pre-install gate reaching a package 9ms
    FAIL   unit  scripts/__tests__/check-pre-install-import-graph.test.ts > the real tree — every step objectui#6148 measured is still derived > finds no pre-install gate reaching a package
      ↳ 失败原因: AssertionError: a script this repository runs before `pnpm install` needs `node_modules` to load: expected [ Array(1) ] to deeply equal []
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 9 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

3 participants