Skip to content

ci(journeys): publish every browser's journeys in one report - #193

Merged
ixuz merged 2 commits into
mainfrom
merge-journeys-reports
Oct 19, 2026
Merged

ixuz merged 2 commits into
mainfrom
merge-journeys-reports

Conversation

@ixuz

@ixuz ixuz commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What and why

The report at /journeys/ held only the Chromium job's journeys. It said nothing of Firefox, WebKit or the phone, which are where the login's redirects, a guest's storage and the restore of a session differ. The engines' report existed only as a CI artifact, gone after a week.

What changes:

  • In CI, each journeys job also writes its run as a Playwright blob, report-chromium.zip or report-engines.zip (playwright.config.ts, PLAYWRIGHT_BLOB_OUTPUT_NAME). On main it uploads the blob as journeys-blob-<chromium|engines>, kept 7 days.
  • A new journeys-report job runs on main after both journeys jobs, even when one failed. It merges their blobs with npx playwright merge-reports --reporter html and uploads the result as journeys-report, which deploy.yml already publishes at /journeys/. deploy.yml doesn't change.
  • When both journeys jobs passed, journeys-report fails without both blobs. A run the deploy may publish then never has a report that's missing a browser.
  • The blobs are kept as long as the report, 7 days. Re-running one failed job a few days later, as releasing.md suggests for a flaky release run, still finds the other job's blob.
  • ci-ok doesn't need journeys-report, since the job runs on main alone and would show as skipped on pull requests. On main, its failure still fails the run, so it holds back the release and the deploy.
  • Each journeys job keeps its own HTML report, summary and artifact for debugging a failure, as before.
  • docs/testing.md has a row for the new job in the CI jobs table, and says the online report holds every browser's journeys. docs/deployment.md says the same.

The merge job runs only on main, so this pull request's CI doesn't exercise it. The first real run is the push after the merge, and the full report appears at /journeys/ with the next release's deploy. I checked locally that two blobs from separate runs merge into one report with all four projects. I also tested the merge step's shell logic for each case: both jobs passed, one failed, blobs missing.

A second commit fixes a race in the render-error journey that failed this branch's first CI run: "Go to the start" waited for the address to end in #/, which the app replaces at once with the deck list, so the check passed only when a poll caught that moment. The page object now waits for the error screen to go, and the journey's next step checks the deck list (SomethingWentWrong.ts).

Closes: #192

Checklist

  • One issue. This pull request addresses one issue, linked above.
  • One step. This pull request does not finish its issue, so it says Refs:, not Closes:. Leave this unchecked otherwise.
  • Done, as AGENTS.md defines it: npm run check passes (lint, Markdown and spelling included), npm run lint:reuse too if a file was added, and npm run crosscheck too if ns/ or decks/ changed. (lint:reuse n/a: no file added; crosscheck n/a: ns/ and decks/ unchanged. npm run lint:workflows passes too.)
  • Tests. New and changed code has its tests, and every package stays at 100% coverage (docs/testing.md). (n/a: only CI configuration and the journeys' reporter list change, no package code)
  • Docs. The docs say what changed, and a new page in docs/ is listed in docs/docs.ttl.
  • Languages. A new or changed message is in every locale: en, sv, ko, de, es and fr (packages/ui/src/i18n/). (n/a: no messages changed)
  • Vocabulary and shapes. Any change to ns/ was agreed in an issue first; no published ns/shapes/*/vN.ttl or decks/*/vN.ttl was edited; generated files were regenerated, not edited. (n/a: ns/ and decks/ unchanged)
  • Breaking? Nothing in the public API breaks, or a commit says so (! and a BREAKING CHANGE: footer). npm run check:breaking sees a line of api/ that goes, but not a feature removed, nor a reader that reads less than before.
  • Pod data. Any change to what the app writes to a pod comes with its migration (docs/migrations.md). (n/a: the app is unchanged)
  • Policy. Any change to an external rule the code follows (SM-2, the plural rules, a specification) is cited. (n/a: no such rule changes)
  • Licenses. New content is MIT, or a deck states its own license; a new file has its SPDX header (npm run lint:reuse), or a new deck its line in REUSE.toml; third-party content is credited. (n/a: no new file or content)
  • Architecture. Imports follow docs/boundaries.md and the layers of docs/architecture.md. (n/a: no imports changed)
  • Run. I ran the app (npm run dev) and tried the change, and considered the user journeys. (n/a for the app, which is unchanged; the journeys run as before, with one more reporter in CI)
  • Authorship. I read and understand every line of this pull request, and answer for it, whatever tools helped write it (CONTRIBUTING.md).
  • Commits. Each commit is one logical change, in Conventional Commits form (type(scope): subject, ! and a BREAKING CHANGE: footer if it breaks something), with an imperative subject and a body that explains why (CONTRIBUTING.md).

The report at /journeys/ held only the Chromium job's journeys, so it
said nothing of Firefox, WebKit or the phone, where the login's
redirects, a guest's storage and the session's restore differ. Their
report was a CI artifact alone, gone after a week.

Each journeys job now also writes its run as a blob, and on main a
journeys-report job merges both with `playwright merge-reports` into
the journeys-report artifact the deploy already publishes. The blobs
are kept as long as the report, so re-running one failed job a few
days later still finds the other's; when both jobs passed, the merge
fails without both blobs, so a run the deploy may publish never has a
report missing a browser.

Closes: #192
@ixuz ixuz added enhancement New feature or request ci/cd Countinuous integration and deployment labels Oct 19, 2026
"Go to the start" sets the address to #/, which the app at once
replaces with the deck list of the one instance it knows. Waiting for
the URL to end in #/ passed only when a poll caught that moment: on
this branch's CI it came too late, and the journey failed though the
app was right. The page object now waits for the crash screen to go,
and the journey's next step checks the deck list it leads to.

Refs: #147, #192
@ixuz
ixuz added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit bd7e390 Oct 19, 2026
24 checks passed
@ixuz
ixuz deleted the merge-journeys-reports branch October 19, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd Countinuous integration and deployment enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The journeys' report at /journeys/ shows only Chromium, not Firefox, WebKit or the phone

1 participant