Repository navigation
ci(journeys): publish every browser's journeys in one report - #193
Merged
Merged
Conversation
The report at /journeys/ held only the Chromium job's journeys, so it said nothing of Firefox, WebKit or the phone, where the login's redirects, a guest's storage and the session's restore differ. Their report was a CI artifact alone, gone after a week. Each journeys job now also writes its run as a blob, and on main a journeys-report job merges both with `playwright merge-reports` into the journeys-report artifact the deploy already publishes. The blobs are kept as long as the report, so re-running one failed job a few days later still finds the other's; when both jobs passed, the merge fails without both blobs, so a run the deploy may publish never has a report missing a browser. Closes: #192
"Go to the start" sets the address to #/, which the app at once replaces with the deck list of the one instance it knows. Waiting for the URL to end in #/ passed only when a poll caught that moment: on this branch's CI it came too late, and the journey failed though the app was right. The page object now waits for the crash screen to go, and the journey's next step checks the deck list it leads to. Refs: #147, #192
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The report at /journeys/ held only the Chromium job's journeys. It said nothing of Firefox, WebKit or the phone, which are where the login's redirects, a guest's storage and the restore of a session differ. The engines' report existed only as a CI artifact, gone after a week.
What changes:
report-chromium.ziporreport-engines.zip(playwright.config.ts,PLAYWRIGHT_BLOB_OUTPUT_NAME). Onmainit uploads the blob asjourneys-blob-<chromium|engines>, kept 7 days.journeys-reportjob runs onmainafter both journeys jobs, even when one failed. It merges their blobs withnpx playwright merge-reports --reporter htmland uploads the result asjourneys-report, whichdeploy.ymlalready publishes at/journeys/.deploy.ymldoesn't change.journeys-reportfails without both blobs. A run the deploy may publish then never has a report that's missing a browser.ci-okdoesn't needjourneys-report, since the job runs onmainalone and would show as skipped on pull requests. Onmain, its failure still fails the run, so it holds back the release and the deploy.The merge job runs only on
main, so this pull request's CI doesn't exercise it. The first real run is the push after the merge, and the full report appears at/journeys/with the next release's deploy. I checked locally that two blobs from separate runs merge into one report with all four projects. I also tested the merge step's shell logic for each case: both jobs passed, one failed, blobs missing.A second commit fixes a race in the
render-errorjourney that failed this branch's first CI run: "Go to the start" waited for the address to end in#/, which the app replaces at once with the deck list, so the check passed only when a poll caught that moment. The page object now waits for the error screen to go, and the journey's next step checks the deck list (SomethingWentWrong.ts).Closes: #192
Checklist
Refs:, notCloses:. Leave this unchecked otherwise.npm run checkpasses (lint, Markdown and spelling included),npm run lint:reusetoo if a file was added, andnpm run crosschecktoo ifns/ordecks/changed. (lint:reusen/a: no file added;crosscheckn/a:ns/anddecks/unchanged.npm run lint:workflowspasses too.)docs/is listed in docs/docs.ttl.packages/ui/src/i18n/). (n/a: no messages changed)ns/was agreed in an issue first; no publishedns/shapes/*/vN.ttlordecks/*/vN.ttlwas edited; generated files were regenerated, not edited. (n/a:ns/anddecks/unchanged)!and aBREAKING CHANGE:footer).npm run check:breakingsees a line ofapi/that goes, but not a feature removed, nor a reader that reads less than before.npm run lint:reuse), or a new deck its line inREUSE.toml; third-party content is credited. (n/a: no new file or content)npm run dev) and tried the change, and considered the user journeys. (n/a for the app, which is unchanged; the journeys run as before, with one more reporter in CI)type(scope): subject,!and aBREAKING CHANGE:footer if it breaks something), with an imperative subject and a body that explains why (CONTRIBUTING.md).