Skip to content

feat(webkit): roll to r2366 - #42879

Open
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2366
Open

microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2366

Conversation

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

Browser build: https://github.com/microsoft/playwright-browsers/commit/cb092a970a8382d1f393b9abfa9521ec70a9e61f. Build has full platform coverage - check that all tests pass there.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

5 flaky ⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/tracing.spec.ts:533 › should produce screencast frames fit `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`

52075 passed, 1241 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

6 failed
❌ [chrome] › mcp/cli-webmcp.spec.ts:162 › webmcp-call disambiguates duplicate tool names by frame @mcp-macos-latest-chrome
❌ [firefox] › mcp/annotate.spec.ts:291 › should enter annotate mode on fresh dashboard.tsx mount with -s --annotate @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-devtools.spec.ts:140 › request* and response* commands support --filename @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-devtools.spec.ts:201 › recording-start-stop @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-mouse.spec.ts:38 › mousewheel @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-parsing.spec.ts:88 › should preserve leading zeros in string arguments @mcp-windows-latest-firefox

8699 passed, 1474 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Nothing here points at this PR, but one failure I can't fully clear

Hi, I'm the Playwright bot and I took a look at the red CI.

This PR is a WebKit binary roll — browsers.json and webkit/protocol.d.ts, nothing else. All 6 failures are on chrome and firefox, and WebKit itself is green. Five of the six are tests with a documented history of flaking on unrelated SHAs. The sixth has no prior failure on record, so I'm leaving it uncertain rather than calling it a flake.

Details

Main suite (tests 1) is clean: 52075 passed, 5 flaky, 0 failed. Everything below is from the MCP report, run 35847214605. Five of the six failures land on the single mcp-windows-latest-firefox shard, which carries at least one failure in 319 of 702 recorded runs and three or more in 49 of them — a multi-failure run on that bot is routine, not a signal.

This PR's run isn't in the aggregated results DB yet, so every run cited below is a different SHA.

Pre-existing flake / infra

Uncertain

  • [firefox] › mcp/cli-devtools.spec.ts:140 › request* and response* commands support --filename — this one has passed 724 of 724 recorded runs on firefox and has never failed on any project, so I can't call it a flake on the evidence. What makes me doubt it's real: a WebKit binary revision has no path to a Firefox MCP test, and it failed in the same shard as four proven flakes above. To settle it, a rerun of the mcp-windows-latest-firefox job — if it goes green there, that's a first data point for the flake column.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 2"

2 fatal errors, not part of any test
3 failed
❌ [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) @chrome-macos-latest
❌ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe @firefox-macos-15-xlarge
❌ [android-page] › page/page-set-content.spec.ts:57 › should include shadow roots

25 flaky ⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-windows-latest`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:179 › serviceWorker(), and fromServiceWorker() work `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chromium-macos-14-xlarge`
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:179 › should take screenshot `@chromium-macos-14-xlarge`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-macos-14-xlarge`
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:179 › should take screenshot `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:202 › cli codegen › should upload a file via hidden input triggered by button `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/selector-generator.spec.ts:457 › selector generator › should match in shadow dom `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:703 › selector generator › should generate noText: description from external aria-describedby `@chrome-macos-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@tracing-firefox`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:43 › should poll on interval `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:89 › should support viewport option `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-windows-latest`
⚠️ [firefox-library] › library/signals.spec.ts:25 › should close the browser when the node process closes `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1714 › should highlight locator in iframe while typing `@firefox-macos-15-xlarge`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:2006 › canvas disabled title `@webkit-macos-15-xlarge`
⚠️ [webkit-library] › library/tracing.spec.ts:533 › should produce screencast frames scale `@webkit-macos-15-xlarge`
⚠️ [webkit-library] › library/screencast.spec.ts:57 › applies backpressure while async onFrame callback is pending `@webkit-macos-15-large`

112067 passed, 4744 skipped, 29 did not run


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟢 None of these failures come from this PR

Hi, I'm the Playwright bot and I took a look at the red CI.

This roll changes the WebKit revision 23652366 and a doc comment in webkit/protocol.d.ts — nothing else. The three failures in the latest report are on Chromium, Firefox and Android, and WebKit itself is green. All three have a failure record on SHAs this PR can't be responsible for; the Android one is currently broken on main, including this PR's own base commit.

Details

Latest report is tests 2, run 35853563796 — 112067 passed, 3 failed, 25 flaky. tests 1 is clean (0 failed) and MCP was triaged separately.

Pre-existing flake / infra

  • [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) @chrome-macos-latest — fails with Test timeout of 30000ms exceeded. Error: request.response: Worker closed, 3 of 341 runs on this bot, green on the other 17 Chromium bots (~6800 runs, zero failures). Same test, same bot, same error on push bcb3563a and on feat(webkit): roll to r2358 #42438 (run 33129340741). It also flaked-and-recovered three more times inside this very report (chromium-macos-15-large, chrome-beta-ubuntu-22.04).

  • [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe @firefox-macos-15-xlarge — the most established flake of the three: 14 failures in 311 runs on this bot, zero on the other four Firefox bots. Recent unrelated failures include pushes 2a095f6f, bf9d04f2, 500c9c82, d78a9e71 and PR feat(firefox): roll to r1548 #42747 (run 35105295536) — always the same expect(locator).toHaveText on the iframe. Its sibling at :257 flaked on four separate bots in this report too.

  • [android-page] › page/page-set-content.spec.ts:57 › should include shadow roots — not a flake, a real pre-existing breakage: page.content: TypeError: root.getHTML is not a function. It has failed 13 of 13 recorded runs on the android bot, all of them pushes to main since 2026-09-22, and never passed in the retained window. That includes push e125b2ff — which is exactly this PR's base commit, failing identically without the roll. The Android bot's WebView is too old for Element.getHTML(), so this needs a fix of its own; it's unrelated to a WebKit revision bump.

The 2 fatal errors and 29 did-not-run entries sit alongside the aborted Android shard, so I'd read them as the same story rather than a fourth one.

Triaged by the Playwright bot

Triaged by the Playwright bot - agent run

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants