You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implements wp-tooling perf — a zero-runtime-dep CLI runner that collects Core Web Vitals (web-vitals + Lighthouse) and optional server-side xhprof profiling per URL, with sensible degradation for every failure mode
Adds setup/perf scaffold so consumers can bootstrap config, npm scripts, and the PHP xhprof shim in one command
Normalises xhprof output (ct/wt/cpu/mu/pmu) into the report's server section alongside frontend metrics, with a fidelity note explaining CLI-context limitations
Perf runner: per-URL collector that runs web-vitals under headless Chromium, Lighthouse, and optional xhprof server profiling — each layer degrades independently on failure. Config resolution with built-in DEFAULTS and section-level merge, supporting --url override and optional config files. Binary/module resolution walks consumer node_modules with npx --no-install fallback, never fetching from registry. Normalisation pipeline rates metrics against CWV thresholds, extracts Lighthouse scores + failing audits, builds human-readable assessment lines from a single THRESHOLDS registry. CLI layer with dry-run plan printer, text/json emitter, exit codes 0/1/2/3 matching the a11y convention, and a RunnerError hierarchy with machine-readable codes.
Setup/perf scaffold: scaffold manifest with six configurable inputs (base URL, page paths, server enable toggle, WP-CLI env-cwd path). Minimal .perfrc.json template emitting only URLs and server settings — all other sections fall through to built-in defaults, preventing silent drift. Server-enabled boolean separated from env-cwd path so profiling the WordPress root (--env-cwd=.) is expressible. Hardened PHP shim that detects xhprof/tideways backends, degrades gracefully to [] when the profiler is absent, unhooks redirect_canonical to prevent early exit(), installs a shutdown-fallback emitter with buffer drain, populates $_GET and REQUEST_URI from the positional path argument, and writes a route diagnostic to stderr.
xhprof normalisation: WP-CLI invocation layer that spawns the shim with correct argument order, parses JSON output with preamble tolerance, and captures the stderr diagnostic — every failure degrades to { data: null, error } instead of throwing. URL splitter separating origin from path+query for the shim's two-argument contract. Normaliser maps raw xhprof function data into the report's server section with ct/wt/cpu/mu/pmu fields and a fidelity note explaining CLI-context limitations. Server layer wired into the per-URL collector independently of the browser layer — still profiles even when the frontend scan failed.
How I verified
npm run check
Fixture testing for all exit codes and degradation paths. Scaffold render tests confirm default and custom config output. Also tested end-to-end by running wp-tooling perf against the features skeleton plugin.
Acceptance criteria
Runtime behavior
wp-tooling perf --dry-run resolves and prints the plan without executing anything.
A JSON report for two URLs validates the shape above; exit codes match a11y (0/1/2/3).
The engine core has no TTY-UI dependency (runs headless/CI/AI).
wp-tooling add setup/perf yields a runnable test:perf on a consumer, with the dev deps surfaced as developer actions (not auto-installed).
wp eval-file server-profile.php --url=... --top=15 --format=json returns top-N JSON when the extension is present, and [] when it is not.
No file overwrite, no package-manager execution, no secret values.
A perf report from a consumer run carries both the frontend and the server layer per URL.
The fidelity note is present in the report.
Fixture tests for the normalizer are green (frontend-only, server-only, and combined).
Code quality
Lint — zero errors
Static analysis — zero errors
Tests pass
Housekeeping
CHANGELOG.md entry under ## Unreleased
.claude/issues/<N>-<slug>.md updated with final state
Full decision log and verification history for this work lives in .claude/issues/wp-devtools-22-36-37-perf-runner.md on this branch.
…ver xhprof)
Mirrors the a11y runner: two-layer normalized report (lab Core Web
Vitals + Lighthouse, plus optional server-side xhprof hotspots over
WP-CLI). Ships src/perf/*, the perf CLI command, package.json exports,
tests, the setup/perf scaffold with a hardened server-profile.php shim,
and CHANGELOG/issue-tracking entries. Includes two pre-existing
lint-gate fixes (no-shadow + prettier in src/init/index.js and
tests/ui/selects.test.js) needed for npm run check to pass, matching
the fix already on the a11y branch.
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 35 out of 35 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
node-packages/wp-tooling/src/perf/config.js:39
Using npx wp-env without --no-install allows npm to download @wordpress/env when it is absent locally, violating the runner's no-fetch behavior and potentially prompting or modifying the npm cache in CI. Make the fallback explicitly local-only.
The generated config likewise invokes npx wp-env without --no-install, so running the optional server layer can fetch @wordpress/env instead of degrading when it is not installed. Include the no-install flag here and update the scaffold expectations accordingly.
The default text report drops both server.note and server.diagnostic. As a result, users do not see the required CLI-context fidelity warning or the captured route diagnostic unless they choose JSON output; emit both fields when present.
…, server resilience
- read attribution.target (not element) for web-vitals v5; use ?? for settleMs/timeoutMs
- skip malformed xhprof entries instead of aborting the entire perf run
- unwind output buffers to pre-render level in server-profile.php
- tryParse iterates [/{ candidates past preambles; non-zero exit checked before parsing
- dry-run uses resolveBin so lighthouse --version is never spawned
- pre-render script templates before file writes; validate checks scripts alongside files
- emit server fidelity note in default text output
A configured topAudits: 0 is replaced with 5 because 0 is falsy, so consumers cannot request zero audit details while retaining Lighthouse scores. Use nullish defaulting so the documented maximum of zero is preserved.
const topAudits = options.topAudits || 5;
node-packages/wp-tooling/src/perf/run.js:286
failedUrls also includes a vitalsError for pages that loaded successfully but returned no metrics, so this summary can incorrectly say those URLs “failed to load.” Use wording that covers both navigation and metric-collection failures.
This issue also appears on line 495 of the same file.
This diagnostic is also emitted for vitalsError, where the page did load (and Lighthouse may have run). Saying every failed URL “failed to load” obscures the actual metric-harvest failure reported in the per-URL notes.
if (report.summary.failedUrls > 0) {
process.stderr.write(
`perf: ${report.summary.failedUrls} URL(s) failed to load — treating as a run failure.\n`
);
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Server execution can fetch packages, and browser-layer error handling can incorrectly suppress Lighthouse.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (8)
Previously missed (6) — in code that hasn't changed since the last review.
node-packages/wp-tooling/src/perf/config.js:39
The default server command invokes bare npx, so enabling server without a locally installed wp-env can download and execute it from the registry. That contradicts this PR's never-fetch/no-package-manager-execution contract. Use the same no-install resolution policy as Lighthouse, and keep the scaffold template in sync. node-packages/wp-tooling/src/perf/normalize.js:101
An explicit topAudits: 0 is replaced by the default because 0 is falsy, so consumers cannot request zero audit details. Use nullish defaulting so the documented maximum is respected at this boundary. node-packages/wp-tooling/src/perf/run.js:183
scanError covers every exception from collectVitals, including injection, evaluation, and page-close failures after navigation succeeded. This guard then skips Lighthouse as if the URL were unreachable, so a collector-specific failure disables the supposedly independent Lighthouse layer. Distinguish navigation failures from collection failures, or run Lighthouse after non-navigation errors. node-packages/wp-tooling/package.json:30
Adding this public export leaves the package guide stale: node-packages/wp-tooling/AGENTS.md:61-72 still says the exports map has eight entries and omits ./perf; its CLI directory listing also omits the new command. Update that authoritative package documentation so future changes do not treat this export as unsupported. node-packages/wp-tooling/src/perf/resolve-module.js:96
This variable-path require violates the package rule forbidding dynamic require paths (node-packages/wp-tooling/AGENTS.md:104). Replace the generic loader with a constrained consumer-module mechanism, such as a dedicated Puppeteer loader using module.createRequire with a statically named package. node-packages/wp-tooling/src/perf/run.js:286
failedUrls also includes vitalsError results where navigation succeeded but no metrics were harvested, so reporting all of them as “failed to load” is inaccurate. Use wording such as “failed to collect metrics” that covers both load and harvest failures.
This issue also appears on line 495 of the same file.
On Windows, npm's extensionless .bin/<name> file is a POSIX shim; execFileSync cannot execute it. Because this path is preferred whenever it exists, locally installed Lighthouse is detected but its version probe fails. Resolve the package's JavaScript bin entry and invoke it with process.execPath, rather than selecting an extensionless or shell-based shim.
const candidate = path.join(dir, 'node_modules', '.bin', binName);
if (fs.existsSync(candidate)) {
node-packages/wp-tooling/src/perf/run.js:498
This message says every failed URL failed to load, but failedUrls is also incremented for an empty web-vitals harvest after a successful load. Report a generic collection failure, or track load and harvest failures separately.
if (report.summary.failedUrls > 0) {
process.stderr.write(
`perf: ${report.summary.failedUrls} URL(s) failed to load — treating as a run failure.\n`
);
A configured topAudits: 0 is silently replaced with 5, so consumers cannot request an empty audit list even though this option is a maximum count. Preserve explicit zero with a nullish fallback.
A configured topAudits: 0 is replaced with the default 5, so consumers cannot suppress failing-audit details while retaining Lighthouse scores. Since this option is documented as a maximum, preserve zero with nullish defaulting.
Report Lighthouse probe failures with EBINFAIL
node-packages/wp-tooling/src/perf/run.js:102
detectBin() distinguishes an installed binary that failed its --version probe from a missing one, but this discards that distinction and always reports EBINMISSING/“not found.” A corrupt or incompatible local Lighthouse install therefore gets the wrong message and exit code; mirror the a11y runner's binUnavailableError classification so local/hoisted probe failures become EBINFAIL with lighthouseBin.error.
When a scaffold reports more than one npm script (as setup/perf now does), these lines are emitted without commas, so the advertised package.json snippet is invalid JSON. Add commas between entries; the current test only covers the single-entry case and misses this generated output.
This issue also appears on line 199 of the same file.
Document Mustache interpolation in scaffold scripts
This changes scripts from verbatim passthrough to Mustache-rendered content, but the schema contract still says “The engine treats this as a verbatim passthrough” at src/scaffolds/schema.js:194-198. Update the scaffold-author documentation/schema comment to describe interpolation and undefined-placeholder failures so authors do not rely on the old contract.
Config fields need type/value validation before being merged. For example, { "server": { "enabled": "false" } } leaves a truthy string here, so the runner unexpectedly executes WP-CLI; similarly, a string lighthouse.categories later throws at .join() and is reported as a degraded layer instead of an invalid config. Reject invalid section fields with a configuration error so user typos cannot invert behavior or produce a false clean run.
Test independent Lighthouse handling after navigation timeouts
This assertion codifies the coupling that prevents Lighthouse from degrading independently. A Puppeteer networkidle2 timeout is also reported as ENAVFAIL, even though Lighthouse may still produce a valid result; update this test to verify that Lighthouse is attempted and independently recorded or degraded.
This test locks in the same platform-specific .bin shim behavior as the resolver. Update the fixture to declare a package bin entry and assert that resolveBin returns process.execPath plus that entry path, matching the established a11y resolver's cross-platform contract.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
wp-tooling perf— a zero-runtime-dep CLI runner that collects Core Web Vitals (web-vitals + Lighthouse) and optional server-side xhprof profiling per URL, with sensible degradation for every failure modesetup/perfscaffold so consumers can bootstrap config, npm scripts, and the PHP xhprof shim in one commandCloses
Closes https://github.com/rtcamp/wp-devtools/issues/22
Closes https://github.com/rtcamp/wp-devtools/issues/36
Closes https://github.com/rtcamp/wp-devtools/issues/37
Changes
--urloverride and optional config files. Binary/module resolution walks consumer node_modules withnpx --no-installfallback, never fetching from registry. Normalisation pipeline rates metrics against CWV thresholds, extracts Lighthouse scores + failing audits, builds human-readable assessment lines from a single THRESHOLDS registry. CLI layer with dry-run plan printer, text/json emitter, exit codes 0/1/2/3 matching the a11y convention, and a RunnerError hierarchy with machine-readable codes..perfrc.jsontemplate emitting only URLs and server settings — all other sections fall through to built-in defaults, preventing silent drift. Server-enabled boolean separated from env-cwd path so profiling the WordPress root (--env-cwd=.) is expressible. Hardened PHP shim that detects xhprof/tideways backends, degrades gracefully to[]when the profiler is absent, unhooksredirect_canonicalto prevent earlyexit(), installs a shutdown-fallback emitter with buffer drain, populates$_GETandREQUEST_URIfrom the positional path argument, and writes a route diagnostic to stderr.{ data: null, error }instead of throwing. URL splitter separating origin from path+query for the shim's two-argument contract. Normaliser maps raw xhprof function data into the report's server section with ct/wt/cpu/mu/pmu fields and a fidelity note explaining CLI-context limitations. Server layer wired into the per-URL collector independently of the browser layer — still profiles even when the frontend scan failed.How I verified
Fixture testing for all exit codes and degradation paths. Scaffold render tests confirm default and custom config output. Also tested end-to-end by running wp-tooling perf against the features skeleton plugin.
Acceptance criteria
Runtime behavior
wp-tooling perf --dry-runresolves and prints the plan without executing anything.wp-tooling add setup/perfyields a runnabletest:perfon a consumer, with the dev deps surfaced as developer actions (not auto-installed).wp eval-file server-profile.php --url=... --top=15 --format=jsonreturns top-N JSON when the extension is present, and[]when it is not.serverlayer per URL.fidelitynote is present in the report.Code quality
Housekeeping
CHANGELOG.mdentry under## Unreleased.claude/issues/<N>-<slug>.mdupdated with final stateFull decision log and verification history for this work lives in
.claude/issues/wp-devtools-22-36-37-perf-runner.mdon this branch.