Repository navigation
Description
Parent
Validated infrastructure gap
At a794557384adcc3293a77043f6b9aff1c33d152c, package scripts and CI provide correctness/build checks but no dedicated repeatable performance baseline runner. MCP has in-process Vega-Lite, ECharts, and Chart.js renderers, so cold initialization, compilation, rendering, and artifact conversion need separate measurements rather than compiler-only timing.
A local npm run build:js on Node v26.12.1 failed with ERR_WORKER_OUT_OF_MEMORY during the tsup/DTS build under an approximately 2144 MiB V8 heap limit. This is diagnostic evidence only: CI uses Node 20/22, the host is shared, and there is no hosted-CI reproduction. The site CI job already explicitly allows a larger heap. Do not prescribe a memory-limit increase or claim a supported-runtime build defect without additional evidence.
Upstream issue listings and all 21 open PR titles were checked on 2026-11-11; no equivalent benchmark-infrastructure work was found.
Proposed scope
Add a small opt-in, reproducible benchmark runner using existing dependencies and deterministic fixtures. Start with compiler and supported in-process MCP render paths, cold and warm runs, and build command/environment reporting. Keep hosted-CI duration assessment separate and report unavailable evidence honestly.
Acceptance criteria
- Record source revision, Node/platform identities, fixture shape/size, backend, output format/dimensions, cold vs warm state, individual samples, and summary timing plus memory metrics with precisely defined meanings.
- Validate artifact output so errors/empty charts cannot appear as fast successful runs; preserve failed attempts and report skipped/unsupported cases explicitly.
- Support at least three repeats; serialize benchmark runs on the shared host. Do not add a noisy performance gate to ordinary CI or change default npm build/test behavior.
- Distinguish process RSS/heap snapshots from true allocation totals and instrumented timings from ordinary timings.
- Document reproduction and limitations. Use existing tooling; avoid a large new benchmark framework.
- Any later build or CI optimization must have its own material measured evidence and preserve coverage, packaging, and supported Node compatibility.
- One focused instrumentation PR, no automatic merge. This issue is not itself a claim of runtime or CI speedup.
Activity
Chenglong-MS commentedon Nov 15, 2026
Done in #171 (merged into
dev):npm run benchmark:renderrecords cold and warm compiler and MCP render baselines; seedocs/render-performance.md. Ships in 1.0.2.