Skip to content

chore: refresh benchmarks for v1.25.1#962

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Jun 27, 2026
Merged

chore: refresh benchmarks for v1.25.1#962
jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Jun 27, 2026

Copy link
Copy Markdown
Owner

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.25.0; the workspace is now 1.25.1. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.25.0 -> 1.25.1
  • deno: 2.8.3 -> 2.9.0

Public ratios: warm installs vs Bun 1.3x -> 4x; warm installs vs pnpm 3x -> 6x; repeat test vs Bun 6x -> 5x; repeat test vs pnpm 39x -> 35x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 1147ms -> 437ms (-62%) 1508ms -> 1729ms (+15%) 1377ms -> 2004ms (+46%) 2900ms -> 2461ms (-15%) 7456ms -> 7198ms (-3%) 9547ms -> 9533ms (0%)
Fresh install (cold cache) 15253ms -> 14371ms (-6%) 4915ms -> 5883ms (+20%) 7751ms -> 8073ms (+4%) 9854ms -> 10681ms (+8%) 9072ms -> 8789ms (-3%) 12189ms -> 12442ms (+2%)
npm install && npm run test 9ms -> 10ms (+11%) 53ms -> 52ms (-2%) 77ms -> 63ms (-18%) 355ms -> 349ms (-2%) 689ms -> 715ms (+4%) 1549ms -> 1566ms (+1%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark artifacts only; no application or install logic changes.

Overview
Re-runs the hermetic benchmark suite (mise run bench:bump) and pins benchmarks/results.json to aube 1.25.1 (was 1.25.0) and deno 2.9.0 (was 2.8.3), with a new updated timestamp and refreshed timings/stats for warm install, cold install, and install && test.

The README BENCH_RATIOS paragraph is regenerated from those results, so public speed claims shift—for example warm installs are now described as ~6× vs pnpm and ~4× vs Bun (previously ~3× and ~1.3×), and repeat-test ratios move to up to 35× vs pnpm and up to 5× vs Bun. The standout data change is aube’s warm fresh install dropping 1147ms → 437ms in the published table.

Reviewed by Cursor Bugbot for commit 7c23b4e. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated benchmark claims in the README to reflect the latest fast-install and repeat-command speed comparisons.
  • Chores

    • Refreshed benchmark metadata and recorded tool version updates.
    • Replaced stored benchmark results with new measurements across install and warm/cold test scenarios.

@coderabbitai

coderabbitai Bot commented Jun 27, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 4ae0afe1-4349-42dd-bfd5-926e70a1ba5f

📥 Commits

Reviewing files that changed from the base of the PR and between 1c2520f and 7c23b4e.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

📝 Walkthrough

Walkthrough

The PR refreshes benchmark data and metadata in benchmarks/results.json and updates the README’s Fast installs benchmark ratios to match the new figures.

Changes

Benchmark refresh

Layer / File(s) Summary
Results metadata and rows
benchmarks/results.json
Updated the results timestamp, tool versions, and benchmark values/stats for gvs-warm, gvs-cold, and install-test.
README benchmark ratios
README.md
Revised the Fast installs benchmark ratios for warm installs and repeat test commands.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

  • jdx/aube#907: Updates the same benchmark-facing artifacts, including README.md Fast installs claims and benchmarks/results.json rows, versions, and timestamps.
  • jdx/aube#922: Also refreshes the same benchmark rows in benchmarks/results.json and the matching README benchmark figures.
  • jdx/aube#947: Overlaps on the same benchmark refresh pattern: README ratio text plus regenerated benchmark metadata and results.

Poem

🐰 I hopped through numbers, quick and spry,
With warm installs racing by.
The README hummed, the stats did sing,
And benchmark carrots caught a spring.
Hop hop — the table’s freshly bright!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: refreshing benchmark data for v1.25.1.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR refreshes the published benchmark artifacts for v1.25.1. The main changes are:

  • Updated the README benchmark-ratio block.
  • Updated benchmark metadata and tool versions.
  • Replaced the stored benchmark timings and statistics.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
README.md Regenerated the benchmark-ratio text to match the refreshed public benchmark numbers.
benchmarks/results.json Refreshed benchmark timestamp, tool versions, measured values, and stats without changing the artifact shape.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@jdx jdx merged commit 834b315 into main Jun 27, 2026
18 checks passed
@jdx jdx deleted the bench-refresh branch June 27, 2026 12:50
@cursor cursor Bot mentioned this pull request Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant