Repository navigation
Conversation
…ity-summary.csv Only the first input's quality survived because the key went through setdefault. Union quality across inputs like failureReasons/imageBuild, fill environment.quality from the first input that has it, and write quality-summary.csv when quality is present. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
P (run 38102505695): 125 of 126 cells ok. dumi nf50-cs5 has no result: its artifact upload failed on a ':' in a copied quality site path. P nf50-cs5 carries the spec 009 quality pass for 62 generators (61 ok, quarkdown error: no index.html at the site root). M (run 38102507250): 116 of 126 cells ok. At 1000 x 50 KB: docusaurus, dumi, statiq and vitepress abort at the heap cap (exit 134); marmite, nextra, quartz, rspress and vite-ssg hit the 8 GB container limit; quarto times out. Merged locally with merge_results from this branch, because the round's merge job ran master, which drops quality for all but one input. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GG (run 38102510926): 20 of 126 cells ok (18 at 100,000 x 5 KB, hugo and pelican at 100,000 x 50 KB). Besides heap and 8 GB limits, the failures include JVM generators at the default 25% heap, EMFILE at 65,536 open files, ink's awk argument list, a nimib segfault, and a toolset ReadTimeout removing large containers (lektor, hakyll and nextjs-export at 50 KB). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
G (run 38102508908): 78 of 126 cells ok (50 at 10,000 x 5 KB, 28 at 10,000 x 50 KB). Failures are mostly the 6 GB V8 heap and the 8 GB container limit; the JVM generators (cryogen, laika, quarkdown) hit Java heap space at 10,000 x 50 KB, and tanstack-start and vike fail converting a Rust string to a napi string. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Publishes the 2026-10-11 benchmark round (63 generators) and the merge fix it needs.
fix(merge_results): union the spec 009qualitysection across cells and writequality-summary.csv, with tests intests/test_merge_results.py.docs/results/2026-10-11/: suites P, M, G and GG, each withsuite-summary.csv/.mdand per-cellresults.json, plus the roundREADME.mdlinking the workflow runs.The site in BenchCommons/website-results pins its
data/benchmark-toolsubmodule to this branch (4091c07); after merge the pin moves tomaster(website-results#2).Does not touch README.md, AGENTS.md or .gitmodules, so it does not conflict with #34.
🤖 Generated with Claude Code