Repository navigation
docs(spec): 010 catalog contract - #32
Merged
Merged
Conversation
Define the contract benchmark-tool owns and catalog-ssg, catalog-algorithms and catalog-web-frameworks consume: catalog.json manifest, implementation layout, pinned Dockerfile, byte-identical runner, metadata base schema and index, untimed verification, generic BENCH_* result markers with SSGBERK_* aliases, and the datarate, algorithm and http test types. Refs #29 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #29
Spec only (no code):
docs/specs/010-catalog-contract/{spec.md,plan.md,tasks.md}.This repo owns the catalog contract (
.productCONTRACTS.md); catalog-ssg, catalog-algorithms and catalog-web-frameworks reference it.What it defines
catalog.jsonmanifest per catalog:contractVersion, test types, implementation glob, runner, metadata file/schema, index file and its--checkcommand.<id>.dockerfile(exact base tag,ARG *_VERSIONpins, engine inside the image),benchmark_config.jsonin the TechEmpower shape plustest_type, metadata validated against a BT base schema, byte-identical runner copy.httpis verified by the toolset (TechEmpower verify stage) and has no runner.BENCH_*markers (VERIFY_OK/FAIL,RESULT_BEGIN engine=… version=…/RESULT_END,UNSUPPORTED, …); everySSGBERK_*stays an accepted alias for all ofcontractVersion: 1, withmarkerPrefixrecorded per result.datarate(SSG),algorithm(algorithm,n, optional dataset),http(endpoints, concurrency; interface only);--catalog/--catalog-dir,catalogs/<name>/submodules, suites tagged by catalog and test type.results.jsonstaysschemaVersion: 2with an optionalcatalogkey; v3 and perfratio parsing belong to Replace hyperfine with perfratio as the measurement engine #30 / Requirements for measuring the same algorithm across many languages (catalog-algorithms) perfratio#240.tools/check_catalog.pyfor catalog CI.Out of scope: perfratio switch, renames (
ssgberkwrapper, image names), thehttpload generator/verifier implementation, CA's algorithm set, changes inside catalogs.Migration table in
spec.md: catalog-ssg conforms by adding onlycatalog.json; its output parses exactly as today.Note: docs/specs/ROADMAP.md is not updated here, to avoid touching existing specs; it can be updated in a follow-up.
🤖 Generated with Claude Code