[Router] Interfaces + fixtures stub (foundation)#2408
Open
ramkrishna2910 wants to merge 6 commits into
Open
Conversation
f6030ca to
2b99fb6
Compare
16 tasks
2b99fb6 to
371aa92
Compare
Land the shared C++ contract surface + schema fixtures the rest of the
routing engine codes against — the first development step after schema
sign-off, gating the parallel engine/wiring tracks.
- routing_policy.h: RouteContext, Score (label->score), ClassifierServices
(embed/run_classifier/chat injection seam), Classifier, MatchExpr AST,
Condition + EvalContext + LeafFactory (the runtime-evaluation seam shared by
the evaluator and the registry), Rule, Decision/TraceEntry, RoutePolicy,
RoutingPolicyEngine ctor. Std + nlohmann/json only; no Router/backend include.
route() declared, not defined.
- src/cpp/resources/schemas: frozen route_policy + decision JSON Schemas + README.
Both carry a required root `version` ("1") and pin engine-owned v1 semantics
(keywords=substring, regex=ECMAScript, inclusive bands w/ default 0.5,
min/max_chars=UTF-8 bytes, on_error default match_false, router desugaring).
README documents the back-compat contract (never redefine a shipped field;
never delete a major's schema; migrate-on-load).
- Mechanical enforcement: schema-lock.json + test/test_schema_lock.py
(canonical-hash snapshot guard); test/test_routing_fixtures.py (schemas
self-valid + fixtures conformant).
- test/cpp/fixtures/routing: lean L0a-L3 collection.router examples + decision.
- fake_classifier_services.h + test_routing_policy_contract.cpp (CTest
RoutingPolicyContractTest): header compiles standalone, types construct,
Condition/EvalContext/LeafFactory seam exercised, fake services callable,
engine constructible, fixtures satisfy locked invariants.
Comments describe concepts, not tracker IDs; the issue linkage lives in this
commit and the PR. Depends on schema sign-off; landing as draft until then.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
371aa92 to
d5f9772
Compare
The routing contract test and schema guards were added but nothing in CI ran them — making the schema-lock and fixture validation decorative. Wire them in: - cpp_server_build_test_release.yml: build test_routing_policy_contract and add RoutingPolicyContract to the ctest filter (both the Debian and macOS C++ unit test steps). - routing_schema_tests.yml: new fast, server-less PR lane that runs test/test_routing_fixtures.py (fixtures validate against the schemas) and test/test_schema_lock.py (frozen-major snapshot guard). Verified in-tree: cmake builds the target, ctest -R RoutingPolicyContract passes, and both python tests pass as invoked. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`candidates` was overloaded: routing.candidates is the routing-target set (a core concept), while the semantic_similarity classifier reused the same word for its exemplar phrases — visibly ambiguous in the L2 fixture, which had `candidates` meaning two different things. Rename the classifier field to `reference_phrases` (per review on #2375); routing.candidates is unchanged. Lock refreshed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SlawomirNowaczyk
approved these changes
Jun 25, 2026
eddierichter-amd
approved these changes
Jun 25, 2026
eddierichter-amd
left a comment
Contributor
There was a problem hiding this comment.
Just some small nits. Otherwise looks good.
(cherry picked from commit 26b640f)
From @eddierichter-amd's review on #2408: - Add request.schema.json + request_example.json fixture: covers the request-side extension surface from #2376 (OpenAI `metadata` with string values + optional `route_trace`) that was missing — completes the schema deliverable. Validates only the extension fields (additionalProperties: true); no version (rides on the stock OpenAI request). - Constrain `min_score`/`max_score` to [0, 1] (the Score contract range). - Classifier type-specific required fields via if/then: semantic_similarity needs reference_phrases, classifier needs model, llm needs model + prompt — moving type validation into the schema instead of leaving it to the parser. Lock refreshed; README + both test harnesses updated (incl. a non-string-metadata rejection test and a conditional-requirement test). Fixes a dangling-temporary bug in the request check (nlohmann .items() on a .value() temporary). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Thanks @eddierichter-amd — all three addressed in the latest commit:
Lock refreshed, README + both test harnesses updated. Good catches — the request schema in particular was a real gap against #2376's deliverable. |
This was referenced Jun 25, 2026
Sign-off approves the design and unblocks development; it does not freeze the schema. `released` stays false through implementation (schema still refinable with a reviewed lock refresh) and flips to true at product release, when real policies exist and immutability protects users. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
@ramkrishna2910 one more question now that I am scoping out #2379. Should the classifier contract expose declared The registry/leaf factory needs this to:
Something like: const std::vectorstd::string& labels() const; and corresponding protected constructor fields would make #2379 implementable without keeping a sidecar metadata table outside the Classifier. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2407.
Foundation for the Lemonade Router milestone (#2389): the shared C++ contract surface, the policy/decision JSON schemas, the back-compat guardrails, and the per-tier example policies that every downstream engine/wiring issue codes against. Interfaces only — no engine behavior (
route()is declared, not implemented; that's #2382).Contract header —
src/cpp/include/lemon/routing_policy.hStd library + nlohmann/json only — no Router/backend include, per the engine invariant.
RouteContext,Score(label→score),OnError,Rule,MatchExprAST,TraceEntry,Decision,RoutePolicy.ClassifierServices(embed/run_classifier/chat—chatpowers the L0a llm router),ClassifierContext,Classifier.Condition,EvalContext(per-request state: classifier memo + trace),LeafFactory— the shared contract that lets the evaluator ([Router] Core types + match-expression evaluator (M6/M7) #2378) and registry ([Router] Classifier + Condition registry (M3) #2379) be built in parallel without colliding at the leaf boundary.RoutingPolicyEnginector signature;route()declared, intentionally undefined.Schemas —
src/cpp/resources/schemas/route_policy.schema.json([Schema][Router] route_policy — routing block in collection.router JSON #2375),request.schema.json(request-sidemetadata+route_trace, [Schema][Router] Decision + trace object (pure selection) #2376),decision.schema.json([Schema][Router] Decision + trace object (pure selection) #2376),README.md.version("1"); the request schema rides on the stock OpenAI body so it has no version.[0,1](defaultmin_score: 0.5),min/max_chars= UTF-8 bytes,metadatamatch = case-sensitive comma-split token set,on_errordefaultmatch_false,routing.routerdesugaring behavior-equivalent.semantic_similarity→reference_phrases,llm→prompt, …) viaif/then.Back-compat guardrails
The README documents the contract — never redefine a shipped field; never delete a major's schema; migrate-on-load — enforced mechanically, not by convention:
schema-lock.json+test/test_schema_lock.py— canonical-hash snapshot guard (buf breaking-lite). A released major is immutable (any change fails CI → ship a new major); an unreleased major may change only with a refreshed lock in the same diff, so every schema edit is a visible, reviewed change. (v1 isreleased: falseuntil sign-off.)test/test_routing_fixtures.py— schemas are self-valid and every fixture conforms.(Behavioral back-compat — golden policy →
Decision— is the conformance corpus #2425, non-gating, tracked separately.)Fixtures —
test/cpp/fixtures/routing/Lean local-form examples, verbatim from the locked level issues:
l0a_llm_router.jsonl1_keywords.jsonl1_metadata.jsonmetadatamatch —equals/any/exists)l2_semantic.jsonl3_classifier.jsondecision_example.jsonThe
metadataleaf condition (lets rules branch on caller-supplied OpenAImetadataliketask_class/consent) was contributed by @SlawomirNowaczyk.Tests
test/cpp/test_routing_policy_contract.cpp→ CTestRoutingPolicyContractTest: header compiles standalone, every type constructs, theCondition/EvalContext/LeafFactoryseam is exercised, the fakeClassifierServicesis callable, the engine is constructible, and fixtures satisfy the locked cross-field invariants (default_model∈ candidates; everyroute_to∈ candidates; classifier refs resolve).test/cpp/fake_classifier_services.h: configurable fixed embeddings / scores / replies.test/test_routing_fixtures.py: fixtures validate against the JSON Schemas + cross-field invariants (jsonschema, added totest/requirements.txt).test/test_schema_lock.py: the frozen-major snapshot guard.Acceptance (#2407) — all met
ClassifierServicesbuilds and is usable from a trivial testVerification
/W4(MSVC, VS2026); builds in-tree via CMake;ctest -R RoutingPolicyContractpasses.test_routing_fixtures.pyandtest_schema_lock.pypass; Black-clean.RoutingPolicyContractTestis built + run in the C++ unit-test steps, and a fast server-lessRouting Schema Testslane runs the two Python guards (so the schema lock + fixture validation are enforced, not decorative).🤖 Generated with Claude Code