feat(model-json): streaming projecting + validating JSON converter#1907
Open
jfallows wants to merge 1 commit into
Open
feat(model-json): streaming projecting + validating JSON converter#1907jfallows wants to merge 1 commit into
jfallows wants to merge 1 commit into
Conversation
Re-platform the model-json read/write converters onto the common-json streaming pipeline (parser -> validator -> projector -> generator), producing a compact, schema-pruned, validated document into a MutableDirectBuffer with no full-document DOM and no per-call builder allocation. Projection pointers are derived from the schema via JsonSchema.retainedPaths(); the JsonSchema is compiled and cached per schemaId and the pipeline cached per schema. On a validation failure the streaming pipeline reports REJECTED; the converter abuts a diagnostic pass to surface the schema diagnostic via validationFailure and returns -1 so the binding aborts the downstream stream (engine RESET). Field extraction is retained as a streaming pass over the input. The validator handler remains streaming validate-only for now; converter + validator unification is tracked separately. Relates to #1835
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Surfaces the streaming, single-pass projecting + validating path as the
model-jsonconverter, built on thecommon-jsonpipeline primitives (#1855/#1846).The read/write converters are re-platformed onto the
common-jsonstreaming pipelineparser → schema.validator() → projector(schema) → generator, producing a compact, schema-pruned, validated document into aMutableDirectBuffer— no full-document DOM, no per-call builder allocation. Projection pointers are derived from the schema viaJsonSchema.retainedPaths(). TheJsonSchemais compiled and cached perschemaId; the pipeline is cached per schema.Per maintainer direction, this lands the converter layer as streaming + projecting and keeps the validator handler streaming validate-only for now; the converter/validator unification (single fragment-aware
convert) is tracked by #1836. The converter currently receives the whole value in oneconvert(...)call, so the pipeline is driven with a singlefeed(..., last=true).Failure model — streaming-validate-then-abort
On a late validation failure the pipeline reports
REJECTED; the converter runs a diagnostic pass to surface the schema diagnostic viavalidationFailureand returns-1, so the binding aborts the downstream stream (engine RESET) rather than buffering to reject before forwarding.EventITexercises this end-to-end and still asserts the exact diagnostic ([1,11][/id] expected string but was number).Changes
JsonModelHandler— per-schemaIdJsonSchemacache + per-schema pipeline cache;project(...)drives the pipeline into a grow-on-demand buffer (wrapped at full capacity with headroom, since the sink suspends whenremaining() < 16); diagnostic pass on rejection; field extraction retained as a streaming pass over the input.JsonReadConverterHandler/JsonWriteConverterHandler— emit the projected document (read forwards the projected buffer then extracts; write encodes the projected buffer).JsonValidatorHandler— unchanged (already streaming validate-only).JsonConverterTest— updated test-first to assert the projected (compacted, pruned) output and lengths.Scope / compatibility
Isolated to
model-json: no other module routesmodel: jsonthroughsupplyReadConverter/supplyWriteConverter(e.g.binding-mcp-httpresolves schema text itself for its own projector), so the change in forwarded bytes does not affect other bindings.Tests
./mvnw verify -pl runtime/model-json— BUILD SUCCESS: 16 unit tests +EventITgreen; checkstyle, license, and the JaCoCo coverage gate all pass.Fixes #1835
🤖 Generated with Claude Code
Generated by Claude Code