Skip to content

feat(model-json): streaming projecting + validating JSON converter#1907

Open
jfallows wants to merge 1 commit into
developfrom
claude/laughing-noether-qptywk
Open

feat(model-json): streaming projecting + validating JSON converter#1907
jfallows wants to merge 1 commit into
developfrom
claude/laughing-noether-qptywk

Conversation

@jfallows

Copy link
Copy Markdown
Contributor

Description

Surfaces the streaming, single-pass projecting + validating path as the model-json converter, built on the common-json pipeline primitives (#1855 / #1846).

The read/write converters are re-platformed onto the common-json streaming pipeline parser → schema.validator() → projector(schema) → generator, producing a compact, schema-pruned, validated document into a MutableDirectBuffer — no full-document DOM, no per-call builder allocation. Projection pointers are derived from the schema via JsonSchema.retainedPaths(). The JsonSchema is compiled and cached per schemaId; the pipeline is cached per schema.

Per maintainer direction, this lands the converter layer as streaming + projecting and keeps the validator handler streaming validate-only for now; the converter/validator unification (single fragment-aware convert) is tracked by #1836. The converter currently receives the whole value in one convert(...) call, so the pipeline is driven with a single feed(..., last=true).

Failure model — streaming-validate-then-abort

On a late validation failure the pipeline reports REJECTED; the converter runs a diagnostic pass to surface the schema diagnostic via validationFailure and returns -1, so the binding aborts the downstream stream (engine RESET) rather than buffering to reject before forwarding. EventIT exercises this end-to-end and still asserts the exact diagnostic ([1,11][/id] expected string but was number).

Changes

  • JsonModelHandler — per-schemaId JsonSchema cache + per-schema pipeline cache; project(...) drives the pipeline into a grow-on-demand buffer (wrapped at full capacity with headroom, since the sink suspends when remaining() < 16); diagnostic pass on rejection; field extraction retained as a streaming pass over the input.
  • JsonReadConverterHandler / JsonWriteConverterHandler — emit the projected document (read forwards the projected buffer then extracts; write encodes the projected buffer).
  • JsonValidatorHandler — unchanged (already streaming validate-only).
  • JsonConverterTest — updated test-first to assert the projected (compacted, pruned) output and lengths.

Scope / compatibility

Isolated to model-json: no other module routes model: json through supplyReadConverter / supplyWriteConverter (e.g. binding-mcp-http resolves schema text itself for its own projector), so the change in forwarded bytes does not affect other bindings.

Tests

./mvnw verify -pl runtime/model-jsonBUILD SUCCESS: 16 unit tests + EventIT green; checkstyle, license, and the JaCoCo coverage gate all pass.

Fixes #1835

🤖 Generated with Claude Code


Generated by Claude Code

Re-platform the model-json read/write converters onto the common-json
streaming pipeline (parser -> validator -> projector -> generator),
producing a compact, schema-pruned, validated document into a
MutableDirectBuffer with no full-document DOM and no per-call builder
allocation. Projection pointers are derived from the schema via
JsonSchema.retainedPaths(); the JsonSchema is compiled and cached per
schemaId and the pipeline cached per schema.

On a validation failure the streaming pipeline reports REJECTED; the
converter abuts a diagnostic pass to surface the schema diagnostic via
validationFailure and returns -1 so the binding aborts the downstream
stream (engine RESET). Field extraction is retained as a streaming pass
over the input. The validator handler remains streaming validate-only
for now; converter + validator unification is tracked separately.

Relates to #1835
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

model-json: streaming projecting + validating JSON converter (on common-json)

2 participants