feat(model-protobuf): re-platform converter onto common-protobuf, drop protobuf-java#1912
Open
jfallows wants to merge 3 commits into
Open
feat(model-protobuf): re-platform converter onto common-protobuf, drop protobuf-java#1912jfallows wants to merge 3 commits into
jfallows wants to merge 3 commits into
Conversation
…p protobuf-java Re-platform the model-protobuf converter off protobuf-java/protobuf-java-util and ANTLR onto the common-protobuf library (which provides common-json): - ProtobufModelHandler caches a common-protobuf ProtobufSchema per schemaId, compiled from catalog .proto source via Protobuf.schema(text). The schema-registry message-index framing (zigzag varint + single-0x00) is preserved verbatim; the record's index path resolves via ProtobufSchema.messageIndexes / messageByIndexes. - Read converter validates via ProtobufSchema.validate; view:json renders through the streaming ProtobufJson generator with proto field names and default inclusion (reproducing the prior JsonFormat configuration). The $.field extract feature is preserved with identical value representations. - Write converter encodes JSON -> wire via the ProtobufJson parser pipeline (view:json) or validates wire and prepends the index prefix. - Remove protobuf-java/protobuf-java-util, the shade plugin, and ANTLR from the build; delete the ANTLR grammars, parser, listeners, and DescriptorTree (the .proto compiler now lives in common-protobuf). module-info requires common.protobuf. NOTICE regenerated. - Add ProtobufModelConfiguration placeholder. Known follow-ups (tracked, pending a common-protobuf change): restore spec-default rejection of unknown JSON fields on the write path, and surface descriptive validation-failure messages from the pipeline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MfzS5MmFH2NQBefvoB3o7d
…del-protobuf re-platform
…lidation reasons Consumes the common-protobuf strictness/reason APIs (#1913): - Write (view: json): enable ProtobufJson.REJECT_UNKNOWN_FIELDS so unknown JSON fields are rejected again (restoring the pre-replatform proto3 JSON behavior), and emit pipeline.reason() as the validation-failure detail instead of the generic "Invalid Protobuf event". - Read: validate via a reusable ProtobufSchema.validatorPipeline (added here) and emit its reason() on failure; fall back to the generic message when a reject carries no reason (e.g. a structural validator reject). - ProtobufSchema.validatorPipeline(name) exposes the reusable parser->validator-> discard pipeline that validate() already built per-call, so callers can read reason(); validate() now delegates to it. Tests: - ProtobufModelTest.shouldWriteInvalidProtobufEventFormatJson reverted to the unknown-field case (well-formed JSON with a field absent from the schema), asserting rejection. - ProtobufModelTest.shouldReadInvalidProtobufEvent captures the emitted event and asserts the descriptive message "...truncated field: need 8 bytes." Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MfzS5MmFH2NQBefvoB3o7d
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The converter layer of #1857: re-platform the
model-protobufconverter offprotobuf-java/protobuf-java-utiland ANTLR onto thecommon-protobuflibrary (merged in #1906 + #1910, which providescommon-json). This realizes the issue's goal of dropping the protobuf/jackson dependencies frommodel-protobuf.Changes
ProtobufModelHandlercaches acommon-protobufProtobufSchemaperschemaId, compiled from the catalog's.protosource viaProtobuf.schema(text). The schema-registry message-index framing (zigzag varint + single-0x00optimization) is preserved verbatim; a record's index path resolves viaProtobufSchema.messageIndexes/messageByIndexes. Padding is recomputed againstProtobufSchema/ProtobufMessage.ProtobufSchema.validate;view: jsonrenders through the streamingProtobufJsongenerator withFIELD_NAMES=PROTO+INCLUDE_DEFAULTS=true, reproducing the priorJsonFormat.preservingProtoFieldNames().includingDefaultValueFields(). The$.fieldextract feature is preserved with byte-identical value representations.ProtobufJsonparser pipeline (view: json) or validates wire and prepends the index prefix.protobuf-java/protobuf-java-util, the shade plugin, and ANTLR from the build; deleted the ANTLR grammars,ProtobufParser,Proto2/3Listener, andDescriptorTree(the.protocompiler now lives incommon-protobuf).module-inforequirescommon.protobuf;NOTICEregenerated. Zerocom.google.protobuf/protobuf-java/ ANTLR references remain in source.ProtobufModelConfigurationplaceholder.Behavioral notes (two tracked follow-ups)
Surfaced during review; both require a small
common-protobufchange and will land as a follow-up PR that this branch then consumes:common-protobuf'sProtobufJsonparser currently ignores them. Follow-up: add a reject-unknown option toProtobufJson.parserand enable it here (restoring spec-default + legacy strictness; theshouldWriteInvalidProtobufEventFormatJsontest is temporarily on malformed JSON instead of an unknown field).Invalid Protobuf event.); the descriptiveprotobuf-javatext isn't available from the streaming pipeline. Follow-up: surface a rejection reason from the parser/validator.Tests
EventITparity preserved. Length/index-prefix (+1/+3, read strips),shouldExtract, and theview: jsonoutput assertions (e.g.{"content":"OK","date_time":"01012024"}) are unchanged.ProtobufParserTest(the parser moved tocommon-protobuf, covered there byProtobufSourceCompilerTest).ProtobufModelEventFormatterTest../mvnw clean install -pl runtime/model-protobuf -DskipITsand./mvnw clean verify -pl runtime/model-protobuf(EventIT) both pass.🤖 Generated with Claude Code
https://claude.ai/code/session_01MfzS5MmFH2NQBefvoB3o7d
Generated by Claude Code