perf: typed RDF literal storage + resolveLiteral API — indexed WHERE, boot migrations, envelope-free writes#842
Conversation
|
Warning Review limit reached
More reviews will be available in 53 minutes and 7 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (26)
📝 WalkthroughWalkthroughReplaces the string-based ChangesResolve-literal model and storage convergence
Sequence Diagram(s)sequenceDiagram
participant Client
participant Ad4mModel
participant sparql_store
participant oxigraph
rect rgba(70, 130, 180, 0.5)
note over Client,oxigraph: Write path (resolveLiteral === true, default)
Client->>Ad4mModel: setProperty("name", "Alice")
Ad4mModel->>Ad4mModel: valueToLiteralIri("Alice") → "literal:string:Alice"
Ad4mModel->>sparql_store: addLink(target="literal:string:Alice")
sparql_store->>sparql_store: target_to_storage_term → "Alice"^^xsd:string
sparql_store->>oxigraph: store typed Literal triple
end
rect rgba(180, 100, 50, 0.5)
note over Client,oxigraph: Query path
Client->>Ad4mModel: query(where={name: "Alice"})
Ad4mModel->>sparql_store: SPARQL VALUES ?name { "Alice"^^xsd:string }
sparql_store->>oxigraph: evaluate pattern
oxigraph-->>sparql_store: Literal("Alice", xsd:string)
sparql_store->>sparql_store: storage_term_to_target_string → "literal:string:Alice"
sparql_store-->>Ad4mModel: result rows
Ad4mModel-->>Client: model instances
end
rect rgba(60, 160, 60, 0.5)
note over Client,oxigraph: Migration path (startup)
sparql_store->>sparql_store: migrate_signed_envelopes_to_plain_literals (v3)
sparql_store->>sparql_store: migrate_iri_literals_to_typed_literals (v4)
sparql_store->>oxigraph: rewrite legacy NamedNode targets → typed Literals
end
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
a9ed90b to
d82cc9a
Compare
Hypothesis fix for the model-test failures on #842 caused by the typed-literal storage layer eating relation targets. When a model instance is constructed without an explicit baseExpression, we used to generate `Literal.from(makeRandomId(24)).toUrl()` → `literal:string:<id>`. That format is fine pre-typed-literal because the storage layer treats it as a NamedNode IRI both ways. After #842's `target_to_storage_term`, however, any `literal:string:X` *target* gets stripped down to a typed `"X"^^xsd:string` literal — which is correct for property values but wrong for relation targets that need to remain NamedNode IRIs pointing at sibling instances. A `<post> --has_comment--> <literal:string:abc>` link would land as `<post> --has_comment--> "abc"^^xsd:string`, and the include sub-query that later does `VALUES ?source { <literal:string:abc> }` would no longer find the comment as a NamedNode source. Switch to a dedicated `ad4m://obj/<id>` IRI scheme so auto-generated baseExpressions can never be mistaken for property string literals. Production callers that pass an explicit `baseExpression` are unaffected. `Literal.fromUrl` is only used in tests and runtime code against `link.data.target` (property values), never against instance IDs, so this scheme change doesn't break any existing decoder.
Hypothesis fix for the model-test failures on #842 caused by the typed-literal storage layer eating relation targets. When a model instance is constructed without an explicit baseExpression, we used to generate `Literal.from(makeRandomId(24)).toUrl()` → `literal:string:<id>`. That format is fine pre-typed-literal because the storage layer treats it as a NamedNode IRI both ways. After #842's `target_to_storage_term`, however, any `literal:string:X` *target* gets stripped down to a typed `"X"^^xsd:string` literal — which is correct for property values but wrong for relation targets that need to remain NamedNode IRIs pointing at sibling instances. A `<post> --has_comment--> <literal:string:abc>` link would land as `<post> --has_comment--> "abc"^^xsd:string`, and the include sub-query that later does `VALUES ?source { <literal:string:abc> }` would no longer find the comment as a NamedNode source. Switch to a dedicated `ad4m://obj/<id>` IRI scheme so auto-generated baseExpressions can never be mistaken for property string literals. Production callers that pass an explicit `baseExpression` are unaffected. `Literal.fromUrl` is only used in tests and runtime code against `link.data.target` (property values), never against instance IDs, so this scheme change doesn't break any existing decoder.
3a8c2ea to
5b27d38
Compare
…st IDs The five type-filtered relation tests on this PR's branch reused `Literal.from(label).toUrl()` to mint deterministic baseExpressions for the Article/Comment/Message instances they create AND for the `target` of the relation links between them. The typed-literal storage layer (#842) strips `literal:string:X` link targets down to `"X"^^xsd:string` typed literals, which then can't be subjects under the `?target <type-pred> <type-value>` conformance probe that drives the type-filter — so every test came back with 0 matches and failed. Switching the test IDs to a dedicated `ad4m://test/<slug>` scheme keeps them as NamedNode IRIs end-to-end: as link targets they stay routable, and the type-filter conformance probe finds the corresponding subject triples. Production behaviour is unchanged — the auto-generated baseExpressions in Ad4mModel already use the matching `ad4m://obj/<id>` scheme; this is a pure test fixture adjustment. Updated: - can constrain collection entries through 'where' clause - can constrain relation entries through SPARQL getter - should filter collection by type with class reference - should filter collection by type with string class name - should filter results in findAll() by type
28389fd to
526942c
Compare
Hypothesis fix for the model-test failures on #842 caused by the typed-literal storage layer eating relation targets. When a model instance is constructed without an explicit baseExpression, we used to generate `Literal.from(makeRandomId(24)).toUrl()` → `literal:string:<id>`. That format is fine pre-typed-literal because the storage layer treats it as a NamedNode IRI both ways. After #842's `target_to_storage_term`, however, any `literal:string:X` *target* gets stripped down to a typed `"X"^^xsd:string` literal — which is correct for property values but wrong for relation targets that need to remain NamedNode IRIs pointing at sibling instances. A `<post> --has_comment--> <literal:string:abc>` link would land as `<post> --has_comment--> "abc"^^xsd:string`, and the include sub-query that later does `VALUES ?source { <literal:string:abc> }` would no longer find the comment as a NamedNode source. Switch to a dedicated `ad4m://obj/<id>` IRI scheme so auto-generated baseExpressions can never be mistaken for property string literals. Production callers that pass an explicit `baseExpression` are unaffected. `Literal.fromUrl` is only used in tests and runtime code against `link.data.target` (property values), never against instance IDs, so this scheme change doesn't break any existing decoder.
…st IDs The five type-filtered relation tests on this PR's branch reused `Literal.from(label).toUrl()` to mint deterministic baseExpressions for the Article/Comment/Message instances they create AND for the `target` of the relation links between them. The typed-literal storage layer (#842) strips `literal:string:X` link targets down to `"X"^^xsd:string` typed literals, which then can't be subjects under the `?target <type-pred> <type-value>` conformance probe that drives the type-filter — so every test came back with 0 matches and failed. Switching the test IDs to a dedicated `ad4m://test/<slug>` scheme keeps them as NamedNode IRIs end-to-end: as link targets they stay routable, and the type-filter conformance probe finds the corresponding subject triples. Production behaviour is unchanged — the auto-generated baseExpressions in Ad4mModel already use the matching `ad4m://obj/<id>` scheme; this is a pure test fixture adjustment. Updated: - can constrain collection entries through 'where' clause - can constrain relation entries through SPARQL getter - should filter collection by type with class reference - should filter collection by type with string class name - should filter results in findAll() by type
526942c to
94da674
Compare
94da674 to
7bbbf2e
Compare
Hypothesis fix for the model-test failures on #842 caused by the typed-literal storage layer eating relation targets. When a model instance is constructed without an explicit baseExpression, we used to generate `Literal.from(makeRandomId(24)).toUrl()` → `literal:string:<id>`. That format is fine pre-typed-literal because the storage layer treats it as a NamedNode IRI both ways. After #842's `target_to_storage_term`, however, any `literal:string:X` *target* gets stripped down to a typed `"X"^^xsd:string` literal — which is correct for property values but wrong for relation targets that need to remain NamedNode IRIs pointing at sibling instances. A `<post> --has_comment--> <literal:string:abc>` link would land as `<post> --has_comment--> "abc"^^xsd:string`, and the include sub-query that later does `VALUES ?source { <literal:string:abc> }` would no longer find the comment as a NamedNode source. Switch to a dedicated `ad4m://obj/<id>` IRI scheme so auto-generated baseExpressions can never be mistaken for property string literals. Production callers that pass an explicit `baseExpression` are unaffected. `Literal.fromUrl` is only used in tests and runtime code against `link.data.target` (property values), never against instance IDs, so this scheme change doesn't break any existing decoder.
…st IDs The five type-filtered relation tests on this PR's branch reused `Literal.from(label).toUrl()` to mint deterministic baseExpressions for the Article/Comment/Message instances they create AND for the `target` of the relation links between them. The typed-literal storage layer (#842) strips `literal:string:X` link targets down to `"X"^^xsd:string` typed literals, which then can't be subjects under the `?target <type-pred> <type-value>` conformance probe that drives the type-filter — so every test came back with 0 matches and failed. Switching the test IDs to a dedicated `ad4m://test/<slug>` scheme keeps them as NamedNode IRIs end-to-end: as link targets they stay routable, and the type-filter conformance probe finds the corresponding subject triples. Production behaviour is unchanged — the auto-generated baseExpressions in Ad4mModel already use the matching `ad4m://obj/<id>` scheme; this is a pure test fixture adjustment. Updated: - can constrain collection entries through 'where' clause - can constrain relation entries through SPARQL getter - should filter collection by type with class reference - should filter collection by type with string class name - should filter results in findAll() by type
Hypothesis fix for the model-test failures on #842 caused by the typed-literal storage layer eating relation targets. When a model instance is constructed without an explicit baseExpression, we used to generate `Literal.from(makeRandomId(24)).toUrl()` → `literal:string:<id>`. That format is fine pre-typed-literal because the storage layer treats it as a NamedNode IRI both ways. After #842's `target_to_storage_term`, however, any `literal:string:X` *target* gets stripped down to a typed `"X"^^xsd:string` literal — which is correct for property values but wrong for relation targets that need to remain NamedNode IRIs pointing at sibling instances. A `<post> --has_comment--> <literal:string:abc>` link would land as `<post> --has_comment--> "abc"^^xsd:string`, and the include sub-query that later does `VALUES ?source { <literal:string:abc> }` would no longer find the comment as a NamedNode source. Switch to a dedicated `ad4m://obj/<id>` IRI scheme so auto-generated baseExpressions can never be mistaken for property string literals. Production callers that pass an explicit `baseExpression` are unaffected. `Literal.fromUrl` is only used in tests and runtime code against `link.data.target` (property values), never against instance IDs, so this scheme change doesn't break any existing decoder.
…or typed-literal storage
Legacy notification triggers and Flux-style mention-detection SPARQL
queries call `<ad4m://fn/parse_literal>(?target)` to unwrap the inner
`data` field of signed-expression envelopes — without it, the trigger
filter ends up matching the envelope's `author` field too (which is
always the local agent's DID), producing false positives on every
incoming message.
The typed-literal storage layer dropped the registration along with
`<ad4m://fn/parse_literal>`'s only previous role (NamedNode→string
unwrap), but the envelope-unwrap use-case has no equivalent in plain
SPARQL once values land as `"{...}"^^ad4m:json` typed literals.
Re-register `parse_literal` as a back-compat shim covering both the new
storage and any pre-migration NamedNode form:
- Typed `ad4m:json` literal → extract `.data` field (envelope unwrap)
- Typed `xsd:*` literal → return lexical form (same as STR)
- NamedNode `literal:string|number|boolean|json:…` → legacy decode
(kept for any feeds still emitting that wire format)
- Anything else → return unchanged
Restores parity with the runtime test "can detect mentions in
notifications (Flux example)" without forcing every callsite to
migrate to a new function name in this PR.
…st IDs The five type-filtered relation tests on this PR's branch reused `Literal.from(label).toUrl()` to mint deterministic baseExpressions for the Article/Comment/Message instances they create AND for the `target` of the relation links between them. The typed-literal storage layer (#842) strips `literal:string:X` link targets down to `"X"^^xsd:string` typed literals, which then can't be subjects under the `?target <type-pred> <type-value>` conformance probe that drives the type-filter — so every test came back with 0 matches and failed. Switching the test IDs to a dedicated `ad4m://test/<slug>` scheme keeps them as NamedNode IRIs end-to-end: as link targets they stay routable, and the type-filter conformance probe finds the corresponding subject triples. Production behaviour is unchanged — the auto-generated baseExpressions in Ad4mModel already use the matching `ad4m://obj/<id>` scheme; this is a pure test fixture adjustment. Updated: - can constrain collection entries through 'where' clause - can constrain relation entries through SPARQL getter - should filter collection by type with class reference - should filter collection by type with string class name - should filter results in findAll() by type
Separates the literal expression language contract from deterministic
indexed storage. `resolveLiteral: true` (the default) stores values as
deterministic `literal:string:X` IRIs for efficient Oxigraph POS-index
lookups. `resolveLiteral: false` routes through `expression_create` on
the literal language, producing signed-envelope URIs with
author/timestamp/proof.
This resolves the design tension identified in PR review: the AD4M
expression language contract (expressionCreate → address →
expressionGet → {author,timestamp,data,proof}) is now preserved for
properties that opt into it, while the default path remains optimised
for indexed storage. Backward compat with old SHACL data
(ad4m://resolveLanguage → "literal") is handled in both TS and Rust.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests still referenced the removed resolveLanguage property, causing TS compilation failures in CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Ad4mModel.ts: defer all resolveLiteral:false props during creation, not just objects — primitives need expression path too 2. shacl-gen.ts: infer SHACL datatype from JS field type, not from resolveLiteral flag — numbers/booleans keep their proper xsd types 3. SHACLShape.ts: fall back to legacy resolve_language/resolveLanguage keys when loading SHACL JSON, mapping "literal" → resolveLiteral:true 4. mod.rs: v4 migration failure now blocks perspective init (matching v3) to prevent mixed IRI/typed-literal state 5. shape.rs: non-literal legacy resolveLanguage values map to Some(false) instead of being silently dropped (both SHACL-link and JSON paths) 6. perspective_instance.rs: propagate expression_create failure instead of falling back to raw JSON storage Skipped: sdna.ts resolveLiteral !== false → === true suggestion. The default is true (per PropertyOptions docs), so !== false correctly treats undefined as "use default". Changing to === true would break properties that don't explicitly set resolveLiteral. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
455b4ba to
3729e1f
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/src/model/Ad4mModel.ts (1)
423-450:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate the constructor docs to stop recommending literal-backed IDs.
The implementation now avoids
literal:string:<id>because relation targets with that shape can be stored as typed literals instead of IRIs, but the JSDoc still says the default is a “random Literal URL” and shows"literal:..."as the explicit-ID example. Please update the public docs/example to usead4m://obj/<id>or another non-literal:*IRI.Proposed doc fix
- * `@param` baseExpression - Optional expression URI for this instance. - * If omitted, a random Literal URL is generated. + * `@param` baseExpression - Optional expression URI for this instance. + * If omitted, a random `ad4m://obj/<id>` IRI is generated. @@ - * // Create with specific base expression - * const recipe = new Recipe(perspective, "literal:..."); + * // Create with specific base expression + * const recipe = new Recipe(perspective, "ad4m://obj/custom-id");🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/src/model/Ad4mModel.ts` around lines 423 - 450, Update the JSDoc comment for the constructor method to reflect the actual implementation. Change the description from saying a "random Literal URL is generated" to clarify that it uses the `ad4m://obj/<id>` scheme. Update the example that shows `"literal:..."` as the explicit baseExpression to instead show `"ad4m://obj/..."` format to be consistent with the implementation and stop recommending literal-backed IDs for relation targets.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/src/model/query-utils.ts`:
- Line 63: The default value of resolveLiteral at line 63 (in the isLiteral
assignment) is currently routing queries through legacy literal-string encoding
paths, which conflicts with the typed-literal query contract. Change the default
behavior so that resolveLiteral does not default to true, ensuring that queries
use typed-literal handling instead of legacy compatibility behavior. Also review
how resolveLiteral is forwarded at line 200 to ensure consistency with this new
default and that typed comparisons are not inadvertently routed through the
legacy literal-string encoding paths.
---
Outside diff comments:
In `@core/src/model/Ad4mModel.ts`:
- Around line 423-450: Update the JSDoc comment for the constructor method to
reflect the actual implementation. Change the description from saying a "random
Literal URL is generated" to clarify that it uses the `ad4m://obj/<id>` scheme.
Update the example that shows `"literal:..."` as the explicit baseExpression to
instead show `"ad4m://obj/..."` format to be consistent with the implementation
and stop recommending literal-backed IDs for relation targets.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fc720108-2dfb-4a9e-8f07-d992b59d5348
📒 Files selected for processing (35)
core/src/Literal.tscore/src/model/Ad4mModel.test.tscore/src/model/Ad4mModel.tscore/src/model/decorators.tscore/src/model/json-schema.tscore/src/model/query-sparql.test.tscore/src/model/query-sparql.tscore/src/model/query-utils.tscore/src/model/sdna.tscore/src/model/shacl-gen.tscore/src/model/shacl-roundtrip.test.tscore/src/model/types.tscore/src/perspectives/PerspectiveProxy.tscore/src/perspectives/SparqlBindings.tscore/src/shacl/SHACLShape.test.tscore/src/shacl/SHACLShape.tscore/src/shacl/index.tsrust-executor/src/languages/literal.rsrust-executor/src/mcp/shacl.rsrust-executor/src/mcp/tools/mod.rsrust-executor/src/mcp/tools/subscriptions.rsrust-executor/src/perspectives/migration.rsrust-executor/src/perspectives/mod.rsrust-executor/src/perspectives/model_query/integration_tests.rsrust-executor/src/perspectives/model_query/projection.rsrust-executor/src/perspectives/model_query/query.rsrust-executor/src/perspectives/model_query/shape.rsrust-executor/src/perspectives/model_query/sparql_builder.rsrust-executor/src/perspectives/model_query/types.rsrust-executor/src/perspectives/model_query/utils.rsrust-executor/src/perspectives/perspective_instance.rsrust-executor/src/perspectives/shacl_parser.rsrust-executor/src/perspectives/sparql_store.rstests/js/tests/mcp-http.test.tstests/js/tests/prolog-and-literals.test.ts
✅ Files skipped from review due to trivial changes (2)
- core/src/shacl/index.ts
- rust-executor/src/mcp/tools/mod.rs
🚧 Files skipped from review as they are similar to previous changes (25)
- core/src/Literal.ts
- core/src/model/shacl-roundtrip.test.ts
- rust-executor/src/mcp/tools/subscriptions.rs
- rust-executor/src/perspectives/migration.rs
- rust-executor/src/perspectives/shacl_parser.rs
- core/src/model/shacl-gen.ts
- core/src/perspectives/SparqlBindings.ts
- rust-executor/src/perspectives/model_query/types.rs
- core/src/model/sdna.ts
- core/src/model/query-sparql.ts
- rust-executor/src/mcp/shacl.rs
- core/src/model/query-sparql.test.ts
- tests/js/tests/mcp-http.test.ts
- rust-executor/src/languages/literal.rs
- rust-executor/src/perspectives/perspective_instance.rs
- rust-executor/src/perspectives/model_query/projection.rs
- rust-executor/src/perspectives/model_query/query.rs
- rust-executor/src/perspectives/model_query/sparql_builder.rs
- rust-executor/src/perspectives/model_query/utils.rs
- core/src/model/Ad4mModel.test.ts
- core/src/perspectives/PerspectiveProxy.ts
- core/src/model/decorators.ts
- core/src/model/json-schema.ts
- rust-executor/src/perspectives/model_query/shape.rs
- rust-executor/src/perspectives/sparql_store.rs
…data model Restores resolve_language (general expression-language selector) as a first-class field on ShapeProperty and PropertyShape, kept beside the resolve_literal optimization flag added on this branch. - types.rs / shacl_parser.rs: resolve_language: Option<String> re-added. - parse_shacl_to_links: emits both ad4m://resolveLanguage and ad4m://resolveLiteral links when set. - shape.rs (load_shape + load_shape_from_meta): populates resolve_language from RDF/meta; resolve_literal still derived from resolveLanguage for backward compat when the explicit flag is absent. resolve_language is the dev mechanism (route values through any language); resolve_literal is the literal-only deterministic-storage optimization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore get_resolve_language_from_shacl (string selector) beside
get_resolve_literal_from_shacl. resolve_property_value now applies the
unified precedence:
1. custom resolve_language (!= "literal") -> expression_create(lang)
-> signed-envelope URI (the dev behavior, lenient fallback on error)
2. literal language + resolveLiteral: false -> expression_create("literal")
3. otherwise -> deterministic literal: IRI (optimized default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lve_literal Add ShapeProperty::is_deterministic_literal() — true only for the literal language (explicit or default) with resolve_literal != Some(false). Custom languages always produce signed envelopes, so they are never deterministic. - sparql_builder / projection: WHERE matching uses is_deterministic_literal() to decide typed-literal VALUES vs FILTER-on-decoded probing. - query.rs resolve_language_transforms: now fetches+transforms any non-deterministic-literal property (custom language OR resolveLiteral:false), restoring dev's general expression-resolution read path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ShaclProperty carries resolve_language beside resolve_literal again, read from ad4m://resolveLanguage in load_class_properties_with_uri. Add resolve_property_resolve_language helper beside resolve_property_resolve_literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… layer resolveLanguage (general expression-language selector, default "literal") is a first-class @Property option again, kept alongside the resolveLiteral literal-storage optimization added on this branch. - decorators / types: resolveLanguage?: string re-added; Property() defaults it to "literal", resolveLiteral to true. - sdna: emits property_resolve_language with the actual language again. - shacl-gen / SHACLShape / json-schema: carry + round-trip both fields (ad4m://resolveLanguage and ad4m://resolveLiteral links). - Ad4mModel write path + query builders (query-sparql, query-utils, PerspectiveProxy): unified precedence — custom language -> createExpression on that language; literal + resolveLiteral:false -> createExpression on literal; else deterministic literal: IRI. Custom languages are never treated as deterministic-literal storage in WHERE building. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…empty-string - Rust: struct-literal test fixtures carry both fields; shape.rs preserves an explicit empty resolve_language verbatim (matching dev) rather than coercing to None. New round_trip tests for resolve_literal true/false and a custom resolve_language, asserting is_deterministic_literal() classification. - integration test exercises the non-literal FILTER path via a custom resolveLanguage on the target shape. - TS: SHACLShape round-trip tests for resolveLanguage (toJSON/fromJSON and toLinks/fromLinks) and the resolveLanguage+resolveLiteral pairing. Full Rust suite (640) and core TS suite (260) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The rebase onto dev brought in test code that references
ShapeProperty.resolve_language, which was removed by this branch
(replaced with resolve_literal). Removed all field references from
integration_tests.rs, test_helpers.rs, and round_trip_tests.rs.
Updated the round-trip test to assert resolve_literal: Some(true)
instead of resolve_language: Some("literal"), matching this branch's
backward-compat mapping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… deterministic literals
Two read/write-path gaps surfaced by the resolveLanguage+resolveLiteral
integration tests:
1. resolve_property_value (write, createSubject / MCP path) only encoded
String and Number for deterministic-literal properties; Bool, Object and
Array fell through to `value.to_string()` and were stored as opaque raw
targets (`false`, `{...}`) instead of `literal:boolean:` / `literal:json:`
IRIs. The typed-literal storage layer then kept them as NamedNodes, so
reads returned strings and indexed WHERE filters never matched. Encode
Bool as `literal:boolean:{b}` and Object/Array via Literal::from_json,
matching the TS `valueToLiteralIri` encoding.
2. resolve_language_transforms skipped deterministic-literal properties, so a
property with `resolveLanguage: "literal"` plus a transform never had its
transform applied on read. Include properties that carry a transform, and
use the already-decoded hydrated value as the transform focus (never
re-interpreting it as an expression URL).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…th-resolve-literal' into feat/restore-resolve-language-with-resolve-literal
…rebase" This reverts commit 61f48af.
…h-resolve-literal Restore resolveLanguage alongside resolveLiteral
…s first-class After restoring resolveLanguage alongside resolveLiteral, two comments still framed behavior as resolveLiteral-only: - SHACLShape transform doc said transforms are "only used for properties with resolveLiteral set". Transforms are a resolveLanguage feature — applied to values resolved through a custom language or the literal language with resolveLiteral: false, plus any property that declares a transform. - mcp resolve_property_value doc only mentioned the resolveLiteral setting; it respects both resolveLanguage and resolveLiteral. Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…l examples The class-level example now demonstrates resolveLanguage (the value-resolution selector: "literal" or a custom language address) alongside resolveLiteral (the literal-language storage optimization), including a custom-language property where resolveLiteral does not apply. Reflects that resolveLanguage is first-class and resolveLiteral is additive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eral:false props migrate_signed_envelopes_to_plain_literals flattened every literal:json signed envelope to a plain literal, regardless of the property's resolution settings. That destroys the expression-level signature for properties that explicitly keep signed-expression literals (resolveLiteral: false, or a custom resolveLanguage) — provenance that is distinct from and not recoverable via the link-level reifier proof. Build a "keep-envelope" predicate set from the SHACL property shapes (resolveLiteral: false or a non-"literal" resolveLanguage) and skip those predicates during flattening. Deterministic-literal (default) properties are still normalized to typed literals so indexed WHERE equality keeps working on upgraded perspectives. If the SHACL lookup yields nothing, every envelope is flattened as before — no regression. Adds test_migration_v3_preserves_envelopes_for_resolve_literal_false. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nto signed envelopes
Reworks the storage-mode semantics so resolveLiteral is a purely additive
performance optimization and resolveLanguage keeps its dev meaning, instead of
the default silently flipping literal properties from signed envelopes to
deterministic literals.
Effective storage mode (shared TS effectiveLiteralStorage / Rust
is_deterministic_literal), from the property's explicitly-set options:
- neither set → deterministic literal (the perf default)
- resolveLiteral: true → deterministic literal (explicit opt-in)
- resolveLiteral: false → signed literal envelope
- resolveLanguage: "literal" → signed literal envelope (provenance in value)
- resolveLanguage: <custom> → expression on that language
resolveLiteral (when set) wins over the language-implied default.
Neither option is defaulted at the decorator / json-schema layer anymore, so
"explicitly literal" (envelope) is distinguishable from "unspecified"
(deterministic). SHACL emits only what was set; readers no longer derive
resolveLiteral from resolveLanguage.
Flux opts in per-property: channels omit resolveLanguage (deterministic,
index-friendly), messages keep resolveLanguage:"literal" for signed-expression
provenance distinct from the link-level reifier proof.
Query path:
- WHERE/projection builders use parse_literal to compare on the decoded value
for non-deterministic (envelope/custom) properties, while deterministic
properties keep indexed typed-literal matching. Fixes value-WHERE on
envelope properties and removes an invalid relative-IRI match for
non-absolute string values.
- mention detection unwraps the envelope (parse_literal) before substring
matching, so author/proof DIDs in an envelope no longer produce false
mentions.
- v3 envelope-flattening migration keeps explicit resolveLanguage:"literal"
envelopes (alongside resolveLiteral:false and custom languages).
Tests: prolog 75, model 149, mcp-http 67 green; new coverage — a Flux message
envelope round-trip (prolog), message-body provenance (mcp-http), and
is_deterministic_literal round-trip cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eterministic default) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Important
Description updated after merging #865. The original description (preserved below, struck through) framed this as
resolveLanguagebeing replaced byresolveLiteral. That is no longer the design:resolveLanguageis retained as a first-class option and behaves exactly as before.resolveLiteralis an additive per-property optimization. Corrected summary follows.Summary
Replaces the NamedNode-IRI storage model for property values with native typed RDF literals, so Oxigraph's POS index can serve equality/comparison WHERE filters directly instead of scanning through a custom SPARQL function. Adds
resolveLiteralas a literal-storage optimization alongside the existingresolveLanguageselector.Typed RDF literal storage (Rust executor)
Wire-format
literal:string:X/:number:N/:boolean:B/:json:Jtargets are now stored as typed RDF terms ("X"^^xsd:string,"42"^^xsd:integer,"true"^^xsd:boolean,"…"^^ad4m://json) rather than as NamedNode IRIs. This is the core change — it means the SPARQL engine can evaluate?source <pred> "active"^^xsd:string .as an index lookup instead of binding every row and filtering withfn/parse_literal.Key files:
sparql_store.rs—target_to_storage_term()/storage_term_to_target_string()handle the wire↔storage translation;make_direct_triple()returns aTerminstead of aNamedNode.Index-friendly WHERE builders
Both the root model query builder (
sparql_builder.rs) and the projection WHERE builder (projection.rs) now emit typed-literal terms directly:VALUES ?x { "val"^^xsd:string }(withUNION <val>fallback when the value is IRI-shaped)?source <pred> "true"^^xsd:boolean ."42"^^xsd:integer .or"3.14"^^xsd:decimal .VALUES ?x { "a"^^xsd:string "b"^^xsd:string }Deterministic properties match against typed literals (indexed); envelope / custom-
resolveLanguageproperties are matched on the decoded value viafn/parse_literal(correct, not index-served). The build-timeis_deterministic_literal()decides which.resolveLanguage(unchanged) +resolveLiteral(additive, opt-in) — TS + RustNeither option is defaulted at the decorator / json-schema layer. The effective storage mode is derived from what a property explicitly sets (shared TS
effectiveLiteralStorage/ Rustis_deterministic_literal):resolveLiteral: trueresolveLiteral: falseresolveLanguage: "literal"resolveLanguage: <custom>expression_createon that languageresolveLiteral(when set) wins over the language-implied default; it is ignored for a customresolveLanguage.resolveLanguagekeeps itsdevmeaning — naming the literal language produces a signed expression ({author, timestamp, data, proof}). The only behavior change vsdev: a property with no resolve options now stores a deterministic literal (perf) instead of an envelope.SHACL emits only what was explicitly set; readers no longer derive
resolveLiteralfromresolveLanguage. No breaking change to existingresolveLanguageuser code. Flux opts in per-property: channels omitresolveLanguage(deterministic, indexed; provenance on the reifier), messages keepresolveLanguage:"literal"(signed-expression provenance in the value).A property's
transformis applied on read to expression-resolved values (custom language, orresolveLanguage:"literal"/resolveLiteral:false) and to any property that declares a transform.Write path
literal_decodein Rust no longer wraps primitives in synthetic{author, timestamp, data, proof}envelopes — returns the decoded value directly.Ad4mModel.setProperty/ Rustresolve_property_value: deterministicliteral:IRIs for the deterministic mode (booleans/objects encode asliteral:boolean:/literal:json:);expression_create("literal", …)for the envelope mode;expression_create(lang, …)for a custom language.fn/parse_literalbefore substring matching, so author/proof DIDs inside an envelope don't produce false mentions.Boot-time migrations
Two versioned migrations run at perspective initialization:
migrate_signed_envelopes_to_plain_literals(): rewritesliteral:json:<envelope>NamedNode targets to the plainliteral:<kind>:<data>form of their inner.datavalue, landing directly on typed-literal storage in a single pass. Shape-aware: predicates whose property is envelope-mode (resolveLiteral:false, explicitresolveLanguage:"literal", or a custom language) are skipped, so signed-expression provenance is preservedmigrate_iri_literals_to_typed_literals(): rewrites any remainingliteral:*:NamedNode IRIs to their corresponding typed RDF literalsBoth are idempotent (version-gated) and preserve reifier metadata. v3 failure blocks perspective init to avoid mixing legacy and migrated targets.
Instance ID scheme change
Auto-generated
baseExpressionvalues now usead4m://obj/<id>instead ofLiteral.from(randomId).toUrl(), since the typed-literal storage layer would stripliteral:string:<id>targets to"<id>"^^xsd:string— which can't be a triple subject. Thead4m://obj/scheme keeps instance IDs unambiguously as NamedNode IRIs.Bool/JSON write encoding + transform-on-read fixes (#865)
resolve_property_value(createSubject / MCP write path) encodesBoolasliteral:boolean:andObject/Arrayasliteral:json:, matching the TSvalueToLiteralIriencoding (previously these fell through to a rawvalue.to_string()target).resolve_language_transformsapplies a property's transform to deterministic-literal properties that declare one, not only to expression-resolved properties.Test plan
mcp-http,prolog-and-literals, model) — green locally and in CIresolveLanguagetriples loads correctlyresolveLanguage(literal + custom) coexists withresolveLiteral; empty-stringresolveLanguagepreserved🤖 Generated with Claude Code
Summary
Replaces the NamedNode-IRI storage model for property values with native typed RDF literals,
and consolidates the decorator API from. Together these changes let Oxigraph's POS index serve equality/comparison WHERE filters directly instead of scanning through a custom SPARQL function.resolveLanguage: stringtoresolveLiteral: booleanTyped RDF literal storage (Rust executor)
Wire-format
literal:string:X/:number:N/:boolean:B/:json:Jtargets are now stored as typed RDF terms ("X"^^xsd:string,"42"^^xsd:integer,"true"^^xsd:boolean,"…"^^ad4m://json) rather than as NamedNode IRIs. This is the core change — it means the SPARQL engine can evaluate?source <pred> "active"^^xsd:string .as an index lookup instead of binding every row and filtering withfn/parse_literal.Key files:
sparql_store.rs—target_to_storage_term()/storage_term_to_target_string()handle the wire↔storage translation;make_direct_triple()returns aTerminstead of aNamedNode.Index-friendly WHERE builders
Both the root model query builder (
sparql_builder.rs) and the projection WHERE builder (projection.rs) now emit typed-literal terms directly:VALUES ?x { "val"^^xsd:string }(withUNION <val>fallback when the value is IRI-shaped)?source <pred> "true"^^xsd:boolean ."42"^^xsd:integer .or"3.14"^^xsd:decimal .VALUES ?x { "a"^^xsd:string "b"^^xsd:string }The
fn/parse_literalcustom function is preserved as a back-compat shim for legacy SDNA queries and Flux mention detection, but the model query engine no longer depends on it.Signed-envelope removal from write path
literal_decodein Rust no longer wraps primitives in synthetic{author, timestamp, data, proof}envelopes — returns the decoded value directlyAd4mModel.setPropertyusesvalueToLiteralIri()(deterministic) forresolveLiteral: true(default), orcreateExpression("literal", ...)forresolveLiteral: falseSTR(?target)directly instead offn/parse_literalBoot-time migrations
Two versioned migrations run at perspective initialization:
migrate_signed_envelopes_to_plain_literals(): rewritesliteral:json:<envelope>NamedNode targets to the plainliteral:<kind>:<data>form of their inner.datavalue, landing directly on typed-literal storage in a single passmigrate_iri_literals_to_typed_literals(): rewrites any remainingliteral:*:NamedNode IRIs to their corresponding typed RDF literalsBoth are idempotent (version-gated) and preserve reifier metadata. v3 failure blocks perspective init to avoid mixing legacy and migrated targets.
— SUPERSEDED: see "resolveLanguage (unchanged) + resolveLiteral (new, additive)" aboveresolveLiteralreplacesresolveLanguage(TS + Rust)(both options exist;PropertyOptions.resolveLanguage?: string→PropertyOptions.resolveLiteral?: booleanresolveLanguageretained)resolveLiteral: true(default) — deterministicliteral:*:IRIs, no expression language involvedresolveLiteral: false— routes throughexpression_create("literal", ...)for signed-envelope storagead4m://resolveLiteral → literal:true|false(new) with backward compat readingad4m://resolveLanguage → literal:string:literalasresolveLiteral: truein both TS (SHACLShape.fromLinks) and Rust (shape.rs,shacl.rs,perspective_instance.rs)Instance ID scheme change
Auto-generated
baseExpressionvalues now usead4m://obj/<id>instead ofLiteral.from(randomId).toUrl(), since the typed-literal storage layer would stripliteral:string:<id>targets to"<id>"^^xsd:string— which can't be a triple subject. Thead4m://obj/scheme keeps instance IDs unambiguously as NamedNode IRIs.Other changes
Literal.ts: addedboolean:prefix parsingparseLit(SparqlBindings.ts): simplified — JSON objects are JSON-stringified directly instead of extracting.data(envelope unwrapping removed)validate_iri(utils.rs): now rejects control and whitespace characters(json-schema now defaults bothjson-schema.ts: simplified — removed cascadingresolveLanguagedefaulting logic, replaced with singleresolveLiteral ?? trueresolveLanguage: "literal"andresolveLiteral: true)sdna.ts:resolveLiteral !== falseemitsproperty_resolve_language(uuid, prop, "literal")for Prolog compatLiteral.from("text").toUrl()toad4m://test/<name>scheme; assertions updated fromliteral.datatoliteral(no envelope unwrapping)Files changed (34 files, +1918/-563)
TypeScript core (16 files):
Literal.ts,Ad4mModel.ts,decorators.ts,types.ts,SHACLShape.ts,shacl-gen.ts,query-sparql.ts,query-utils.ts,json-schema.ts,sdna.ts,PerspectiveProxy.ts,SparqlBindings.ts,shacl/index.ts,Ad4mModel.test.ts,query-sparql.test.ts,shacl-roundtrip.test.tsRust executor (15 files):
sparql_store.rs(typed-literal storage + migrations),sparql_builder.rs+projection.rs(indexed WHERE),perspective_instance.rs(resolveLiteral),shape.rs+types.rs(resolve_literal),query.rs,utils.rs,shacl_parser.rs,mcp/shacl.rs,mcp/tools/mod.rs+subscriptions.rs,languages/literal.rs,perspectives/mod.rs+migration.rs,integration_tests.rsIntegration tests (2 files):
mcp-http.test.ts,prolog-and-literals.test.tsTest plan
mcp-http,prolog-and-literals) via CircleCIresolveLanguagetriples loads correctly🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
literal:boolean:URLs.Breaking Changes
Property metadata now uses(superseded — both options exist;resolveLiteral: booleaninstead ofresolveLanguage: string.resolveLanguageretained) Database migrations automatically convert existing literal storage to the new typed format.Improvements