Support forward-compatible read-side validation for wire protocols

[turadg]

Description

Endo Patterns currently describe the shape that is valid "now". This works well for authority boundaries and exact validation, but creates friction for wire protocols and published data streams that need forward compatibility.

Consider a published status object:

 const StatusShape = M.splitRecord(
   {
     phase: M.string(),
   },
   {
     progress: M.number(),
   },
 ); 

A publisher validates against StatusShape before writing to vstorage.

Later, the publisher evolves to include a new field:

 {
   phase: 'running',
   progress: 0.5,
   estimatedCompletion: 12345,
 } 

An older client using the same pattern to validate reads may reject the object, even though it understands all fields it depends on.

For wire protocols, producers and consumers often have different compatibility requirements:

• Producers should emit the canonical schema.
• Consumers should accept width-compatible extensions and ignore unknown fields.

Many schema systems support this distinction through "open" vs "closed" object validation.

Proposal

Provide a way to derive or express a read-side pattern that:

• Requires all fields described by the pattern.
• Continues validating optional fields described by the pattern.
• Ignores additional unknown fields.

Possible APIs:

 allowExtraProperties(StatusShape) 

or

  M.openRecord(...) 

or another mechanism consistent with the existing Pattern design.

The goal is to support forward-compatible readers for wire protocols, vstorage data, durable state snapshots, and similar evolvable data formats without requiring duplicate schema definitions.

Motivation

Patterns currently serve both:

1. Authority/interface validation.
2. Data protocol validation.

The first often benefits from exact validation. The second often benefits from forward-compatible width subtyping.

Providing an explicit read-side validation mode would make protocol evolution easier while preserving strict validation where desired.

[gibson042]

In what sense is this not already satisfied by splitRecord, whose rest argument defaults to a universal acceptor (i.e., splitRecord defaults to "open" validation)?

import { M, matches } from "@endo/patterns";

const required = harden({ phase: M.string() });
const optional = harden({ progress: M.number() });

const specimen = harden({
  phase: 'running',
  progress: 0.5,
  estimatedCompletion: 12345,
});

console.log(matches(
  specimen,
  M.splitRecord(required, optional),
));
// => true

console.log(matches(
  specimen,
  M.splitRecord(required, optional, {}),
));
// => false

[turadg]

@gibson042 that's the perfect solution when the producer is just required keys. When there are no "parts" to coordinate, the consumer writes M.splitRecord(required) and is done. (And to be clear, it composes correctly with optional keys too: opening only loosens undeclared keys, so a declared-but-wrong-typed optional is still rejected rather than slipping into the permissive rest.)

This issue is about adjusting any producer guard to have optional keys. So that guard can be the single source of truth on the producer side:

const producerGuard = M.splitRecord(
  { phase: M.string() },
  { progress: M.number() },
  {}, // exact: validate writes
);

To build the consumer's open guard via the from-parts approach, the consumer needs the same required and optional record literals. So you either export those parts and reassemble in both places, or re-declare them in the consumer. Both duplicate the producer's schema structure into the consumer.

What I'm asking for is a guard→guard transform — consume a splitRecord (optional keys included) and return one that allows arbitrary extra keys, so the producer guard stays the one source of truth:

const consumerGuard = allowExtraProperties(producerGuard);

Or using M,

const consumerGuard = M.allowExtra(producerGuard);

Separately: I'd argue for a distinct name rather than leaning on splitRecord's defaulting. Empirically people misremember which arg-count is open vs. closed; a named transform with one obvious meaning ("derive a forward-compatible reader") avoids that footgun.