feat(notion-md): v-next frictionless, progressively-disclosed sync (#774)

CHANGELOG.md

@@ -6,6 +6,8 @@ All notable changes to this project will be documented in this file.
 
 ### Changed
 
+- **@overeng/notion-md**: Route datasource-scoped property writes through the shared `@overeng/notion-property-write` core via a new standalone live proof provider (`makeStandaloneLiveProof`). For a page whose parent is a Notion data source, each writable property is evaluated against a freshly re-read data-source schema + live page (stable property identity, write class, base completeness, relation availability) before the write proceeds; green paths evaluate to `allowed()` so existing behavior is unchanged, blocked verdicts surface as `NmdPropertyWriteBlockedError` instead of a silent property update, and a `source: remote` page refuses local property mutation as drift. The gateway gains a `retrieveDataSource` operation; standalone (non-datasource) pages keep their current path untouched. Full live schema-drift / relation-completeness coverage (Phase 8) is marked with `TODO(phase-8-live-l6)`.
+
 - **@overeng/utils-dev/otelite**: Resolve the `otelite` binary from `OTELITE_BIN` before falling back to `PATH`, and document the plain-shell Nix workflow for focused wrapper tests.
 
 - **@overeng/otel-contract**: Add branded/refined OTEL name schemas (`OtelAttributeKey`, `OtelSpanName`, `OtelMetricName`, `OtelServiceName`), validate contract names/keys at definition time, add an Effect `Metric` runtime bridge for schema-first metric contracts, and extend the raw-OTEL lint rule to ban raw Effect `Metric.*` APIs outside approved contract/test boundaries.
@@ -16,6 +18,16 @@ All notable changes to this project will be documented in this file.
 
 - **CI / Nix packages**: Refresh the stale `genie`, `megarepo`, `notion-md`, and `tui-stories` pnpm fixed-output hashes after the schema-first OTEL contract change updated the workspace dependency closure.
 
+- **@overeng/notion-md**: Move the v-next public CLI to `track` / `status` / `sync`: `track` is now the only page-id bootstrap command, `sync`/`status` operate on local self-describing files, write-capable paths support `--dry-run`, and `sync --watch` routes through the same source-aware reconcile engine as one-shot sync.
+
+- **@overeng/notion-md**: Add explicit v-next destructive sync modes for unsupported-block deletion (`--allow-delete-unknown-blocks`) and literal Roughdraft markup writes (`--allow-review-markup`), validate referenced object-store payloads on v-next reads, and add `sync --gc-objects` with dry-run planning for unreachable content-addressed objects.
+
+- **@overeng/notion-md**: Complete the v-next source-dispatched sync contract by requiring explicit `.nmd` `source`, removing legacy sync helpers from the public package surface, preserving watch as a first-class reconcile path, adding live watch and OTEL span verification, and aligning VRS/user docs with the stateless Mirror Sync vs base-backed Shared Sync split.
+
+- **@overeng/notion-md**: Refresh the fidelity corpus from live Notion through repeatable capture tooling, add live corpus verification, and fold Notion's lossless code-fence language alias expansion (`js`/`ts` → `javascript`/`typescript`) into the semantic-equivalence oracle while keeping JavaScript and TypeScript fences distinct.
+
+- **@overeng/notion-md**: Add schema-decoded Notion webhook trigger ingestion for watch mode. Page webhook payloads normalize to secret-safe trigger signals and feed the existing batch watch queue as `webhook` reasons; comment events are decoded and classified as an explicit non-body boundary until comments API/client support exists.
+
 - **CI / Nix packages**: Refresh the stale `workflow-report` pnpm fixed-output hash so the Storybook preview reporting step can build `#workflow-report` again after the branch rebase updated the workspace dependency closure.
 
 - **@overeng/restate-effect**: Made `Restate.run`'s type HONEST. A durable `ctx.run` step carries NO catchable typed failure: the inner effect runs via `Runtime.runPromise` inside `ctx.run`, so a typed `Effect.fail` only REJECTS the step (Restate retries; a give-up maps to a `RestateError` DEFECT) and never reaches the outer failure channel — the old `run<A, E, R>(…): Effect<A, E, …>` advertised a typed `E` that `catchTag`/`catchAll` would typecheck against but that could never fire. `run` is now `run<A, R>(name, effect: Effect<A, never, R>, options?): Effect<A, never, …>`, and `runExit` is `runExit<A, R>(…): Effect<Exit<A>, never, …>` — the honest OBSERVATION form, whose failure channel is `never` (an observed failure is a defect/interrupt `Cause`, not a phantom typed `E`). Domain errors now belong in the HANDLER body (classify the step's result there) or are encoded as VALUES inside the step; to force a durable retry, DIE inside the step. A passed typed-`E` inner effect is now a COMPILE error (negative-type assertion in `capability-inference.types.ts`). Callers reconciled: the saga integration test's failing `pay` step `Effect.die`s (was `Effect.fail`), and `examples/12-self-reschedule.ts`'s `pollComposedSource` returns a tagged VALUE with `E = never` (classified in the cycle body, unchanged). `examples/14-http-error-classification.ts` already used the die-the-step / classify-in-body strategies; only its prose was corrected. VRS: decision 0003 (#4 — corrects the earlier "keep the inner `E` flowing through `run`"), 03-effect-runtime / 04-error-boundary specs, the guide handbook, and a DEFERRED typed-failure-transport `run` note (an encoded `fail(E)` journaled via an error schema). No dependency changes.
@@ -25,11 +37,15 @@ All notable changes to this project will be documented in this file.
 
 ### Fixed
 
+- **@overeng/notion-md**: Make the SM6.2 comment-write boundary mutation-implying instead of presence-based. The guard previously blocked any push/pull/shared write to a page that merely HAS comments — including a body-only edit — but `updateMarkdown({_tag:'replace_content'})` writes only body content and is structurally incapable of creating/editing/resolving a Notion comment, so that block was fictitious and over-blocked legitimate body edits on commented pages. `classifyCommentWrite` now compares the comment inventory the write would PRODUCE against the current inventory and blocks only on a genuine add/remove/modify; for today's body-only write paths `produced === current`, so a body edit on a commented page proceeds (`pushed`/`pulled`/`shared-merged`) and the named `CommentWriteUnsupported` guard is a dormant fail-closed gate that trips only when a real comment-mutation path is ever wired. The `shared` comment guard moved from the top of `reconcileSharedFile` into the `merge`/`force` write branches (symmetric to how single-source guards scope to push/pull), so the `noop` shared branch no longer evaluates it. Block-detection coverage lives at the classifier-unit level (the dormant block path is not reachable end-to-end today); the `NmdNonBodyWriteBlockedError.fileIds` JSDoc now documents that the field carries the offending unit ids — file or comment — discriminated by `guard`.
+
 - **@overeng/otelite**: Honor durability-before-ack — flush each export to the kernel before the 200/OK. `tokio::fs::File` buffers writes, so `write_all` alone did NOT guarantee the bytes reached the kernel before the sink acked; an independent reader (or a crash) before the next flush could miss them, contradicting R05 ("flush … before acking") and the `append_line` doc's own "durably reaching the kernel before returning" promise. This surfaced as a CI flake in the `durable_before_ack` gate (a read immediately after the 200 occasionally saw an empty file under thread contention — reproduced ~1/60 at 16 test threads). Fix: `SignalFile::append_line` / `append_json` now `flush()` after `write_all`, before returning. This is a flush, not an fsync — `sync_all` (physical-disk durability) stays deferred to shutdown, so the M2 "no per-export fsync under the lock" throughput decision is preserved. Verified: 0 failures over 200 × 16-thread runs (was ~1/60).
 - **@overeng/otelite**: Make the HTTP-JSON metrics receive path lossless, fixing two silent data-loss bugs a stress test surfaced. The upstream `opentelemetry-proto` `with-serde` deserialize — which the receiver used to BUILD the proto value the sink then re-serialized — silently drops several metric JSON shapes: a `sum`/`gauge` `NumberDataPoint` whose int64 value is the default string form (`"asInt":"7"`) lost its value entirely (captured null), and a regular `histogram` metric was dropped down to `{name,description,unit,metadata}` (its data oneof gone). Both returned HTTP 200 + bumped `counts.metrics` → a silent mis-capture that violates the lossless + "loud, never silent" contracts (decisions/0011). Fix: on the JSON metrics path, `with-serde` still runs purely as the dialect VALIDATOR (Err → 400 + `note_rejected`, gate unchanged), but on success the receiver now persists the VALIDATED RAW JSON body verbatim (re-emitted through `serde_json::Value` via the new `Sink::write_metrics_json`, counting metrics from the JSON structure) instead of the lossy proto re-serialization. Since the body is already canonical OTLP/JSON and `inspect` walks raw JSON, the JSON metrics path is now lossless for string-int64 sums/gauges, regular histograms, AND exponential histograms — the last also RESOLVING the previously-documented exp-histogram-on-JSON limitation for the receive path. Traces/logs JSON paths and all protobuf/gRPC paths are unchanged (already lossless). New gates (real receiver, no mocks): an HTTP-JSON round-trip of a string-int64 sum + histogram + exponential histogram all survive receive → capture → `inspect`; cross-transport equivalence extended to metrics (the same logical string-int64-sum + histogram over HTTP-JSON vs HTTP-protobuf vs gRPC flattens to equivalent `inspect` rows, the proto/gRPC fixture built natively to avoid the lossy `with-serde` source); and a loud-rejection guard that a malformed metrics JSON body still 400s + is captured nowhere. KNOWN RESIDUAL: the upstream metrics `with-serde` is more lenient than the trace one, so for metrics the JSON dialect gate is effectively structural (malformed JSON / hard field-type mismatches), tolerating some non-default dialect shapes (numeric int64 nanos, string enums) rather than rejecting them loudly — a stricter metrics dialect gate is a follow-up (#769, #772).
 
 ### Added
 
+- **@overeng/notion-effect-client / @overeng/notion-md**: Add optional `property_descriptors` field to `NmdFrontmatterV2` — compact, non-authoritative property identity hints (property_id, property_type, data_source_id, config_hash) embedded in `.nmd` frontmatter for datasource page files. Decoded strictly (unknown fields rejected, R13); field is absent for standalone pages and omitted from encoded output when not set (R10). `buildFrontmatterV2` in `sync.ts` gates descriptor emission on datasource parent presence. Standalone validity enforced: a descriptor-bearing `.nmd` round-trips via the standard `notion-md` parse path (R03). `docs/file-format.md` updated with a Property Descriptors section.
+
 - **@overeng/otel-contract**: Add the schema-first OTEL operation and metric-contract DSL (`OtelOperation`, `OtelMetric`, `OtelSpan.withStream`, attribute builders, compiled metadata, `encodeSync`, and checked dynamic span-map annotations) and migrate product instrumentation across the repo off raw `Effect.withSpan` / `Stream.withSpan` / `Effect.annotateCurrentSpan` and normal `unsafe*` contract calls. The contract remains runtime-light: it owns schema-backed names, labels, attributes, cardinality metadata, and encoders, while package-local code keeps exporter/provider setup, service identity, Restate replay gates, and runtime-specific bridges. `@overeng/oxc-config` now ships `overeng/no-raw-otel-primitives` with generated rollout config, `@overeng/utils-dev/otelite` gains reusable metric/log expectation helpers, and `restate-effect` adopts the same idiom for internal spans while preserving hook-owned Restate spans and replay-aware metrics.
 
 - **@overeng/utils/node/otel-attrs**: Add schema-first OTEL attribute and span contracts (`OtelAttr`, `OtelAttrs`, `OtelSpan`) plus otelite expectation helpers that derive span assertions from the same compiled attribute encoders used by runtime instrumentation. Ambiguous encodings fail closed unless explicitly annotated, redacted values only support redacted/drop policies, and span definitions require the dedicated `OtelAttr.spanLabel()` contract.

context/notion-db-markdown-sync/decisions/0001-sqlite-control-plane-markdown-projection.md

@@ -0,0 +1,27 @@
+# Keep one control plane and two minimal user surfaces
+
+Status: accepted
+
+Datasource-sync should expose the smallest useful end-user surface: one SQLite
+data file for tabular/scriptable workflows and one NotionMD `.nmd` page file
+per Notion page for editor workflows. Hidden implementation state may use SQLite,
+sidecars, object stores, leases, base hashes, and own-write tokens, but those
+artifacts are not user API. Markdown page files should feed the same control
+plane and planner as SQLite edits instead of becoming a second sync engine or a
+NotionMD-owned tree feature.
+
+## Considered Options
+
+| Option                                           | Result           | Reason                                                                                                                  |
+| ------------------------------------------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| LocalWorkspacePort backend                       | Rejected for now | The current port is body/materialization shaped, not data-source-page shaped.                                           |
+| Projection/intent adapter over one control plane | Recommended      | Reuses the existing planner, guards, event store, outbox, and watch model while keeping user-visible artifacts minimal. |
+| NotionMD tree feature                            | Rejected         | NotionMD owns page bodies, not data-source schema, page membership, lifecycle, or query completeness.                   |
+| Separate package/CLI                             | Rejected for v1  | A separate user-facing entrypoint would expand the surface before the datasource-sync contract is stable.               |
+
+## Consequences
+
+The default workspace should not expose redundant body files, user-editable
+sidecars, or visible machine metadata. If extra artifacts are needed for safety
+or performance, they live under hidden implementation directories and must be
+rebuildable or repairable from the control plane.

context/notion-db-markdown-sync/decisions/0002-public-data-file-hidden-control-plane.md

@@ -0,0 +1,15 @@
+# Keep the public SQLite data file separate from the hidden control plane
+
+Status: accepted
+
+The user-facing SQLite surface should be `data/v1/<source>.sqlite`, containing only stable
+public tables/views. Private event logs, outbox state, leases, base hashes,
+checkpoints, object state, repair metadata, and implementation projections live
+under hidden `.notion/v1/` implementation state instead of private `_nds_*` tables
+inside the public data file.
+
+This gives users and agents a cleaner rule: data files and `pages/v1/**/*.nmd` are
+the intended surfaces; `.notion/v1/**` is tool state. It sacrifices the earlier
+self-contained single-SQLite-file replica property, but removes a major
+footgun and lets implementation storage evolve without changing the public SQL
+API.

context/notion-db-markdown-sync/decisions/0003-workspace-authority-mode.md

@@ -0,0 +1,24 @@
+# Use one workspace authority mode for both local surfaces
+
+Status: accepted
+
+Datasource workspaces should reuse the NotionMD `source` vocabulary:
+`local`, `remote`, and `shared`. The mode is declared at workspace level and is
+inherited by both data files and `pages/v1/**/*.nmd`; the two user surfaces must not
+declare independent conflicting authority modes.
+
+`remote` is a Notion-authoritative mirror/export mode. `local` is a local
+workspace-authoritative apply mode. `shared` is the bidirectional authoring mode
+and the only mode that promises base-anchored concurrent-edit detection and
+conflict refusal.
+
+## Consequences
+
+Hidden `.notion/v1/` state is required when the system promises durable shared-sync
+behavior: bases, accepted intents, outbox, conflicts, leases, checkpoints,
+tombstones, path claims, own-write suppression, repair, first-class writable
+watch, or destructive-action proof. In single-source mirror modes, hidden state
+may be used as cache/checkpoint material, but deleting it must not change the
+authority contract or correctness; it may only make sync slower or require
+re-observation. Local create/retry safety may still require minimal idempotency
+state.

context/notion-db-markdown-sync/decisions/0004-local-surface-convergence.md

@@ -0,0 +1,21 @@
+# Converge local surfaces before remote planning
+
+Status: accepted
+
+Data files and `pages/v1/**/*.nmd` are both user surfaces, but they must not become
+competing local authorities. Before sync plans remote writes, it must decode
+both surfaces, map facts to stable page/property/body/lifecycle identities,
+coalesce identical desired states, and raise local conflicts for divergent
+desired states.
+
+Local conflicts block remote mutation. Remote planning starts only after there
+is one unambiguous local desired state for each affected surface.
+
+## Considered Options
+
+| Option                                    | Result          | Reason                                                                                            |
+| ----------------------------------------- | --------------- | ------------------------------------------------------------------------------------------------- |
+| Partition writable facts by surface       | Rejected        | Too restrictive; Markdown property edits would stop composing with standalone NotionMD.           |
+| Let one surface win locally               | Rejected        | Creates hidden last-writer-wins and makes user consequences depend on scan order.                 |
+| Workspace chooses one active edit surface | Rejected for v1 | Adds another mode axis and weakens the simple default.                                            |
+| Mandatory local convergence               | Recommended     | Keeps both surfaces real while preserving one unambiguous local desired state before remote sync. |

context/notion-db-markdown-sync/decisions/0005-conflicts-live-in-status-surfaces.md

@@ -0,0 +1,13 @@
+# Surface conflicts through status, not generated page files
+
+Status: accepted
+
+Datasource conflicts, including local-surface conflicts between data files
+and `pages/v1/**/*.nmd`, should be canonical in the data file and CLI/status output.
+The default workspace should not generate page-adjacent conflict files such as
+`pages/foo.conflict.nmd`, because those files expand the user-visible surface and
+can be mistaken for editable source artifacts.
+
+Standalone NotionMD may still create body-specific roughdraft/conflict artifacts
+when the body merge workflow needs an editable conflict artifact. That is a
+body-specific exception, not the generic datasource conflict model.

context/notion-db-markdown-sync/decisions/0006-single-source-mirror-modes.md

@@ -0,0 +1,20 @@
+# Single-source modes mirror the declared authority
+
+Status: accepted
+
+Datasource workspaces should follow the same single-source rule as NotionMD:
+`remote` means Notion wins and local drift is overwritten; `local` means the
+workspace wins and remote drift is overwritten. `status` and `sync --dry-run`
+must report the consequence before mutation, but concurrent-edit detection is
+not promised in single-source modes.
+
+Bidirectional safety belongs to `shared`. That is the mode that requires durable
+bases, accepted intents, outbox, conflict records, leases, checkpoints, and
+repair state under `.notion/v1/`.
+
+## Consequences
+
+In `remote` mode, data files and `pages/v1/**/*.nmd` are generated mirror outputs.
+Hidden `.notion/v1/` state may optimize incremental pulls, but deleting it must not
+change correctness. In `local` mode, hidden state is mostly optional except for
+minimal idempotency/retry state needed for safe local-created pages.