Skip to content

[P1] Adapter.get_schema() Protocol method + schema fingerprinting + drift quarantine #21

@se-jo-ma

Description

@se-jo-ma

Problem

Adapter Protocol (nautilus/adapters/base.py:127-172) defines only connect / execute / close. No get_schema() method, no schema fingerprinting infrastructure, no drift quarantine policy.

Per design §4.6 (Weakness 6 v1 mitigation): if a source's schema changes (columns added, renamed, removed), routing rules / scope constraints / transformation rules that reference those columns break silently.

Design references

  • design-docs/05-ecosystem-roadmap.md:233-285 — full spec including source_schema CLIPS template + quarantine-drifted-source rule example
  • design-docs/05-ecosystem-roadmap.md:411-412 — v1 roadmap items: "Adapter get_schema() method and schema fingerprinting" + "Schema drift detection and source quarantine policy rules"

Code locations to modify

  • nautilus/adapters/base.py:127 — add get_schema() -> SourceSchema to Adapter Protocol
  • nautilus/core/models.py — add SourceSchema model (columns, types, hash)
  • Each adapter — implement get_schema() querying upstream system metadata
  • nautilus/core/broker.py — schedule periodic schema check (configurable interval, default hourly), assert source_schema fact into CLIPS
  • nautilus/rules/ — ship quarantine-drifted-source policy rule
  • nautilus.yaml schema — add schema_check_interval per source

Acceptance

  • Each adapter exposes get_schema() returning canonical column list + types.
  • Schema hash stored as CLIPS fact; recomputed on configured interval.
  • Hash change asserts schema_drift fact; quarantine rule fires; source excluded from routing until operator ack.
  • Admin UI / CLI surfaces quarantined sources (admin UI work is v2).
  • Integration test: mutate test Postgres schema, assert source quarantined within one check cycle.

Priority

P1 — listed as v1 mitigation in design doc but Protocol surface entirely missing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority: load-bearing v1 gaparea/adaptersAdapter SDK + built-in adaptersenhancementNew feature or requestsize/L<1 week: cross-cutting feature

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions