-
Notifications
You must be signed in to change notification settings - Fork 2
docs: governance page + slickness pass #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| --- | ||
| title: Glossary | ||
| description: The vocabulary of model-ledger — DataNode, Snapshot, ModelRef, Composite, and the rest, in one place. | ||
| --- | ||
|
|
||
| # Glossary | ||
|
|
||
| The whole system is a handful of nouns. (These terms also get hover-definitions | ||
| wherever they appear in the docs.) | ||
|
|
||
| `Backend` | ||
| : Pluggable storage behind the `LedgerBackend` protocol — in-memory, SQLite, JSON | ||
| files, Snowflake, or a remote HTTP service. Swapping it never changes your code. | ||
|
|
||
| `Composite` | ||
| : A governed group whose members are themselves models — a business-level entity (e.g. | ||
| a "Credit Decision System") that rolls up its scorecard, rules, and ETL. See | ||
| [Composites](concepts/composite.md). | ||
|
|
||
| `Connector` | ||
| : A source that emits `DataNode`s from a platform (SQL, REST, GitHub, …) via the | ||
| `SourceConnector` protocol. See [Connectors & discovery](guides/connectors.md). | ||
|
|
||
| `DataNode` | ||
| : The core graph primitive: anything with typed input/output ports — an ML model, a | ||
| heuristic rule, an ETL job, an alert queue. See [DataNode & the graph](concepts/datanode.md). | ||
|
|
||
| `DataPort` | ||
| : A named connection point on a `DataNode`, optionally carrying schema so identically | ||
| named outputs from different models don't falsely link. | ||
|
|
||
| `Dependency graph` | ||
| : The links between nodes, built automatically when an output port name matches an | ||
| input port name (`connect()`). | ||
|
|
||
| `Event log` | ||
| : The inventory itself — an append-only sequence of immutable Snapshots. Nothing is | ||
| overwritten, so history is always reconstructable. | ||
|
|
||
| `ModelRef` | ||
| : A model's stable identity: name, owner, type, risk `tier`, purpose, status. The | ||
| minimum a regulator needs. See [Snapshots & the event log](concepts/snapshot.md). | ||
|
|
||
| `Point-in-time` | ||
| : Reconstruction of the inventory as it stood on any past date, via `inventory_at()`. | ||
|
|
||
| `Profile` | ||
| : A pluggable compliance check (`sr_11_7`, `eu_ai_act`, `nist_ai_rmf`) that validates a | ||
| model's completeness against a framework. See [Governance](governance.md). | ||
|
|
||
| `Snapshot` | ||
| : An immutable, content-addressed record of one thing that happened to a model — a | ||
| registration, a retrain, a validation. The unit of the event log. | ||
|
|
||
| `Tag` | ||
| : A mutable named pointer to a specific Snapshot (e.g. `production`, `latest-validated`). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| --- | ||
| title: Governance | ||
| description: A complete, auditable, point-in-time model inventory — the durable building blocks every model-risk regime asks for, mapped to model-ledger primitives. | ||
| --- | ||
|
|
||
| # Governance | ||
|
|
||
| Model-risk regimes change their names and their numbers. What they *ask for* barely | ||
| changes. Strip away the acronyms and every regime — US banking, EU, insurance — wants | ||
| the same six things from your model inventory. model-ledger is built to produce them as | ||
| a byproduct of normal use, not as a separate compliance chore. | ||
|
|
||
| ## What every regime actually asks for | ||
|
|
||
| | The durable need | What an examiner says | The model-ledger primitive | | ||
| |---|---|---| | ||
| | **Complete inventory** | "Show me *every* model — including the shadow ones." | Cross-platform [discovery & connectors](guides/connectors.md) — ML models, rules, and ETL as one graph | | ||
| | **Risk tiering** | "Which are high-materiality?" | `tier` on every [`ModelRef`](reference/index.md); business systems roll up as [composites](concepts/composite.md) | | ||
| | **Change control + audit trail** | "What changed, when, and who did it?" | Immutable, content-addressed [Snapshots](concepts/snapshot.md) — append-only, tamper-evident | | ||
| | **Dependency & lineage** | "How do these components feed each other?" | The [dependency graph](concepts/datanode.md), built from port matching | | ||
| | **Validation records** | "Prove this was validated, and find what wasn't." | `record_validation()` events live in the same immutable log | | ||
| | **Point-in-time reconstruction** | "Show me the inventory as it stood on December 31." | [`inventory_at(date)`](recipes/point-in-time.md) replays the log | | ||
|
|
||
| That's the whole compliance story: **nothing is overwritten, so the answer to "what was | ||
| true then?" is always reconstructable.** | ||
|
|
||
| ## It falls out of normal use | ||
|
|
||
| ```python | ||
| from model_ledger import Ledger | ||
|
|
||
| ledger = Ledger.from_sqlite("./inventory.db") | ||
|
|
||
| # Identity + risk tier — the minimum a regulator needs | ||
| ledger.register( | ||
| name="credit_scorecard", owner="risk-team", | ||
| model_type="ml_model", tier="high", | ||
| purpose="Consumer credit decisioning", | ||
| ) | ||
|
|
||
| # Validation outcomes are just events in the same immutable log | ||
| ledger.record("credit_scorecard", event="validated", actor="mrm-team", | ||
| payload={"result": "pass", "validator": "second-line"}) | ||
|
|
||
| # The full, ordered, tamper-evident history an examiner can replay | ||
| for snap in ledger.history("credit_scorecard"): | ||
| print(snap.timestamp, snap.event_type, snap.actor) | ||
| ``` | ||
|
|
||
| ## Frameworks it maps to | ||
|
|
||
| The primitives above satisfy the documentation and inventory expectations of the major | ||
| model-risk and AI-governance regimes: | ||
|
|
||
| - **US banking — SR 26‑2 / OCC Bulletin 2026‑13** (the 2026 revision that superseded | ||
| SR 11‑7): tiered model inventory, materiality classification, lifecycle documentation, | ||
| and validation status. | ||
| - **EU AI Act — Annex IV**: version-tracked technical documentation, component | ||
| dependencies, and change history for high-risk systems. | ||
| - **NIST AI RMF** and **ISO/IEC 42001**: inventory, risk management, and lifecycle | ||
| governance practices. | ||
|
|
||
| model-ledger ships **pluggable validation profiles** (`sr_11_7`, `eu_ai_act`, | ||
| `nist_ai_rmf`) that check a model's completeness against a framework, and you can add | ||
| your own — profiles are a plugin layer, not the core. Run them with | ||
| `model-ledger validate --profile <name>` (see the [CLI guide](guides/cli.md)). | ||
|
|
||
| !!! note "Framework-agnostic on purpose" | ||
| model-ledger is a model inventory for *any* organization with deployed models — not | ||
| a single-regulation tool. The frameworks above are examples of what the underlying | ||
| capability is good for; they are a thin, swappable layer over a durable foundation. | ||
| When a regulator renumbers a rule, you update a profile — not your inventory. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| --- | ||
| title: CLI | ||
| description: The model-ledger command line — launch the MCP server or REST API over any backend, and work with a local inventory. | ||
| --- | ||
|
|
||
| # CLI | ||
|
|
||
| Install the CLI extra, then `model-ledger --help` lists everything: | ||
|
|
||
| ```bash | ||
| pip install "model-ledger[cli]" | ||
| model-ledger --help | ||
| ``` | ||
|
|
||
| The CLI has two jobs: **launch the agent and HTTP surfaces** (the bridge to the rest of | ||
| this documentation), and **work with a local inventory** from the terminal. | ||
|
|
||
| ## Launch a surface | ||
|
|
||
| These serve the [Ledger](../reference/index.md) over any [backend](backends.md) — in-memory, | ||
| SQLite, JSON, Snowflake, or a remote HTTP service. | ||
|
|
||
| === "MCP (for agents)" | ||
|
|
||
| ```bash | ||
| model-ledger mcp # in-memory | ||
| model-ledger mcp --demo # sample inventory | ||
| model-ledger mcp --backend sqlite --path ./inv.db # persistent | ||
| model-ledger mcp --backend snowflake --schema DB.MODEL_LEDGER | ||
| model-ledger mcp --backend http --path https://model-ledger.internal:8000 | ||
| ``` | ||
|
|
||
| === "REST API" | ||
|
|
||
| ```bash | ||
| model-ledger serve --demo --port 8000 | ||
| # → OpenAPI docs at http://localhost:8000/docs | ||
| ``` | ||
|
|
||
| `--backend` accepts `memory` · `sqlite` · `json` · `snowflake` · `http`; `--path` is the | ||
| file path (sqlite/json) or URL (http); Snowflake reads credentials from the environment | ||
| (see [Choosing a backend](backends.md)). | ||
|
|
||
| ## Work with a local inventory | ||
|
|
||
| These commands operate on a local file-based inventory (`--db`, default `inventory.db` | ||
| or `$MODEL_LEDGER_DB`) and render as a table or `--format json`. | ||
|
|
||
| | Command | What it does | | ||
| |---|---| | ||
| | `model-ledger list` | List registered models | | ||
| | `model-ledger show <name>` | Show one model's details and versions | | ||
| | `model-ledger validate <name> --profile <p>` | Check a model against a compliance profile (`sr_11_7`, `eu_ai_act`, `nist_ai_rmf`) | | ||
| | `model-ledger audit-log <name>` | Print the model's audit trail | | ||
| | `model-ledger export <name> --output <dir>` | Export an audit pack | | ||
| | `model-ledger introspect <artifact> --allow-pickle` | Extract algorithm/features from a fitted model file | | ||
|
|
||
| ```bash | ||
| model-ledger list --format json | ||
| model-ledger validate credit_scorecard --profile sr_11_7 | ||
| model-ledger audit-log credit_scorecard | ||
| ``` | ||
|
|
||
| !!! info "Which command for which surface" | ||
| `mcp` and `serve` expose the full [event-log Ledger](../concepts/snapshot.md) — the one | ||
| the [SDK](../quickstart.md), [agents](agents.md), and [REST API](backends.md) all share. | ||
| Use them to point Claude or a dashboard at your inventory. The `validate` profiles map | ||
| to the frameworks in the [Governance guide](../governance.md). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| *[DataNode]: The core graph primitive — anything with typed input/output ports (model, rule, ETL, queue). | ||
| *[DataPort]: A named connection point on a DataNode; dependency edges form when port names match. | ||
| *[Snapshot]: An immutable, content-addressed record of one thing that happened to a model. | ||
| *[ModelRef]: A model's stable identity — name, owner, type, risk tier, purpose, status. | ||
| *[Composite]: A governed group whose members are themselves models (e.g. a "Credit Decision System"). | ||
| *[MCP]: Model Context Protocol — the agent-native interface; model-ledger's primary surface. | ||
| *[SR 26-2]: 2026 US interagency model-risk-management guidance (OCC 2026-13), which superseded SR 11-7. | ||
| *[Annex IV]: The EU AI Act's technical-documentation requirements for high-risk AI systems. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| --- | ||
| title: Installation | ||
| description: Install model-ledger and just the extras you need — SDK core is tiny; surfaces and backends are opt-in. | ||
| --- | ||
|
|
||
| # Installation | ||
|
|
||
| model-ledger requires **Python 3.10+**. The core is deliberately tiny (`httpx` + | ||
| `pydantic` only); everything else is an opt-in extra, so you install just the surfaces | ||
| and backends you use. | ||
|
|
||
| ```bash | ||
| pip install model-ledger # core: SDK + dependency graph + connectors | ||
| # or | ||
| uv add model-ledger | ||
| ``` | ||
|
|
||
| ## Extras | ||
|
|
||
| | Install | Adds | For | | ||
| |---|---|---| | ||
| | `model-ledger` | SDK, graph, SQL/REST/GitHub connectors | the core library | | ||
| | `model-ledger[mcp]` | MCP server (`model-ledger mcp`) | AI agents — Claude, Goose, Cursor | | ||
| | `model-ledger[rest-api]` | FastAPI app (`model-ledger serve`) | frontends, dashboards | | ||
| | `model-ledger[cli]` | Typer + Rich CLI | terminal use | | ||
| | `model-ledger[snowflake]` | Snowflake backend | production storage | | ||
| | `model-ledger[introspect-sklearn]` | scikit-learn introspector | extract algorithm/features from fitted models | | ||
| | `model-ledger[introspect-xgboost]` | XGBoost introspector | " | | ||
| | `model-ledger[introspect-lightgbm]` | LightGBM introspector | " | | ||
| | `model-ledger[excel]` | openpyxl | spreadsheet import/export | | ||
| | `model-ledger[all]` | Snowflake + pandas + httpx | the common production set | | ||
|
|
||
| Combine them: `pip install "model-ledger[mcp,rest-api,snowflake]"`. | ||
|
|
||
| ## Which extra for which surface | ||
|
|
||
| - **Python SDK** — core install is enough. | ||
| - **Talk to it from an agent** — `[mcp]`, then `claude mcp add model-ledger -- model-ledger mcp` (see the [Agent guide](guides/agents.md)). | ||
| - **Serve it over HTTP** — `[rest-api]`, then `model-ledger serve` (see [Backends](guides/backends.md)). | ||
| - **From the terminal** — `[cli]` (see the [CLI guide](guides/cli.md)). | ||
|
|
||
| Next: the [60-second quickstart](quickstart.md). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| /* Per-page Markdown actions: Copy / View / Open in Claude. | ||
| * Exposes the raw .md the docs already publish (built by docs_hooks/llms_txt.py), | ||
| * so this site is as consumable by an agent as it is by a human — fitting for a | ||
| * tool whose product is an MCP server. Re-runs on Material's instant navigation. */ | ||
| document$.subscribe(function () { | ||
| var content = document.querySelector(".md-content__inner"); | ||
| if (!content) return; | ||
| var h1 = content.querySelector("h1"); | ||
| if (!h1 || content.querySelector(".md-actions")) return; | ||
|
|
||
| // Rendered pages use directory URLs (/x/); their source .md lives at /x.md, | ||
| // and the site root maps to /index.md. The logo href gives the base path. | ||
| var logo = document.querySelector(".md-header__button.md-logo"); | ||
| var base = logo ? new URL(logo.href).pathname : "/"; | ||
| var path = location.pathname; | ||
| var mdUrl = | ||
| path === base || path === base.replace(/\/$/, "") | ||
| ? base.replace(/\/$/, "") + "/index.md" | ||
| : path.replace(/\/$/, "") + ".md"; | ||
|
|
||
| var bar = document.createElement("div"); | ||
| bar.className = "md-actions"; | ||
|
|
||
| var copy = document.createElement("button"); | ||
| copy.type = "button"; | ||
| copy.className = "md-action"; | ||
| copy.textContent = "Copy as Markdown"; | ||
| copy.addEventListener("click", function () { | ||
| fetch(mdUrl) | ||
| .then(function (r) { | ||
| return r.text(); | ||
| }) | ||
| .then(function (text) { | ||
| return navigator.clipboard.writeText(text); | ||
| }) | ||
| .then(function () { | ||
| copy.textContent = "Copied ✓"; | ||
| setTimeout(function () { | ||
| copy.textContent = "Copy as Markdown"; | ||
| }, 1600); | ||
| }) | ||
| .catch(function () { | ||
| copy.textContent = "Copy failed"; | ||
| }); | ||
| }); | ||
|
|
||
| var view = document.createElement("a"); | ||
| view.className = "md-action"; | ||
| view.href = mdUrl; | ||
| view.textContent = "View as Markdown"; | ||
|
|
||
| var claude = document.createElement("a"); | ||
| claude.className = "md-action"; | ||
| claude.target = "_blank"; | ||
| claude.rel = "noopener"; | ||
| claude.href = | ||
| "https://claude.ai/new?q=" + | ||
| encodeURIComponent("Read " + location.origin + mdUrl + " and help me use model-ledger."); | ||
| claude.textContent = "Open in Claude"; | ||
|
|
||
| bar.appendChild(copy); | ||
| bar.appendChild(view); | ||
| bar.appendChild(claude); | ||
| h1.insertAdjacentElement("afterend", bar); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| User-agent: * | ||
| Allow: / | ||
|
|
||
| Sitemap: https://block.github.io/model-ledger/sitemap.xml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With only
model-ledger[rest-api], the console entry point still importsmodel_ledger.cli.app, which unconditionally importstyperandrich, while therest-apiextra inpyproject.tomlonly adds FastAPI/uvicorn. Users following this row will hit an import error beforemodel-ledger servecan run unless they also install[cli]or the REST extra includes the CLI dependencies.Useful? React with 👍 / 👎.