Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 83 additions & 64 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,40 +7,59 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

### Changed

### Fixed

- **`aim-sot consult` no longer returns stale registry entries** — consult read the
derived memory cache (5b) first and returned it whenever non-empty, with no binding
to the committed `.sot/registry.yaml`, so after any registry edit (or when another
project's rows shared the `group_id`) it shadowed the file with stale or cross-state
data. Consult now uses the cache only when the drift cache's `registry_sha` matches
the committed file's SHA, otherwise it falls back to the committed file; it remains
strictly read-only. Also: `verify --proposal` now flags a proposal that lacks an
`entries` key instead of silently verifying it as empty; `detect-propose run` prunes
drift-cache records for components no longer in the registry (parity with reindex);
and the SKILL.md 5b-cache field name is corrected to `type=sot_entry` to match the
engine. (`aim_sot_consult.py`, `aim_sot_verify.py`, `aim_sot_detect_propose.py`, `SKILL.md`)
## [2.6.0] - 2026-06-16

- **BUG-302: tier-2 fallback marker now shows `remaining=` alongside `tokens=` and `budget=`**
— the marker previously printed `tokens=<N> budget=<total>`, which read as a false
contradiction when `tokens < budget` but `tokens > budget − tokens_used`. Adding
`remaining=budget−tokens_used` makes the correct reject self-evident.
(`context_injection_tier2.py` marker block)
### Upgrade Instructions

- **`aim-sot` is now discoverable as a skill in installed projects** — the skill's
engine lives under `_ai-memory/skills/aim-sot/`, which the installer copied as
canonical files but never surfaced to `.claude/skills/`, so Claude Code could not
index it even though its session/drift hooks were registered. `deploy_ai_memory_skills`
now generates a thin discovery shim in `.claude/skills/` for `aim-*` skills that live
under `_ai-memory/skills/` without a full copy. It runs on every install (matching
aim-sot's always-on SOT hooks), never clobbers a skill that already has a full copy,
and is idempotent on re-install; the `aim-*` prefix deliberately excludes the
oversight-internal `parzival-save-*` skills. The `AI_MEMORY_SOT_HOOKS` opt-out is now
documented as a commented line in `docker/.env.example`.
(`scripts/install.sh`, `docker/.env.example`)
- **Existing installs**: re-run `./scripts/install.sh <project-dir>` to pick up the `aim-sot` skill surface + agent-guidance files (both deploy on every install).
- **Embedding image (TD-626)**: run `~/.ai-memory/scripts/stack.sh restart` to recreate services and pull the prebuilt GHCR embedding image (`ghcr.io/hidden-history/ai-memory-embedding`). `scripts/install.sh` brings services up with `--no-recreate`, so a running `embedding` container keeps its previous locally-built image until a restart. Fresh installs pull the image automatically.
- **aim-sot hooks**: the SOT digest/drift hooks auto-register on install for all supported CLIs (default-on); set `AI_MEMORY_SOT_HOOKS=off` in `docker/.env` before install to opt out.
- **Maintainer (one-time)**: set the `ai-memory-embedding` GHCR package to **Public** (repo → Packages → Package settings → Change visibility) so anonymous `scripts/install.sh` pulls succeed without authentication; until then those pulls fall back to a local source build.

### Added

- **`aim-sot` Source-of-Truth subsystem** — new `aim-sot` skill that tracks where the
canonical truth lives for each boundary of the user's own project. A committed
`.sot/registry.yaml` (in the user's own repo) is the registry of record; the skill
ships the schema, templates, and a three-mode engine:
- **consult** — read-only query over the registry (served from the derived memory
cache, falling back to the committed file); answers "where does X live / who owns it"
for any component.
- **detect-propose** — hybrid auto-discover → propose: scans for candidate components,
computes actual state (SHA-256 of each `sot_location` file), and emits a **proposed
patch** on drift or new candidates. **Never writes the registry** — the propose-only
guarantee is unconditional; every registry change goes through the human-review +
verify gate. The baseline SHA is held (not advanced) when drift is detected, so the
proposal re-fires until a human confirms the change. Cold-start `drift_status` is
`unverified`, not `clean`.
- **verify** — 16-check gate (Schema · Referential · Completeness · Content) returning
PASS / CONDITIONAL / FAIL. K1 (content-hash check) reports CONDITIONAL — not a
silent pass — when no baseline exists. Verdicts distinguish checks that ran and
produced a result from no-op/inert checks and skipped-no-baseline checks.

Two runtime caches support the feature: a per-install drift cache
(`~/.ai-memory/drift-state/sot_drift_{project_id}.json`, machine-local, never
committed) and a derived memory cache (Qdrant `conventions` collection,
`memory_type=sot_entry`, rebuildable from the committed registry at any time via
`detect-propose reindex`). The session-start digest hook and drift Stop hook are
**auto-registered on install for all supported CLIs** (Claude Code, Codex, Cursor,
Gemini) — default on; set `AI_MEMORY_SOT_HOOKS=off` in `docker/.env` to skip
registration. The engine also runs standalone (`detect-propose run`) as the default
no-hook path. All trigger paths are fail-open (exit 0 on any error).

**Data safety**: SOT `owner` and `added_by` fields carry GitHub handles that must
survive the shared PII scanner intact. For `sot_entry` writes only, the scanner's
`HANDLE` (GitHub @-handle) redaction is exempted so ownership attribution is
preserved faithfully in the derived cache; all other masking — secrets, email
addresses, IP addresses, SSNs — is unchanged for SOT entries and for every other
memory type.

- **Per-source budget ledger in `select_results_greedy`** — `meta["per_source"]` now
carries a per-collection breakdown of `requested_tokens`, `loaded_tokens`, and
`dropped` counts by reason (`budget_exceeded`, `score_gap`, `freshness_block`,
Expand All @@ -51,16 +70,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`per_source`. Observe-only: selection output is unchanged. (`injection.py`,
`context_injection_tier2.py`)

### Changed

- **Agent-guidance files now refresh on every install** — existing installations
pick up updated guidance automatically by re-running
`./scripts/install.sh <project-dir>`. No special flag is needed; the guidance
file for each configured CLI is deployed on every install, not only on the
initial setup.

### Added

- **Sanctum content-drift detection (`aim-content-drift`)** — a new skill that
compares an operator's scaffolded sanctum files (BOND, CAPABILITIES, CREED, INDEX,
LORE, MEMORY, PERSONA, PULSE) against the reference templates and surfaces
Expand Down Expand Up @@ -131,36 +140,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `INSTALL-GUIDE-POV.md` gains an `add-project` / shared-stack section documenting
the `INSTALL_MODE=add-project` flow: when it triggers, what the installer skips
vs. runs, prerequisites, per-project configuration, and verification steps.
- **`aim-sot` Source-of-Truth subsystem** — new `aim-sot` skill that tracks where the
canonical truth lives for each boundary of the user's own project. A committed
`.sot/registry.yaml` (in the user's own repo) is the registry of record; the skill
ships the schema, templates, and a three-mode engine:
- **consult** — read-only query over the registry (served from the derived memory
cache, falling back to the committed file); answers "where does X live / who owns it"
for any component.
- **detect-propose** — hybrid auto-discover → propose: scans for candidate components,
computes actual state (SHA-256 of each `sot_location` file), and emits a **proposed
patch** on drift or new candidates. **Never writes the registry** — the propose-only
guarantee is unconditional; every registry change goes through the human-review +
verify gate. The baseline SHA is held (not advanced) when drift is detected, so the
proposal re-fires until a human confirms the change. Cold-start `drift_status` is
`unverified`, not `clean`.
- **verify** — 16-check gate (Schema · Referential · Completeness · Content) returning
PASS / CONDITIONAL / FAIL. K1 (content-hash check) reports CONDITIONAL — not a
silent pass — when no baseline exists. Verdicts distinguish checks that ran and
produced a result from no-op/inert checks and skipped-no-baseline checks.

Two runtime caches support the feature: a per-install drift cache
(`~/.ai-memory/drift-state/sot_drift_{project_id}.json`, machine-local, never
committed) and a derived memory cache (Qdrant `conventions` collection,
`memory_type=sot_entry`, rebuildable from the committed registry at any time via
`detect-propose reindex`). The Claude `Stop` hook and multi-CLI adapters (Codex,
Cursor, Gemini) ship with the install but are **not auto-registered** — opt-in only
(see `docs/AIM-SOT.md`). The engine also runs standalone (`detect-propose run`)
as the default no-hook path. All trigger paths are fail-open (exit 0 on any error).

### Changed

- **Agent-guidance files now refresh on every install** — existing installations
pick up updated guidance automatically by re-running
`./scripts/install.sh <project-dir>`. No special flag is needed; the guidance
file for each configured CLI is deployed on every install, not only on the
initial setup.

- **Sanctum templates** — aligned the eight scaffolded sanctum files (`CREED`,
`PERSONA`, `BOND`, `LORE`, `MEMORY`, `CAPABILITIES`, `INDEX`, `PULSE`) to a
per-file memory-type content model: each file states its memory type, and content
Expand Down Expand Up @@ -204,6 +192,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **`aim-sot consult` no longer returns stale registry entries** — consult read the
derived memory cache (5b) first and returned it whenever non-empty, with no binding
to the committed `.sot/registry.yaml`, so after any registry edit (or when another
project's rows shared the `group_id`) it shadowed the file with stale or cross-state
data. Consult now uses the cache only when every row's stamped `registry_sha` matches
the committed file's SHA **and** the row count matches the committed entry count; a
SHA mismatch or count shortfall falls back to the committed file. Consult remains
strictly read-only. Also: `verify --proposal` now flags a proposal that lacks an
`entries` key instead of silently verifying it as empty; `detect-propose run` prunes
drift-cache records for components no longer in the registry (parity with reindex);
and the SKILL.md 5b-cache field name is corrected to `type=sot_entry` to match the
engine. (`aim_sot_consult.py`, `aim_sot_verify.py`, `aim_sot_detect_propose.py`, `SKILL.md`)

- **BUG-302: tier-2 fallback marker now shows `remaining=` alongside `tokens=` and `budget=`**
— the marker previously printed `tokens=<N> budget=<total>`, which read as a false
contradiction when `tokens < budget` but `tokens > budget − tokens_used`. Adding
`remaining=budget−tokens_used` makes the correct reject self-evident.
(`context_injection_tier2.py` marker block)

- **`aim-sot` is now discoverable as a skill in installed projects** — the skill's
engine lives under `_ai-memory/skills/aim-sot/`, which the installer copied as
canonical files but never surfaced to `.claude/skills/`, so Claude Code could not
index it even though its session/drift hooks were registered. `deploy_ai_memory_skills`
now generates a thin discovery shim in `.claude/skills/` for `aim-*` skills that live
under `_ai-memory/skills/` without a full copy. It runs on every install (matching
aim-sot's always-on SOT hooks), never clobbers a skill that already has a full copy,
and is idempotent on re-install; the `aim-*` prefix deliberately excludes the
oversight-internal `parzival-save-*` skills. The `AI_MEMORY_SOT_HOOKS` opt-out is now
documented as a commented line in `docker/.env.example`.
(`scripts/install.sh`, `docker/.env.example`)

- **aim-sot verify** — `verify run --registry <path>` no longer crashes on a
non-conforming (flat) registry path. A registry outside `<root>/.sot/` makes the
project root resolve to `None`, which previously reached the path checks
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ Memory retrieval fires automatically on:

See [docs/HOOKS.md](docs/HOOKS.md) for complete hook documentation and the full trigger keyword reference.
See [docs/AI_MEMORY_ARCHITECTURE.md](docs/AI_MEMORY_ARCHITECTURE.md) for the full system architecture reference.
See [docs/AIM-SOT.md](docs/AIM-SOT.md) for the Source-of-Truth subsystem — registry schema, consult / detect-propose / verify engine, and automatic drift detection hooks.

---

Expand Down
30 changes: 17 additions & 13 deletions docs/AIM-SOT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The registry lives in **your own repository** as a committed `.sot/registry.yaml
|------|----------|----------|
| **Registry of record** | `.sot/registry.yaml` | Your repo (committed, diff-reviewable) |
| **Skill** | `aim-sot` (consult · detect-propose · verify) | `_ai-memory/skills/aim-sot/` |
| **Triggers (opt-in)** | Claude `Stop` hook; Codex, Cursor, Gemini adapters | Shipped unregistered — see [Opt-In](#enabling-automatic-drift-detection) |
| **Triggers (default-on)** | Claude session-start digest + `Stop` hook; Codex, Cursor, Gemini adapters | Auto-registered on install; opt out with `AI_MEMORY_SOT_HOOKS=off` — see [Automatic Drift Detection](#automatic-drift-detection) |
| **Per-install drift cache (5a)** | `~/.ai-memory/drift-state/sot_drift_{project_id}.json` | Runtime; never committed |
| **Derived memory cache (5b)** | `conventions` collection, `memory_type=sot_entry` | Qdrant; rebuildable |

Expand Down Expand Up @@ -77,7 +77,7 @@ bash "${AI_MEMORY_INSTALL_DIR:-$HOME/.ai-memory}/scripts/memory/run-with-env.sh"

### consult — Orient before acting

Read-only query over your committed registry, served from the derived memory cache (5b) with a fallback to the committed file. Useful for agents that need to locate a component's owner or canonical location before editing it.
Read-only query over your committed registry. Results are served from the derived memory cache (5b) only when every cached row's stamped `registry_sha` matches the committed registry's SHA **and** the row count matches the committed entry count — otherwise the engine falls back to the committed file directly. This ensures consult never returns stale, partial, or cross-state entries. Useful for agents that need to locate a component's owner or canonical location before editing it.

```bash
bash "${AI_MEMORY_INSTALL_DIR:-$HOME/.ai-memory}/scripts/memory/run-with-env.sh" \
Expand Down Expand Up @@ -190,21 +190,25 @@ This is an unconditional structural guarantee: the engine, the Stop hook, and al

All trigger paths are **fail-open**: any error is logged to stderr and the adapter exits 0, so a transient failure never blocks a Claude Code (or Codex/Cursor/Gemini) session.

## Enabling Automatic Drift Detection
## Automatic Drift Detection

The Stop hook and all CLI adapter hooks ship **unregistered**. The installer does not modify your `settings.json`, `.cursor/hooks.json`, `.codex/hooks.json`, or `.gemini/settings.json`. Opt in manually per CLI.
The per-session-start SOT digest hook and the end-of-session drift hook are **registered automatically on install** for every configured IDE (Claude Code, Codex, Cursor, Gemini). To opt out, set `AI_MEMORY_SOT_HOOKS=off` (case-insensitive; only `off` disables — `false`/`0`/`no` leave hooks on) before running `install.sh`. Note: `AI_MEMORY_SOT_HOOKS=off` prevents registration on install but does not remove hooks already written by a prior install; to disable an existing install, remove the SOT hook entries from the relevant config file manually.

The engine also runs standalone with no hook required — use this as the default no-hook path, or schedule it as a cron job:
The engine also runs standalone with no hook required — use this as an alternative, or schedule it as a cron job:

```bash
bash "${AI_MEMORY_INSTALL_DIR:-$HOME/.ai-memory}/scripts/memory/run-with-env.sh" \
"${AI_MEMORY_INSTALL_DIR:-$HOME/.ai-memory}/_ai-memory/skills/aim-sot/scripts/aim_sot_detect_propose.py" \
run
```

### Claude Code — Stop hook
### Claude Code

Add the following entry to your project's `.claude/settings.json` under `hooks.Stop`:
The installer registers two SOT hooks in your project's `.claude/settings.json`:

**Session-start digest** (under `hooks.SessionStart`, matcher `resume|compact`): invokes `sot_digest_session_start.py` and injects a compact SOT registry digest as ambient context on session resume or compact.

**Stop drift hook** (under `hooks.Stop`): invokes `sot_drift_stop.py` at every session end and prints a one-line drift summary to stderr when drift or new candidates are detected. The installed entry:

```json
{
Expand All @@ -225,11 +229,9 @@ Add the following entry to your project's `.claude/settings.json` under `hooks.S
}
```

Once registered, the hook fires at every Claude Code session end and prints a one-line drift summary to stderr when drift or new candidates are detected.

### Codex — Stop hook

Add under `hooks.Stop` in `.codex/hooks.json`:
The installer registers a session-start digest hook and a Stop drift hook in `.codex/hooks.json`. The Stop entry:

```json
{
Expand All @@ -251,7 +253,7 @@ Add under `hooks.Stop` in `.codex/hooks.json`:

### Cursor — stop hook

Add under `hooks.stop` in `.cursor/hooks.json`:
The installer registers a session-start digest hook and a stop drift hook in `.cursor/hooks.json`. The stop entry:

```json
{
Expand All @@ -269,7 +271,7 @@ Add under `hooks.stop` in `.cursor/hooks.json`:

### Gemini CLI — AfterAgent hook

Add under `hooks.AfterAgent` in `.gemini/settings.json`:
The installer registers a session-start digest hook and an AfterAgent drift hook in `.gemini/settings.json`. The AfterAgent entry:

```json
{
Expand Down Expand Up @@ -308,7 +310,9 @@ The cache is keyed by `project_id` (resolved from the `AI_MEMORY_PROJECT_ID` env

After each approved apply, the engine reindexes the committed registry into this cache so `aim-sot consult` (and any agent using `aim-search`) can retrieve SOT entries via semantic search.

**Indexed fields**: `id`, `description`, `sot_location`, `owner`, `provenance_note`, `status`.
**Indexed fields**: `id`, `description`, `sot_location`, `owner`, `added_by`, `provenance_note`, `status`.

**Owner fidelity**: `owner` and `added_by` handles are stored exactly as authored in the registry. The shared security scanner's HANDLE-pattern filter is exempted for SOT entries so that ownership handles (e.g. `@platform-team`) are preserved faithfully — only PII and secrets screening applies.

**Not indexed here**: machine drift state (`last_verified_at`, `last_verified_sha`, `drift_status`) — that lives in 5a.

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "ai-memory"
version = "2.5.0"
version = "2.6.0"
description = "AI Memory Module - Vector-based memory system for AI agents"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
2 changes: 1 addition & 1 deletion scripts/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ EMBEDDING_PORT="${AI_MEMORY_EMBEDDING_PORT:-28080}"
MONITORING_PORT="${AI_MEMORY_MONITORING_PORT:-28000}"
STREAMLIT_PORT="${AI_MEMORY_STREAMLIT_PORT:-28501}"
CONTAINER_PREFIX="${AI_MEMORY_CONTAINER_PREFIX:-ai-memory}"
INSTALLER_VERSION="2.5.0"
INSTALLER_VERSION="2.6.0"

# Logging functions
log_info() {
Expand Down
2 changes: 1 addition & 1 deletion src/memory/__version__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Follows PEP 440 and semantic versioning principles.
"""

__version__ = "2.5.0"
__version__ = "2.6.0"
__version_info__ = tuple(int(part) for part in __version__.split("."))

# Version history:
Expand Down
2 changes: 1 addition & 1 deletion version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.5.0
2.6.0
Loading