Skip to content

Commit f9fe5d9

Browse files
committed
chore: update .claude (5 files), .gzkit (7 files), artifacts (9 files), data +5 more (gz git-sync)
Ceremony: gz-git-sync
1 parent 60c65a5 commit f9fe5d9

32 files changed

Lines changed: 1989 additions & 30 deletions
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"obpi_id": "OBPI-0.0.26-03-clustering-chore",
3+
"parent_adr": "ADR-0.0.26-evaluation-feedback-loop-doctrine",
4+
"lane": "heavy",
5+
"entry": "full",
6+
"execution_mode": "normal",
7+
"current_stage": 1,
8+
"started_at": "2026-05-03T19:46:53.426274+00:00",
9+
"updated_at": "2026-05-03T19:46:53.426274+00:00",
10+
"receipt_state": "PASS",
11+
"blockers": [],
12+
"required_human_action": null,
13+
"next_command": "Stage 2: Implement",
14+
"resume_point": "Stage 2"
15+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"obpi_id": "OBPI-0.0.26-03-clustering-chore",
3+
"parent_adr": "ADR-0.0.26-evaluation-feedback-loop-doctrine",
4+
"lane": "heavy",
5+
"entry": "full",
6+
"execution_mode": "normal",
7+
"current_stage": 1,
8+
"started_at": "2026-05-03T19:46:53.426274+00:00",
9+
"updated_at": "2026-05-03T19:46:53.426274+00:00",
10+
"receipt_state": "PASS",
11+
"blockers": [],
12+
"required_human_action": null,
13+
"next_command": "Stage 2: Implement",
14+
"resume_point": "Stage 2"
15+
}
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"obpi_id": "OBPI-0.0.26-03-clustering-chore",
3+
"timestamp": "2026-05-03T19:37:00Z",
4+
"verdict": "PASS",
5+
"plan_file": "OBPI-0.0.26-03-clustering-chore.md",
6+
"gaps_found": 0,
7+
"notes": "ADR<->OBPI alignment: 7/7 checks aligned. Plan authored in-session covering all 13 REQs across 8 ordered steps with TDD-first ordering. Scope-collision warnings (advisory) are for completed ADRs sharing test directories \u2014 not alignment gaps."
8+
}
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
{
2-
"obpi_id": "OBPI-0.0.26-02-justify-binding-gate",
3-
"timestamp": "2026-05-03T17:23:24Z",
2+
"obpi_id": "OBPI-0.0.26-03-clustering-chore",
3+
"timestamp": "2026-05-03T19:37:00Z",
44
"verdict": "PASS",
5-
"plan_file": "OBPI-0.0.26-02-justify-binding-gate.md",
5+
"plan_file": "OBPI-0.0.26-03-clustering-chore.md",
66
"gaps_found": 0,
7-
"notes": "ADR<->OBPI alignment: all checks aligned. Path discrepancies noted in plan (trust_audits.py is a package; commands/lifecycle.py is src/gzkit/lifecycle.py) \u2014 documented in plan, no alignment gap."
7+
"notes": "ADR<->OBPI alignment: 7/7 checks aligned. Plan authored in-session covering all 13 REQs across 8 ordered steps with TDD-first ordering. Scope-collision warnings (advisory) are for completed ADRs sharing test directories \u2014 not alignment gaps."
88
}
Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
# Plan: OBPI-0.0.26-03-clustering-chore
2+
3+
**OBPI:** OBPI-0.0.26-03-clustering-chore
4+
**Parent ADR:** ADR-0.0.26-evaluation-feedback-loop-doctrine
5+
**Lane:** Heavy (foundation-kind parent)
6+
**Date:** 2026-05-03
7+
8+
## Context
9+
10+
OBPI-0.0.26-01 (persist `adr-evaluation` ledger events) is ATTESTED COMPLETED —
11+
event shape confirmed stable. `AdrEvaluationEvent` fields: `artifact_id`,
12+
`artifact_type`, `dimensions: dict[str, float]`, `scores: dict[str, float]`,
13+
`weighted_total`, `red_team_challenges_fired: list[str]`, `evaluator_persona`,
14+
`timestamp`. Ledger stores as `event == "adr-evaluation"`.
15+
16+
`gz-justify` artifacts live at `artifacts/justify/<slug>-<timestamp>.md` with
17+
YAML frontmatter (`anchor_id`, `anchor_kind`, `generated_at`, `scaffold_version`)
18+
followed by 8 markdown sections.
19+
20+
`data/eval_feedback_thresholds.json` exists with `low_score_threshold` and
21+
`red_team_count_threshold`; needs `cluster_min_recurrence: 3` added.
22+
23+
Chore two-surface layout (ADR-0.0.21): canonical package at
24+
`src/gzkit/chores/eval-feedback-cluster/`, project overlay at
25+
`.gzkit/chores/eval-feedback-cluster/`. `proofs/` is always project-local only.
26+
27+
## Files
28+
29+
### New files
30+
- `src/gzkit/chores/eval_feedback_cluster_lib.py` — clustering engine
31+
- `src/gzkit/chores/eval-feedback-cluster/CHORE.md`
32+
- `src/gzkit/chores/eval-feedback-cluster/acceptance.json`
33+
- `src/gzkit/chores/eval-feedback-cluster/README.md`
34+
- `.gzkit/chores/eval-feedback-cluster/proofs/.gitkeep`
35+
- `tests/chores/test_eval_feedback_cluster.py`
36+
37+
### Modified files
38+
- `data/eval_feedback_thresholds.json` — add `cluster_min_recurrence: 3`
39+
- `src/gzkit/chores/registry.json` — add `eval-feedback-cluster` entry
40+
41+
## Steps
42+
43+
### Step 1 — TDD Red: write failing tests
44+
45+
**File:** `tests/chores/test_eval_feedback_cluster.py`
46+
47+
Write 6 test methods in class `TestEvalFeedbackCluster`, each decorated with
48+
`@covers("REQ-0.0.26-03-NN")`. All tests use `tempfile.TemporaryDirectory` for
49+
ledger and justify-artifact fixtures. No network, no real ledger.
50+
51+
Helper: `_write_ledger(path, events)` — writes list of dicts as JSONL.
52+
Helper: `_write_justify(dir, anchor_id, sections)` — writes a minimal justify
53+
markdown with frontmatter + section content.
54+
55+
Test list:
56+
57+
1. `test_zero_evidence_no_proposals` — empty ledger + no justify artifacts →
58+
`run_cluster()` returns `[]`. Covers REQ-0.0.26-03-08 (zero-evidence run).
59+
60+
2. `test_below_threshold_no_proposal` — 2 distinct artifacts with same
61+
dimension/score_band (below `cluster_min_recurrence=3`) → returns `[]`.
62+
Covers REQ-0.0.26-03-02 (below threshold).
63+
64+
3. `test_at_threshold_emits_proposal` — 3 distinct `adr-evaluation` events
65+
sharing `(dimension_name="clarity", score_band="low")` → returns exactly one
66+
`ProposalRecord` with all schema fields populated and all 3 source IDs.
67+
Covers REQ-0.0.26-03-03, REQ-0.0.26-03-05 (at threshold + schema).
68+
69+
4. `test_multiple_clusters_multiple_proposals` — two independent clusters each
70+
at threshold → returns exactly two proposals.
71+
Covers REQ-0.0.26-03-04 (multiple proposals).
72+
73+
5. `test_idempotent_rerun` — run once, write proposals to proofs dir, run again
74+
with identical evidence → second run returns 0 new proposals written (dedup
75+
by content hash).
76+
Covers REQ-0.0.26-03-06 (idempotency).
77+
78+
6. `test_readonly_constraint``run_cluster()` with a read-only ledger path
79+
succeeds (reads only) and never writes outside `proofs_dir`.
80+
Covers REQ-0.0.26-03-07 (read-only at ledger/ADR surfaces).
81+
82+
Run: `uv run -m unittest tests/chores/test_eval_feedback_cluster.py -v`
83+
expect 6 failures (ImportError or AttributeError — lib does not exist yet).
84+
85+
### Step 2 — Implement `src/gzkit/chores/eval_feedback_cluster_lib.py`
86+
87+
Keep module under 600 lines (pythonic.md). All models use
88+
`ConfigDict(frozen=True, extra="forbid")`.
89+
90+
**`ProposalRecord` model** (REQ-0.0.26-03-05):
91+
```python
92+
class ProposalRecord(BaseModel):
93+
model_config = ConfigDict(frozen=True, extra="forbid")
94+
cluster_key: str
95+
recurrence_count: int
96+
source_artifact_ids: list[str]
97+
source_artifact_paths: list[str]
98+
summary: str
99+
proposed_rule_target: str
100+
```
101+
102+
**`_score_band(score: float) -> str`**: maps score to band:
103+
- `< 1.5``"critical"`, `< 2.5``"very_low"`, `< 3.0``"low"`.
104+
105+
**`_read_adr_evaluation_events(ledger_path: Path) -> list[dict]`**:
106+
Read ledger.jsonl line-by-line; filter `event == "adr-evaluation"`. Return list.
107+
Pure read — no writes.
108+
109+
**`_walk_justify_artifacts(justify_root: Path) -> list[dict]`**:
110+
Walk `justify_root` for `*.md` files. Parse YAML frontmatter (between `---` delimiters).
111+
Return list of `{"path": ..., "anchor_id": ..., "sections": ..., "raw": ...}`.
112+
113+
**`_extract_confusion_keywords(sections_text: str) -> list[str]`**:
114+
Simple keyword scan over a predefined vocabulary set:
115+
`{"unclear", "ambiguous", "confusing", "scope drift", "boundary unclear",
116+
"not sure", "uncertain", "vague", "unresolved", "conflicting"}`.
117+
Returns sorted list of matched keywords present in the text.
118+
119+
**`_build_buckets(events, justify_artifacts, score_threshold: float) -> dict[str, list[dict]]`**:
120+
Returns `{cluster_key: [{"artifact_id": ..., "artifact_path": ...}, ...]}`.
121+
Three bucket families:
122+
- `dim:{dimension_name}:{score_band}` — for each `adr-evaluation` event, for each
123+
dimension with score < `score_threshold`.
124+
- `rt:{challenge_id}` — for each entry in `red_team_challenges_fired`.
125+
- `jk:{keyword}` — for each justify artifact, for each confusion keyword found.
126+
127+
**`_content_hash(cluster_key: str, artifact_ids: list[str]) -> str`**:
128+
`hashlib.sha256(json.dumps([cluster_key, sorted(artifact_ids)]).encode()).hexdigest()[:16]`
129+
130+
**`_load_existing_hashes(proofs_dir: Path) -> set[str]`**:
131+
Read all `proposal-*.json` in `proofs_dir`. Return set of
132+
`content_hash` field values.
133+
134+
**`run_cluster(project_root: Path, *, ledger_path: Path | None = None, justify_root: Path | None = None, proofs_dir: Path | None = None, cluster_min_recurrence: int = 3, score_threshold: float = 3.0) -> list[ProposalRecord]`**:
135+
1. Resolve defaults from `project_root`.
136+
2. Read events and artifacts.
137+
3. Build buckets.
138+
4. Load existing hashes.
139+
5. For each bucket with `len(members) >= cluster_min_recurrence`:
140+
- Compute `content_hash`.
141+
- Skip if hash already in existing hashes.
142+
- Build `ProposalRecord`.
143+
- Write to `proofs_dir/proposal-<utc_timestamp>.json`.
144+
6. Return list of new `ProposalRecord` instances written.
145+
146+
Write proofs only to `proofs_dir` — never to `ledger_path` or `justify_root`.
147+
148+
### Step 3 — TDD Green: run tests, iterate to passing
149+
150+
```bash
151+
uv run -m unittest tests/chores/test_eval_feedback_cluster.py -v
152+
uv run ruff check . --fix && uv run ruff format .
153+
uvx ty check . --exclude 'features/**'
154+
```
155+
156+
All 6 tests must pass.
157+
158+
### Step 4 — Add `cluster_min_recurrence` to `data/eval_feedback_thresholds.json`
159+
160+
Add `"cluster_min_recurrence": 3` to the existing JSON object.
161+
162+
### Step 5 — Create chore package `src/gzkit/chores/eval-feedback-cluster/`
163+
164+
**`CHORE.md`** — slug: `eval-feedback-cluster`, lane: Medium (reads ledger,
165+
walks artifacts; no network), version 1.0.0. Include: overview, policy/guardrails
166+
(read-only at ledger/ADR), workflow (run → review proposals), acceptance criteria
167+
table, evidence commands referencing proofs dir.
168+
169+
**`acceptance.json`** — two criteria:
170+
1. `exitCodeEquals`, `uv run -m unittest tests/chores/test_eval_feedback_cluster.py -q`, `0`
171+
2. `exitCodeEquals`, `uv run gz validate --chores-layout`, `0`
172+
173+
**`README.md`** — one-paragraph summary, quick-start command.
174+
175+
### Step 6 — Create project-local overlay `.gzkit/chores/eval-feedback-cluster/`
176+
177+
Create directory + `proofs/.gitkeep`.
178+
179+
### Step 7 — Register chore in `src/gzkit/chores/registry.json`
180+
181+
Add entry at end of `"chores"` array:
182+
```json
183+
{
184+
"slug": "eval-feedback-cluster",
185+
"title": "Evaluation Feedback Clustering (ADR-0.0.26-03)",
186+
"version": "1.0.0",
187+
"path": ".gzkit/chores/eval-feedback-cluster",
188+
"lane": "medium"
189+
}
190+
```
191+
192+
### Step 8 — Full quality sweep
193+
194+
```bash
195+
uv run gz arb ruff
196+
uv run gz arb typecheck
197+
uv run gz arb step --name unittest -- uv run -m unittest -q
198+
uv run gz validate --chores-layout
199+
uv run gz chores show eval-feedback-cluster
200+
uv run gz chores run eval-feedback-cluster
201+
```
202+
203+
## Verification
204+
205+
Per brief:
206+
```bash
207+
uv run gz lint
208+
uv run gz typecheck
209+
uv run gz test
210+
uv run gz arb step --name unittest -- uv run -m unittest tests/chores/test_eval_feedback_cluster.py -v
211+
uv run gz validate --chores-layout
212+
uv run gz chores show eval-feedback-cluster
213+
uv run gz chores run eval-feedback-cluster
214+
```
215+
216+
## Notes
217+
218+
- `score_band` mapping is a derived heuristic; the threshold for what counts
219+
as "low" is `score_threshold` from `eval_feedback_thresholds.json`
220+
(`low_score_threshold: 3.0`). Scores below that threshold are bucketed by
221+
band subdivision.
222+
- Confusion-keyword vocabulary is a static set; no ML, no network.
223+
- `cluster_min_recurrence` (default 3) mirrors the ADR's "≥3 times across
224+
distinct artifacts" language verbatim.
225+
- Cross-platform: all paths via `pathlib.Path`, UTF-8 encoding explicit.
226+
- No `shell=True` in any subprocess; no subprocesses needed (pure Python).
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# CHORE: eval-feedback-cluster — Evaluation Feedback Clustering
2+
3+
**Version:** 1.0.0
4+
**Lane:** Medium
5+
**Slug:** `eval-feedback-cluster`
6+
7+
---
8+
9+
## Overview
10+
11+
Periodically scan recent `adr-evaluation` ledger events and `gz-justify`
12+
artifacts, cluster by recurring weak-dimension or confusion-shape patterns,
13+
and emit structured proposal records when a pattern recurs ≥3 times across
14+
distinct artifacts (ADR-0.0.26 Decision §3).
15+
16+
## Policy and Guardrails
17+
18+
- **Lane:** Medium — reads ledger and justify artifacts; no network required.
19+
- **Read-only** at `.gzkit/ledger.jsonl` and `docs/design/adr/**`; only writes to its own proofs directory.
20+
- No duplicate proposals: idempotent by content hash over `(cluster_key, sorted source_artifact_ids)`.
21+
- Threshold configurable via `data/eval_feedback_thresholds.json` (`cluster_min_recurrence`, default 3).
22+
23+
## Workflow
24+
25+
### 1. Run clustering
26+
27+
```bash
28+
uv run -m unittest tests/chores/test_eval_feedback_cluster.py -q
29+
```
30+
31+
### 2. Review proposals
32+
33+
```bash
34+
ls .gzkit/chores/eval-feedback-cluster/proofs/
35+
```
36+
37+
Proposals are JSON files: `proposal-<timestamp>.json` with schema:
38+
`cluster_key`, `recurrence_count`, `source_artifact_ids`, `source_artifact_paths`,
39+
`summary`, `proposed_rule_target`.
40+
41+
### 3. Validate layout
42+
43+
```bash
44+
uv run gz validate --chores-layout
45+
```
46+
47+
## Acceptance Criteria
48+
49+
| Type | Command | Expected |
50+
|------|---------|----------|
51+
| exitCodeEquals | `uv run -m unittest tests/chores/test_eval_feedback_cluster.py -q` | 0 |
52+
| exitCodeEquals | `uv run gz validate --chores-layout` | 0 |
53+
54+
## Evidence Commands
55+
56+
```bash
57+
ls .gzkit/chores/eval-feedback-cluster/proofs/
58+
```
59+
60+
---
61+
62+
**End of CHORE: eval-feedback-cluster**
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# eval-feedback-cluster
2+
3+
Cluster recurring weak-dimension and confusion-shape patterns from
4+
`adr-evaluation` ledger events and `gz-justify` artifacts. Emits structured
5+
proposal records to `.gzkit/chores/eval-feedback-cluster/proofs/` when a
6+
pattern recurs ≥3 times across distinct artifacts (ADR-0.0.26).
7+
8+
## Quick Start
9+
10+
```bash
11+
uv run -m unittest tests/chores/test_eval_feedback_cluster.py -q
12+
```
13+
14+
## Lane
15+
16+
**medium**
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"criteria": [
3+
{
4+
"type": "exitCodeEquals",
5+
"command": "uv run -m unittest tests/chores/test_eval_feedback_cluster.py -q",
6+
"expected": 0
7+
},
8+
{
9+
"type": "exitCodeEquals",
10+
"command": "uv run gz validate --chores-layout",
11+
"expected": 0
12+
}
13+
]
14+
}

.gzkit/chores/eval-feedback-cluster/proofs/.gitkeep

Whitespace-only changes.

.gzkit/chores/registry.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,6 +271,13 @@
271271
"version": "1.0.0",
272272
"path": ".gzkit/chores/pythonic-design-pattern-application",
273273
"lane": "lite"
274+
},
275+
{
276+
"slug": "eval-feedback-cluster",
277+
"title": "Evaluation Feedback Clustering (ADR-0.0.26)",
278+
"version": "1.0.0",
279+
"path": ".gzkit/chores/eval-feedback-cluster",
280+
"lane": "medium"
274281
}
275282
]
276283
}

0 commit comments

Comments
 (0)