Merge pull request #1 from i-d-e/claude/analyze-project-X6xZd

Monorepo-Schnitt: TEI-Korpus + Schema in ride-static eingezogen

Author: chpollin

.github/workflows/build.yml

@@ -2,17 +2,12 @@
 # requirements.md N10 (one workflow file) and pipeline.md "GitHub Actions
 # workflow (Phase 15)".
 #
-# Status: SKELETON. Steps marked "Phase 8+" are placeholders that fail
-# loudly until the corresponding pipeline phase lands. The deploy chain
-# (upload-pages-artifact + deploy-pages) is the modern Actions-based
-# Pages flow — no /docs folder, no committed build output.
-#
-# Triggers:
-#   - push to main on changes that affect TEI sources, content, or pipeline
-#   - manual dispatch from the Actions tab
-#
-# The ride corpus is checked out as a sibling at ../ride so that the
-# REPO_ROOT.parent / "ride" pattern in scripts/*.py works unchanged in CI.
+# The TEI corpus and schema now live in this repo under issues/{N}/reviews/
+# and schema/. Per-issue pictures still live in i-d-e/ride under
+# issues/issue{NN}/{slug}/pictures/ and are pulled in via the asset pipeline.
+# The picture sibling is checked out alongside so REPO_ROOT.parent / "ride"
+# resolves at build time; when pictures move into this repo too, the second
+# checkout can drop.
 
 name: build
 
@@ -26,18 +21,18 @@ on:
       - "static/**"
       - "config/**"
       - "content/**"
+      - "issues/**"
+      - "schema/**"
       - "tests/**"
       - "requirements.txt"
       - ".github/workflows/build.yml"
   workflow_dispatch:
 
-# Pages deploys need exactly these permissions; nothing else.
 permissions:
   contents: read
   pages: write
   id-token: write
 
-# Don't queue overlapping deploys; the latest push always wins.
 concurrency:
   group: pages
   cancel-in-progress: true
@@ -51,7 +46,10 @@ jobs:
         with:
           path: ride-static
 
-      - name: Checkout ride corpus (sibling)
+      # Pictures still live in the i-d-e/ride sibling repo. The asset
+      # pipeline reads them via REPO_ROOT.parent / "ride" / "issues" / ...
+      # When pictures move into ride-static, drop this step.
+      - name: Checkout ride pictures (sibling, pictures only)
         uses: actions/checkout@v4
         with:
           repository: i-d-e/ride
@@ -64,17 +62,13 @@ jobs:
           cache: pip
           cache-dependency-path: ride-static/requirements.txt
 
-      # WeasyPrint links Pango/Cairo at import time. The Ubuntu runner
-      # ships a partial GTK stack; install the WeasyPrint-listed runtime
-      # libs explicitly so `import weasyprint` succeeds in CI.
       - name: Install WeasyPrint system libraries
         run: sudo apt-get update && sudo apt-get install -y libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz-subset0
 
       - name: Install dependencies
         working-directory: ride-static
         run: pip install -r requirements.txt
 
-      # Cache the 4 MB TEI P5 subset between runs; pipeline.md "Re-deployment flow".
       - name: Cache TEI P5 subset
         uses: actions/cache@v4
         with:
@@ -85,7 +79,6 @@ jobs:
         working-directory: ride-static
         run: python -m pytest -q
 
-      # Tier 1 — independent extractors. Could parallelise; sequential is cheap and ordered.
       - name: Discovery scripts (Tier 1)
         working-directory: ride-static
         run: |
@@ -109,33 +102,11 @@ jobs:
           python scripts/render_data.py
           python scripts/render_schema.py
 
-      # Phase 13 lives here once it lands: pre-build RelaxNG + Schematron validation
-      # against ../ride/schema/ride.odd, with a per-file report.
-      # - name: Pre-build validation (Phase 13)
-      #   working-directory: ride-static
-      #   run: python -m src.validate
-
-      # Phase 8+ — the actual site build.
-      # base-url is the path prefix under which Pages serves the site. For
-      # https://<owner>.github.io/<repo>/ this is "/<repo>" (the implicit
-      # GitHub-Pages-project-page convention). When a custom domain is
-      # configured (e.g. https://ride.i-d-e.de/), drop the flag so paths
-      # resolve from the root instead.
-      - name: Build site (Phase 8+)
+      - name: Build site
         working-directory: ride-static
-        run: |
-          if [ -f src/build.py ] || python -c "import src.build" 2>/dev/null; then
-            python -m src.build --base-url=/${{ github.event.repository.name }} --pdf
-          else
-            echo "::warning::src.build is not implemented yet (Phase 8). Emitting placeholder site/."
-            mkdir -p site
-            echo "<!doctype html><title>ride-static</title><h1>build pending</h1>" > site/index.html
-          fi
-
-      # Phase 11 — Pagefind index. Indexes site/ after the build, writes the
-      # client-side runtime under site/pagefind/. Picked up by static/js/pagefind.js
-      # via the data-pagefind-ui container in base.html.
-      - name: Build Pagefind index (Welle 9)
+        run: python -m src.build --base-url=/${{ github.event.repository.name }} --pdf
+
+      - name: Build Pagefind index
         working-directory: ride-static
         run: npx -y pagefind --site site
 

CLAUDE.md

@@ -8,7 +8,7 @@ TEI XML → Python/Jinja → HTML/PDF, deployed via GitHub Actions.
 
 ## Hard rules
 
-- **TDD with real-corpus drive.** Every script and parser module ships with pytest coverage. **Integration tests use the real corpus** (`../ride/tei_all/*.xml`) — parse a real review and assert the resulting domain shape, rather than constructing `Review`/`Section`/`Block` instances directly from synthetic dataclass values. Synthetic-from-dataclass construction is technical debt: it locks tests to the model contract while bypassing the parser, hiding parser regressions. **Pure-function unit tests** (regex, classifier, formatter) may use synthetic inputs because the function signature is the only data form richer than that — document this in the test docstring. **Edge cases that genuinely do not exist in the corpus** (truly unparseable URLs, future-proofing branches) keep a synthetic fixture but the docstring names the case as an explicit exception. Real-corpus tests skip cleanly when `../ride/` is absent so the unit suite runs on a fresh clone.
+- **TDD with real-corpus drive.** Every script and parser module ships with pytest coverage. **Integration tests use the real corpus** (`issues/{N}/reviews/*-tei.xml`) — parse a real review and assert the resulting domain shape, rather than constructing `Review`/`Section`/`Block` instances directly from synthetic dataclass values. Synthetic-from-dataclass construction is technical debt: it locks tests to the model contract while bypassing the parser, hiding parser regressions. **Pure-function unit tests** (regex, classifier, formatter) may use synthetic inputs because the function signature is the only data form richer than that — document this in the test docstring. **Edge cases that genuinely do not exist in the corpus** (truly unparseable URLs, future-proofing branches) keep a synthetic fixture but the docstring names the case as an explicit exception. Real-corpus tests skip cleanly when the corpus is absent so the unit suite runs on a partial checkout.
 - **`knowledge/` is a clean Obsidian-style vault — Markdown plus referenced image attachments.** Generated JSON belongs in `inventory/`. Cross-references inside the vault use `[[wikilink]]` notation. Hand-written filenames are lowercase. Image attachments (e.g. `image-workflow.png`) live next to the Markdown that references them, the conventional Obsidian-vault layout.
 - **`inventory/` is gitignored** (visible, no leading dot). Always regeneratable from scripts.
 - **Anomalies are explicit.** Known data quirks (no `<back>`, `<num value="3">`, `<list rend="numbered">`, etc. — see `knowledge/data.md`) become named branches in the parser. Unknown ones must raise.
@@ -30,8 +30,12 @@ ride-static/
     render/, build.py     html, pdf, refs, assets, build CLI
   templates/html/         Jinja templates
   static/                 css/, js/, fonts/
-  config/                 element-mapping.yaml + per-issue YAML configs
+  config/                 element-mapping.yaml, navigation.yaml
   content/                editorial Markdown: about, imprint, criteria, reviewers
+  issues/{N}/             TEI corpus, grouped per issue
+    metadata.yaml         editorially curated issue metadata (DOI, editors, …)
+    reviews/*-tei.xml     the TEI review files for this issue
+  schema/                 ride.odd + ride.rng — RIDE TEI ODD + compiled RelaxNG
   inventory/              Generated JSON artifacts — gitignored
     _cache/               Cached upstream downloads (e.g. p5subset.xml)
   knowledge/              Obsidian-style vault, .md only, wikilinks for cross-refs
@@ -64,13 +68,14 @@ Cross-references inside `knowledge/` use **Obsidian wikilinks** (`[[filename]]`
 
 Generated docs (`data.md`, `schema.md`) must not be edited by hand — changes go into `scripts/render_data.py` / `render_schema.py`. Hand-written docs are the only place where wikilinks may be added directly.
 
-Source corpus lives in the **sibling** directory `../ride/`:
+The TEI corpus and schema live **inside this repo**:
 
-- `../ride/tei_all/*.xml` — 107 reviews
-- `../ride/schema/ride.odd` — RIDE-specific TEI ODD
-- `../ride/issues/` — per-issue dirs with images
+- `issues/{N}/reviews/*-tei.xml` — 111 reviews, grouped by issue number (`biblScope @n`)
+- `issues/{N}/metadata.yaml` — per-issue editorial metadata (DOI, editors, contribution order, …)
+- `schema/ride.odd` — RIDE-specific TEI ODD
+- `schema/ride.rng` — compiled RelaxNG used by `src/validate.py`
 
-Scripts dereference this via `REPO_ROOT.parent / "ride" / ...`.
+Path lookups go through `src/_corpus.py` (`iter_tei_files`, `find_tei`, `CORPUS_ROOT`, `SCHEMA_ODD`, …). Per-issue **pictures** still live in the sibling repo `i-d-e/ride` under `../ride/issues/issue{NN}/{slug}/pictures/`; the asset pipeline reads them via `REPO_ROOT.parent / "ride"` and degrades cleanly when the sibling is absent. When pictures move into this repo too, that fallback drops.
 
 ## Stage 0/1 outputs
 

CONTRIBUTING.md

@@ -4,19 +4,17 @@ Goal of this document: any new contributor reaches a productive state within hal
 
 ## Setup
 
-Requirements: Python 3.11, git, a checkout of the sibling corpus repo at `../ride/`.
+Requirements: Python 3.11, git.
 
 ```sh
 git clone <this-repo>
-git clone <ride-corpus-repo> ../ride        # the TEI source corpus
-
 cd ride-static
 python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
 pip install -r requirements.txt
 python -m pytest tests/
 ```
 
-If `../ride/` is not present, the parser still runs and 87 of 88 tests pass — only the real-corpus smoke test skips. Discovery scripts in `scripts/` need `../ride/` to do anything useful.
+The TEI corpus ships in this repo under `issues/{N}/reviews/`, no separate clone is needed. Per-issue **picture assets** still live in `i-d-e/ride`; clone it as a sibling (`git clone <ride-corpus-repo> ../ride`) only if you need figures to render in the local build — the pipeline degrades cleanly without it.
 
 ## Repository layout
 
@@ -36,7 +34,7 @@ See `CLAUDE.md` for the layout reference. In short:
 
 These are the project's non-negotiables. Codified for clarity, not for ceremony.
 
-- **TDD.** Every script and parser module ships with pytest using synthetic TEI fixtures. The parser also has one real-corpus smoke test that skips when `../ride/` is absent.
+- **TDD.** Every script and parser module ships with pytest. Integration tests drive off the real corpus shipped under `issues/{N}/reviews/`; pure-function unit tests may use synthetic inputs.
 - **`knowledge/` stays clean.** Hand-written and generated `.md` only. No JSON, no scripts, no notebooks. Cross-references between knowledge docs use `[[wikilink]]` notation.
 - **`inventory/` is gitignored.** Always regeneratable from scripts. Never edit by hand.
 - **Anomalies are explicit.** Known data quirks become named branches in the parser. Unknown ones must raise.
@@ -61,7 +59,7 @@ If a documented decision conflicts with the code, fix the code. If the code is r
 
 **Commits.** Short title in the form `Area: what changed`, e.g. `Parser: integrate body into parse_review`. Body is optional — use it when the *why* needs explaining. Sign with the standard `Co-Authored-By` trailer if pair-coding with an agent.
 
-**Tests.** Synthetic fixtures live inline in test files. Real-corpus smoke tests skip via `pytest.skip` when `../ride/` is absent. Each new script or parser module gets its own `tests/test_<name>.py`. Assert exact output paths when scripts write files (see `tests/test_odd_extract.py` for the canonical example).
+**Tests.** Synthetic fixtures live inline in test files. Real-corpus tests find files via `src._corpus.find_tei("slug")` or iterate via `iter_tei_files()`. Each new script or parser module gets its own `tests/test_<name>.py`. Assert exact output paths when scripts write files (see `tests/test_odd_extract.py` for the canonical example).
 
 **Type hints.** Domain types are immutable: `@dataclass(frozen=True)` with `tuple[...]` for sequences (hashability). Optional fields default to `None`, never to mutable defaults.
 

Journal.md

@@ -30,6 +30,37 @@ Three persistence layers run in parallel for this project: `CLAUDE.md` for proje
 
 ---
 
+## 2026-05-12 — Monorepo-Schnitt: TEI-Korpus + Schema in ride-static eingezogen
+
+**Ziel:** Den TEI-Korpus aus `i-d-e/ride` ins eigene Repo holen, damit ride-static nicht mehr von einem zweiten Checkout abhängt, und die Issue-Metadaten dabei direkt neben den TEI-Dateien gruppieren.
+
+**Erledigt:**
+- 111 TEI-Reviews aus `i-d-e/ride/tei_all/` per `biblScope @n` ausgewertet und in `issues/{N}/reviews/{slug}-tei.xml` einsortiert; Schema (`ride.odd` + `ride.rng`) liegt nun unter `schema/` im Repo-Root.
+- 22 YAML-Issue-Configs von `content/issues/{N}.yaml` nach `issues/{N}/metadata.yaml` umgezogen; `content/issues/` entfernt.
+- Neues Helfer-Modul `src/_corpus.py` (`iter_tei_files`, `find_tei`, `CORPUS_ROOT`, `SCHEMA_ODD`, `SCHEMA_RNG`, `corpus_available`) ersetzt den verteilten `REPO_ROOT.parent / "ride" / "tei_all"`-Pattern.
+- Alle Pfad-Konstanten in `scripts/{inventory,structure,sections,refs,ids,taxonomy,odd_extract}.py`, `src/build.py`, `src/validate.py`, `src/render/issues_config.py` umgestellt; Globs auf `**/*.xml` bzw. `*/reviews/*.xml` / `*/metadata.yaml`.
+- 21 Test-Dateien automatisiert auf `find_tei("slug")` + neue Globs migriert; Skip-Mechanismus von `test_parser_assets.py` zeigt jetzt auf den Bilder-Sibling.
+- CI (`.github/workflows/build.yml`) verschlankt: `i-d-e/ride` wird nur noch wegen der Picture-Assets als Sibling geklont; alle anderen Schritte greifen auf das in-repo TEI-/Schema-Layout zu.
+- Doku synchronisiert: `CLAUDE.md`, `README.md`, `CONTRIBUTING.md`, `docs/extending.md`, `knowledge/architecture.md`, `knowledge/pipeline.md` — Layout-Tabellen, Stakeholder-Eingangspunkte, Hard-Rule-Quote, "107 reviews"→"111 reviews".
+- Tests: 468 passed, 7 skipped (die Asset-Picture-Tests, weil `../ride/` in der Sandbox fehlt).
+
+**Entscheidungen:**
+- **Per-Issue-Layout `issues/{N}/{metadata.yaml, reviews/*-tei.xml}` statt flachem `corpus/tei_all/`.** Begründung: alles zu einem Issue an einem Ort, Editor sieht Metadaten und Reviews nebeneinander; YAML-Membership wäre eine zweite Wahrheit gegenüber `biblScope @n`, darum bleibt die TEI-Header-Zuordnung kanonisch.
+- **Schema unter `schema/` im Root, nicht unter `corpus/schema/`.** Begründung: das Schema ist Validierungs-Vertrag, nicht Korpus-Inhalt — gleiche Trennung wie zwischen `src/` und `tests/`.
+- **Pictures (~437 MB, 888 Dateien) bleiben vorerst im Sibling `i-d-e/ride`.** Begründung: Repo-Größe und Klon-Zeit; LFS-Quota würde belastet. Die Asset-Pipeline degradiert sauber, wenn Sibling fehlt; CI klont den Sibling weiterhin nur für Bilder.
+- **`src/_corpus.py` als zentrale Wahrheit für Pfade.** Begründung: vor der Migration war `REPO_ROOT.parent / "ride" / "tei_all"` in ~35 Dateien dupliziert; bei der nächsten Layout-Änderung muss nur dieses Modul angefasst werden.
+- **CI: zweiter Checkout bleibt, aber explizit als "pictures only" dokumentiert.** Wenn die Bilder migriert werden, fällt der Step weg.
+
+**Offen:**
+- Bilder-Migration: 437 MB in 22 Issue-Ordnern, vermutlich via Git-LFS. Eigene Entscheidung in einer separaten Session.
+- `RIDE_ROOT` in `src/build.py` und `_RIDE_ROOT` in `tests/test_parser_assets.py` zeigen noch auf den Sibling — kann erst entfallen, wenn die Bilder hier liegen.
+- Knowledge-Doku-CI: Drift zwischen generiertem `knowledge/data.md` und aktuellem Korpus könnte auffliegen, wenn CI strict läuft (4 neue Reviews seit Phase 10-Refresh).
+- Phase 15.B (WCAG-Audit, Matomo-Secrets, Custom-Domain) weiterhin offen — unabhängig von dieser Umstrukturierung.
+
+**Nächster Einstieg:** Lokal `python -m src.build` ohne `../ride/` laufen lassen und prüfen, dass der AssetReport sauber alle fehlenden Bilder meldet statt zu crashen. Wenn das durchläuft, ist die Layout-Migration end-to-end verifiziert.
+
+---
+
 ## 2026-04-29 — Phase 10-Rest: Data-Charts (R9) live
 
 **Ziel:** Den letzten inhaltlichen Brocken aus Phase 10 abräumen — aggregierte Bar-Charts auf `/data/charts/` aus dem realen Korpus statt des seit Welle 6 stehenden Placeholder-Markdown.