Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,8 @@ PublishScripts/
**/packages/*
# except build/, which is used as an MSBuild target.
!**/packages/build/
# except the site's own browse page, which is source rather than a restore target.
!src/src/pages/packages/**
# Uncomment if necessary however generally it will be regenerated when needed
#!**/packages/repositories.config
# NuGet v3's project.json files produces more ignoreable files
Expand Down
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ just validate
```

runs format-check → lint → typecheck → unit tests → asset checks → catalogue checks → build →
link crawl → generated-output checks → built-output tests. It is the same pipeline CI uses
discovery validation → search-metadata checks → link crawl → generated-output checks → built-output
tests. It is the same pipeline CI uses
(`purview-build.json` → `bun run ci:build`), so a green `just validate` is the definition of "safe to
merge". `just check-projects` is part of that chain and fails the build on catalogue drift. It also
prints advisory observations (which never fail the build) when a project's declared `status` or
Expand Down Expand Up @@ -186,9 +187,10 @@ src/src/data/projects/<id>.yml` against a clone of its repository.
| Docs aggregation | `src/src/lib/docs/aggregate.ts` (+ `frontmatter`, `links`, `sidebar`, `staleness`, `diagrams`, `redirect-file`) |
| Consumer skills for product repositories | `skills/purview-site-{integration,docs,assessment}/` |
| Release data | `src/src/lib/releases/*`, `src/scripts/fetch-releases.ts` |
| Search metadata and browse hubs (titles, descriptions, `/topics/`, `/categories/`, `/packages/`) | `src/src/lib/seo.ts`, `src/src/lib/hubs.ts` (ADR 0015) |
| Mirrored project assets | `src/scripts/sync-assets.ts` (`assets:` in the manifest) |
| Pages | `src/src/pages/{index,about,use-cases,docs,projects,releases}/` |
| Live examples | `src/src/pages/examples/<project-id>.mdx` (linked from the project page automatically) |
| Validation | `justfile`, `src/scripts/{check-links,check-generated,check-assets,check-projects}.ts` |
| Validation | `justfile`, `src/scripts/{check-links,check-generated,check-assets,check-projects,check-seo}.ts` |
| Upstream-triggered rebuilds | `.github/workflows/upstream-refresh.yml`, `src/scripts/upstream-state.ts`, `src/src/lib/upstream.ts` (ADR 0014) |
| Architecture decisions | `docs/decisions/*.md` |
146 changes: 146 additions & 0 deletions docs/decisions/0015-search-metadata-and-browse-hubs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# ADR 0015 — Search metadata derived from the catalogue, and browse hubs

- Status: accepted
- Date: 2026-11-16

## Context

ADR 0007 made the site *discoverable*: a partitioned sitemap, `robots.txt`,
per-project `llms.txt` bundles, `discover.json` and IndexNow. Crawlers could find
every URL, and they did. What they could not do was decide that any page answered
a question, because:

1. **Titles were the page's own name.** A documentation page titled
`Union design | Purview Dev` omits the product it documents; a project page
titled `Unions — Purview Dev` omits every word a reader searches with. The
catalogue already knew those words (`shortDescription`, the package id, the
GitHub topics, the category) and used none of them in metadata.
2. **`og:type` was set globally to `website`** in the Astro config's `head`
array, which overrides the `article` Starlight emits for a documentation page.
3. **Every internal link into the documentation carried the same anchor text**
(`Read the guide →`), discarding the strongest on-page signal a docs page has.
4. **There was no browse surface.** The catalogue's metadata produced 17 project
pages and the docs mirror; a reader searching "c# source generator",
"value types" or "aspire" had no destination, and `/projects/` filtering is
client-side, so it is not a crawl path.
5. **The tags were decorative.** GitHub topics were rendered as chips on the
project and documentation pages and linked to nothing.

The only structured data was a minimal `SoftwareSourceCode`/`TechArticle` pair,
and NuGet — the highest-authority place the packages appear — renders its links
`rel="nofollow"`.

## Decisions

### 1. Search metadata is derived from the catalogue, never hand-authored

`src/src/lib/seo.ts` builds every `<title>` and meta description from the
catalogue and the release cache:

- a project page leads with the **product** (the primary package id) and the
keyword phrase taken from `shortDescription`;
- **a documentation page always names its project**, and the folder a page lives
in (ADR 0011) is part of the title and of the breadcrumb, because it is part of
the URL;
- a non-default documentation channel adds its version label, so the two versions
of one page never share a title (ADR 0009);
- descriptions prefer a sentence boundary when they must be truncated, instead of
ending mid-word with an ellipsis.

The **visible H1 does not change**: the title is composed where every other piece
of search metadata is (`src/src/components/starlight/Head.astro`), from the
catalogue, the docs manifest and the page itself. The docs mirror therefore stays a
mirror of the *documentation* — it carries no site metadata — and a title change
needs no documentation re-sync. Starlight's own `<title>` is replaced, not appended
to, so no page can end up with two.

Every derived title is budgeted to what a result actually shows (70 characters
including the brand, enforced by unit tests over the real catalogue). The parts are
dropped, or trimmed at a word boundary, in this order: the folder trail, the word
"documentation", then part of the page title. Nothing is ever dropped that
identifies the page — the project name in a documentation title, the product name
in a catalogue title — because a title that has lost its subject has lost the
reason to rank; and a shape is only shortened when the shortened form still fits,
so two pages never collapse onto one title.

### 2. Browse hubs are generated from the same metadata

`src/src/lib/hubs.ts` derives:

- `/topics/<topic>/` from GitHub repository topics;
- `/categories/<category>/` from the catalogue's `category` vocabulary;
- `/packages/` from every declared package.

Every hub lists the real projects (status, description, documentation link) and
the documentation pages behind them, so the page has content of its own. A hub is
**only published when it indexes at least two projects** (`MIN_HUB_PROJECTS`):
one-link hubs are the doorway pages search engines devalue, and a topic only one
project carries is better served by that project's page.

Hubs are ordinary Astro pages, so they enter the sitemap, `discover.json`,
`robots.txt` and the IndexNow diff through the existing discovery model
(ADR 0007) with no extra registration — but they are a **new URL family**, hence
this ADR.

### 3. `og:type` is never set globally

The site's own pages set `website` themselves in `SiteLayout`; documentation pages
keep Starlight's `article`. A global value in the config silently wins over both.

### 4. Structured data describes the entity, not the template

- Documentation pages: `TechArticle` (with `keywords` from the repository topics,
`articleSection` from the folder trail, `dateModified` from `lastReviewed`,
`mainEntityOfPage`) plus a `BreadcrumbList` that mirrors the visible structure.
- Project pages: `SoftwareSourceCode` **and** `SoftwareApplication` (free offer,
`downloadUrl`, `softwareRequirements` from the declared target frameworks) plus
the existing `BreadcrumbList`. Assertions are made only when true: a project
with no NuGet package is not given `programmingLanguage: C#`.
- Hubs: `CollectionPage` + `ItemList` + `BreadcrumbList`.

### 5. Internal anchor text names its destination

A use case links to a documentation page by name ("Read Union design"), and a
topic chip becomes a link to its hub whenever that hub exists. The set of tags
with hubs is computed once (`getHubs()`), so a chip can never point at a 404.

### 6. `lastmod` sources are derived, not listed

`PAGE_SOURCES` was a hand-kept map of six routes. Any page not in it — the
`/examples/astroc4/` demo page, and every page added since — silently fell back to
the release cache's retrieval time, which is the build time, which is the one
signal ADR 0007 forbids. The index is now built by walking `src/src/pages`
(workspace-root-relative, so keys still match `git log`), with `[param]` routes
matched by pattern.

### 7. The rules are enforced against the built output

`bun run check:seo` (`src/scripts/check-seo.ts`) runs over `dist/` and fails on
structural violations: not exactly one `<h1>`, a missing or duplicated `<title>`,
a missing description, a canonical that is not self-referencing, `noindex` on an
indexable URL, a documentation title that does not name its project, invalid or
missing JSON-LD. Title and description *lengths* are advisories only, printed and
never fatal — a rule that fails on taste is a rule that gets weakened (ADR 0001).

### 8. Off-site signals are a repository responsibility, recorded in the skills

`purview.dev` has no followed inbound links except the GitHub **Website** field of
each repository, and those point at `/projects/<id>/` only. NuGet's project URL
and README links are all `rel="nofollow"`. The consumer skills
(`purview-site-integration`, `purview-site-docs`) therefore require a product
repository to link its published pages, and the assessment skill reports when it
does not — the one part of SEO this repository cannot do for itself.

## Consequences

- Titles change once; previously indexed pages will be re-crawled under their new
titles. Canonicals are unchanged, so no URL moves.
- The sitemap gains a page per shared topic and category, plus `/packages/`.
`discovery:validate` and `check-links` cover them without modification because
they come from the same discovery model.
- A project onboarding now also creates hub memberships; a project whose topics
are poor (fewer than two shared) contributes to no topic hub, which makes the
existing "set real GitHub topics" intake question more consequential than it was.
- `check:seo` joins `just validate`, so a page that loses its title, its canonical
or its structured data fails the build rather than the ranking.

29 changes: 28 additions & 1 deletion docs/discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,11 @@ under `docsChannels` (schemaVersion 2).
### Sitemap partitioning

- **pages.xml** — hand-authored pages (`/`, `/about/`, `/docs/`, `/projects/`,
`/releases/`, `/use-cases/`).
`/releases/`, `/use-cases/`) and the generated browse hubs
(`/topics/`, `/topics/<topic>/`, `/categories/`, `/categories/<category>/`,
`/packages/`). See ADR 0015: the hubs are ordinary Astro pages derived from the
catalogue's GitHub topics and category, so they enter the sitemap through the
same model as every other page.
- **projects.xml** — every project landing page (`/projects/<id>/`) and all of
its aggregated documentation (`/docs/<id>/…`), derived from the catalogue and
the docs manifest.
Expand All @@ -53,6 +57,27 @@ under `docsChannels` (schemaVersion 2).
Error pages (`404`, `500`), non-canonical aliases and non-indexable pages are
excluded.

### Titles, canonicals and structured data

Being *findable* is not only about being crawled. ADR 0015 derives every
`<title>` and meta description from the catalogue (`src/src/lib/seo.ts`, composed
into the page by the `Head` override), publishes the browse hubs above, and keeps
the page's own title as its visible H1 while the search-result title names the
product. Every derived title is budgeted to the ~70 characters a result shows,
with the folder trail and the word "documentation" dropped before anything that
identifies the page. That is enforced against the built output
by `bun run check:seo` (`src/scripts/check-seo.ts`), which is part of
`just validate`: one `<h1>` per page, unique non-empty titles, self-referencing
canonicals, a documentation title that names its project, and valid JSON-LD.
Length guidance is reported as an advisory and never fails the build.

The one signal this repository cannot create for itself is an inbound link from
another site: NuGet renders its project-URL and README links `rel="nofollow"`, so
the followed links come from each product repository's README and GitHub
**Website** field. The consumer skills (`purview-site-integration`,
`purview-site-docs`) require them, and `purview-site-assessment` reports when one
is missing.

### Meaningful `lastmod`

`lastmod` is **never** the build time. Each resource is dated from the source
Expand Down Expand Up @@ -195,6 +220,8 @@ Automated (nothing to do per deploy):
- [x] IndexNow key file written from the secret by the deploy workflow
- [x] IndexNow notifies changed URLs after a successful deploy
- [x] `discovery:validate` enforced in CI
- [x] `check:seo` enforced in CI (titles, canonicals, one H1, project-named docs
titles, valid JSON-LD) — ADR 0015

One-time (operator):

Expand Down
6 changes: 5 additions & 1 deletion justfile
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ check-generated:
check-projects:
bun run check:projects

# Validate search-engine metadata in the built site (titles, canonicals, headings, structured data).
check-seo:
bun run check:seo

# Apply the guard's auto-fixes to the project records (dry run; pass --write to apply).
fix-projects *ARGS:
bun run fix:projects {{ARGS}}
Expand Down Expand Up @@ -124,5 +128,5 @@ clean:
bun run clean

# Authoritative validation: run everything needed to prove the site is safe to merge.
validate: format-check lint typecheck test check-assets check-projects build discovery-validate check-links check-generated test-dist
validate: format-check lint typecheck test check-assets check-projects build discovery-validate check-seo check-links check-generated test-dist
@echo "Validation passed."
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "purview-dev",
"version": "0.10.0",
"version": "0.11.0",
"private": true,
"description": "Purview-Dev public website and unified documentation portal (workspace root).",
"license": "MIT",
Expand All @@ -14,7 +14,8 @@
"check:generated": "bun run --cwd src check:generated",
"check:links": "bun run --cwd src check:links",
"check:projects": "bun run --cwd src check:projects",
"ci:build": "bun run data:sync && bun run typecheck && bun run check:assets && bun run check:projects && bun run build && bun run discovery:validate && bun run check:links && bun run check:generated && bun run test:dist",
"check:seo": "bun run --cwd src check:seo",
"ci:build": "bun run data:sync && bun run typecheck && bun run check:assets && bun run check:projects && bun run build && bun run discovery:validate && bun run check:seo && bun run check:links && bun run check:generated && bun run test:dist",
"clean": "bun run --cwd src clean",
"data:sync": "bun run --cwd src data:sync",
"dev": "bun run --cwd src dev",
Expand Down
2 changes: 1 addition & 1 deletion skills/add-project-intake/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ resolve to a real page under `/docs/<id>/` (see `use-case-authoring`).

| # | Question | Decides |
| --- | --- | --- |
| 23 | Repository topics are missing or poor — they are the site's tag chips. May I set them? | tags (`check-projects` **fails** on no topics) |
| 23 | Repository topics are missing or poor — they are the site's tag chips and the `/topics/` hubs a project appears in. May I set them? | tags (`check-projects` **fails** on no topics); browse hubs (ADR 0015) |
| 24 | The description is thin — it backs the `shortDescription` fallback and social metadata. May I improve it? | `shortDescription` fallback, `og:` metadata |
| 25 | Is there a schema or artefact worth mirroring to a stable `purview.dev` URL (an `assets:` entry), or is it already published elsewhere? | `assets` |

Expand Down
4 changes: 4 additions & 0 deletions skills/github-repo-metadata/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Everything else in the table below is build-time.
| Site surface | Source | Notes |
| --- | --- | --- |
| Tag chips (cards, docs portal, docs-page front matter `tags`) | GitHub **topics** | `projectRepoEnrichment()` in `src/src/lib/releases/repo.ts` |
| The `/topics/<topic>/` browse hub a chip links to | GitHub **topics** | A topic shared by two or more projects gets a hub page (ADR 0015); a topic only one project carries links nowhere, and one hub's worth of links is lost |
| `shortDescription` fallback | GitHub **description** | Used only when the manifest value is empty |
| "supplemental" description on the project page | GitHub description | Shown when it differs from the manifest description |
| Successor/archived messaging | GitHub **archived** flag + `status`/`supersededBy` | |
Expand Down Expand Up @@ -72,6 +73,9 @@ Discussions on an organisation repository requires organisation permission; repo
searches behave predictably.
- Topics are the **only** way tags reach the site. There is no `tags` field in a catalogue record — do
not add one.
- Spell a shared concept the standard way and only the standard way: `csharp`, not `c-sharp`. The
site groups the two spellings into one `/topics/csharp/` hub, but a topic nobody else uses gets no
hub at all, so a project's topics are also how it joins the site's browse surfaces (ADR 0015).

## Stability signals to check against `status`

Expand Down
12 changes: 12 additions & 0 deletions skills/purview-site-assessment/references/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ The text-fence classifier skips folder trees (`├──`, `└──`), which a
RED-004 reads the last 300 commits that touched the docs folder. A page that existed only between two
site builds was never published; ignore those.

## Search visibility

| Rule | Severity | Finding | Fix |
| -- | -- | -- | -- |
| SEO-001 | warning | the README does not link `purview.dev`, so the site has no followed inbound link from this repository | link the published documentation (`https://purview.dev/docs/<id>/`) near the top of the README, with link text a reader would search with |
| SEO-002 | info | the repository has no Website/Homepage field (checked by hand with `gh api repos/<owner>/<repo> --jq .homepage`) | set it to `https://purview.dev/projects/<id>/`; the published pages already link back with their own URL |
| SEO-003 | info | a docs page whose first paragraph does not name the technology it documents | make the first sentence answer "what does this page cover?" — the site uses it as the meta description (ADR 0015) |

NuGet renders the project URL and every README link `rel="nofollow"`, and the
GitHub Website field is followed, so SEO-001/SEO-002 are the only inbound links
the site can earn from a repository. Not a formatting nicety.

## Catalogue (with `--catalogue`)

| Rule | Severity | Finding | Fix |
Expand Down
24 changes: 24 additions & 0 deletions skills/purview-site-assessment/scripts/assess.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,30 @@ for (const site of ['mkdocs.yml', 'mkdocs.yaml', 'docs/mkdocs.yml', 'docs/wiki/m
}
}

// ---------------------------------------------------------------------------
// Search visibility
// ---------------------------------------------------------------------------

// purview.dev's only *followed* inbound links come from this repository: NuGet
// renders the project URL and README links `rel="nofollow"`, and the GitHub
// Website field is a followed link into the catalogue page. Without one of them
// the published page cannot rank for the topic it documents, however good it is
// (ADR 0015).
if (readmePath) {
const readmeText = read(readmePath);
const relativeReadme = toPosix(relative(root, readmePath));
if (!/purview\.dev/.test(readmeText)) {
report(
'SEO-001',
'warning',
relativeReadme,
undefined,
'The README does not link the published site, so purview.dev has no followed inbound link from this repository.',
'Link the published documentation (https://purview.dev/docs/<id>/) near the top, with link text a reader would search with ("Union design", "Installation guide").',
);
}
}

// ---------------------------------------------------------------------------
// Documentation
// ---------------------------------------------------------------------------
Expand Down
Loading
Loading