Skip to content

Offer the link checker to the other repos as a reusable workflow #64

Description

@lesnik512

What

Make the link checker added in #63 available to the other 27 repos in the org, as a
reusable workflow hosted here rather than a file copied into each repo.

Two changes:

  1. Convert .github/workflows/links.yml to on: workflow_call (keeping a thin local
    caller so this repo still checks itself on its own schedule).

  2. Add a ~10-line caller to each repo that wants it:

    name: Links
    on:
      schedule:
        - cron: "0 6 * * 1"
      workflow_dispatch:
    permissions:
      contents: read
      issues: write
    jobs:
      check:
        uses: modern-python/.github/.github/workflows/links.yml@main

Why a reusable workflow, not a copy

The config is not boilerplate — it encodes two org-wide findings that every repo would
otherwise have to rediscover:

  • /stargazers is login-walled. It returns 404 to anonymous clients on every
    repo, ours and psf/requests alike. Every library README carries a Stars badge
    linking there, so a naive config reports them all dead on its first run. In this
    repo that was 26 URLs, 19% of the total.
  • pepy download badges 404 by design until a listed package reaches PyPI.
    AGENTS.md documents that lag as self-healing and explicitly not a blocker.

A repo whose copy is missing either exclusion gets a check full of false positives, and
the reliable outcome of that is someone muting it.

Copying the workflow also reproduces exactly the failure #50 was written about — "a
repo-local template silently wins over the org default and there is no signal when the
two drift"
. A caller pinned to @main cannot drift: there is one copy of the
exclusions.

Mechanics worth knowing before starting

  • Workflows do not propagate from this repo. The default community-health-file
    feature covers CONTRIBUTING, CODE_OF_CONDUCT, SUPPORT, issue/PR templates and
    FUNDING.yml only — never .github/workflows/. There is no zero-touch option; every
    repo needs a file of its own, which is why the caller is as small as possible.
  • This repo is public, so every org repo can call a reusable workflow from it.
  • The alternative — one job here checking out all 28 repos — was considered and
    rejected: every issue would land in .github instead of the repo that owns the
    broken link, and it needs a PAT or GitHub App because GITHUB_TOKEN is scoped to a
    single repo.

Scope

28 non-archived repos. The ones that benefit most are the published libraries, whose
READMEs carry PyPI, coverage, docs and Stars badges, and the repos with docs sites —
mkdocs build --strict validates links inside docs/, never a README's external
URLs.

Not every repo needs it; a repo with three links gains little. Worth deciding per repo
rather than opening 27 PRs by reflex.

Open questions

  • Stagger the crons? 28 repos all firing at 06:00 Monday is a burst of requests at
    shields.io and pepy from one org. Spreading them across the week is one line per
    caller.
  • Does the reusable workflow need inputs? A repo with a genuinely repo-specific
    exclusion would need one (extra-exclude, say). Starting with no inputs and adding
    one when a second repo actually needs it is probably right.
  • Pin @main or a tag? @main propagates exclusion fixes automatically, which is
    the point; a tag would mean re-pinning 27 repos to ship a fix.

Reference

The first real run on this repo: 229 links checked, 178 successful, 51 excluded, 0
errors — see the run from #63. Note that lychee also validates relative file links
across all Markdown, so a caller gets internal-link checking for free alongside the
external URLs.

Activity

  1. lesnik512 commented on Sep 6, 2026

    @lesnik512
    MemberAuthor

    This was generated by AI during triage.

    Triage outcome: enhancement, ready-for-human

    Three checks were run against the org before classifying.

    Not already implemented

    Searched all 28 non-archived repos by concept rather than wording — lychee,
    markdown-link-check, linkinator, linkchecker. The only CI link checking that
    exists anywhere in the org is the workflow merged in #63. No repo has its own.

    There is no .out-of-scope/ knowledge base in this repo, so the prior-rejection check
    had nothing to match against; noting where it looked rather than implying it came back
    clean.

    Prior art points the other way, and should be reconciled with

    httpware/planning/changes/2026-06-08.08-readme-link-cleanup.md evaluated lychee in
    June 2026 and ruled explicitly: "Don't add it to CI." It ran a one-shot audit
    instead, citing false positives from rate-limiting and transient failures.

    That is not a rejection of what this issue proposes — scheduled rather than gated,
    fail: false, a GITHUB_TOKEN for github.com, and exclusions measured rather than
    guessed are exactly the mitigations that decision lacked, and #63's first run bore that
    out at 229 links checked, 0 errors. But it is a documented prior "no" from a sibling
    repo, and whoever picks this up should address it rather than rediscover it.

    This would be the org's first cross-repo workflow reuse

    Searched for uses: modern-python/… across the org: zero hits. Nine repos do use
    workflow_call, but every one is a repo-local _checks.yml invoked by that repo's
    own ci.yml. They are not copies that drifted — all nine differ, 40 to 106 lines,
    because their CI genuinely differs (python matrices, benchmarks, coverage gates).

    That distinction supports centralising this config, whose correctness is org-wide
    truth in a way a test matrix is not. But it reframes the risk, and it promotes the
    issue body's third open question from a footnote to the decision the rest depends on:
    pointing 27 repos at links.yml@main gives every one of them a hard dependency on this
    repo, with instant blast radius on a bad push. A tag bounds that, at the cost of
    re-pinning 27 repos to ship an exclusion fix.

    Brief

    Category: enhancement
    Summary: Host the link checker as a reusable workflow so other repos can adopt it without copying its exclusions.

    Current behavior:
    .github/workflows/links.yml in this repo runs lychee on a weekly schedule, opens an
    issue on failure, and encodes two org-wide findings as exclusions: /stargazers is
    login-walled and 404s for anonymous clients on every repo; pepy download badges 404 by
    design until a listed package reaches PyPI. No other repo has any link checking, and
    workflows do not propagate from this repo — the default community-health-file mechanism
    covers CONTRIBUTING, CODE_OF_CONDUCT, SUPPORT, issue/PR templates and
    FUNDING.yml only.

    Desired behavior:
    Another repo can adopt link checking with a caller of roughly ten lines and inherit both
    exclusions automatically, including any later correction to them. An issue about a
    broken link is opened on the repo that owns the link, not centrally.

    Key interfaces:

    • The workflow gains workflow_call as a trigger. Its path is load-bearing and unlike
      most references in a brief must not be treated as incidental: uses: resolves
      {owner}/{repo}/.github/workflows/{filename}@{ref} literally, so renaming the file
      breaks every caller.
    • This repo keeps a caller of its own, so it still checks itself on its own schedule.
    • A caller supplies its own schedule/workflow_dispatch triggers and needs
      issues: write.
    • Whether the reusable workflow takes inputs (an extra-exclude, say) is open; starting
      with none and adding one when a second repo actually needs it is the cheaper order.

    Acceptance criteria:

    • A repo other than this one runs the check via uses: with no lychee arguments of its own.
    • A correction to an exclusion reaches adopting repos by the agreed mechanism (@main automatically, or a documented re-pin).
    • A broken link opens an issue on the adopting repo.
    • This repo's own checking is unchanged in behaviour and schedule.
    • The httpware "don't add it to CI" decision is either superseded in writing or the repo is deliberately left out.

    Out of scope:

    Why this is not delegable

    Nothing here is hard to write; what remains is three decisions an agent would have to
    guess at, each with org-wide consequences:

    1. @main versus a tag — a risk-appetite call, now known to be novel for this org
      rather than an established pattern.
    2. Which repos get it — judgment about where the value is, not a mechanical sweep.
    3. Whether to stagger the crons — 27 repos firing at 06:00 Monday is one org's burst
      at shields.io and pepy.

    Step 1 in the body (convert to workflow_call, keep a local caller) is mechanical on its
    own and could be delegated once decision 1 is made, since the pinning answer changes
    whether a tag has to be cut as part of it.

  2. lesnik512 commented on Sep 6, 2026

    @lesnik512
    MemberAuthor

    This was generated by AI during triage.

    Closing as superseded by a design session

    Grilling this issue against measurement retired its central premise. Recording why, since
    the reasoning is the part worth keeping.

    The valuable half needs no cross-repo machinery at all. Splitting link checking by
    determinism (#65) put relative-link checking on the PR gate, where it belongs: those links
    break because a diff broke them, they need no network, and they cannot flake. That half is
    ~5 lines in a repo's existing _checks.yml with no dependency on this repo. What was left
    for a reusable workflow was only the external check — and that is the half carrying the
    pinning and blast-radius risk, for the smaller prize.

    The measurements that decided it, run with the lychee container across all 28
    non-archived repos rather than estimated:

    • 77 broken local links org-wide. 77 of 77 inside planning/. Zero in any user-facing
      surface
      — no README, docs page, ADR or root Markdown in any repo. Verified against a
      synthetic repo with deliberately broken links to rule out a scan-scope artefact.
    • 21 of 28 repos are green today. The 7 that are not need only the planning/
      migration, which deletes the offending files.
    • Only ~6 dead external links exist org-wide, against those 77. The external check is
      the smaller problem as well as the riskier one to share.

    Two of this issue's own premises were wrong, both mine:

    • "Stagger the crons" assumed the org spreads its schedules. It does not: all 26
      scheduled workflows use the identical 0 6 * * 1. There is nothing to stagger.
    • "An issue should open on the repo that owns the link" was presented as an argument for
      per-repo adoption. It is free either way: GITHUB_TOKEN in a called reusable workflow is
      scoped to the caller, so a shared workflow already opens issues on the adopting repo. The
      point did not favour either design.

    What survives is tracked separately: the offline gate rollout to the 18 green,
    non-trivial repos, and the ADR publishing change in the two repos that have ADRs.

    Not being done, deliberately: hosting anything as a cross-repo reusable workflow. If the
    external check is ever wanted in other repos, this issue holds the analysis for doing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions