B3: refactor into src-layout package with subcommand CLI#3
Conversation
The five top-level scripts have been reorganised into a
substack_link_checker package under src/ with a single subcommand CLI:
substack-link-checker check (was: python substack_link_checker.py)
substack-link-checker compare (was: python compare_posts.py)
substack-link-checker import (was: python import_checked_posts.py)
substack-link-checker fetch-archive (was: python fetch_archive_urls.py)
substack-link-checker demo (was: python demo_link_checker.py)
Package layout:
src/substack_link_checker/
__init__.py - version, public API re-exports
__main__.py - `python -m substack_link_checker ...` entry
cli.py - top-level subcommand dispatcher
_cli_check.py - argparse parser + main for the check subcommand
checker.py - core SubstackLinkChecker class
compare.py
demo.py
fetch_archive.py
import_history.py
Back-compat: the four helper scripts at the root (compare_posts.py,
import_checked_posts.py, fetch_archive_urls.py, demo_link_checker.py)
remain as thin shims that prepend src/ to sys.path and delegate, so
existing `python compare_posts.py ...` invocations and the bundled
PowerShell scheduled task keep working unchanged.
Breaking: `python substack_link_checker.py ...` no longer works because
the filename collides with the new package name. Migration:
`substack-link-checker check ...` or
`python -m substack_link_checker check ...`. README and CHANGELOG have a
migration table; run_link_checker.ps1 already updated.
pyproject.toml switches to src-layout (`package-dir = { "" = "src" }`,
`packages.find.where = ["src"]`) and the entry point is now
`substack_link_checker.cli:main`. pytest is moved to `importlib` import
mode so the package resolves to the installed location, not the shim
files at the project root.
Tests: 35 passing (was 29). Added tests/test_cli_dispatch.py with 6
tests covering the new subcommand router (no-arg help, --help, unknown
subcommand, dispatch, sys.argv restoration, --version). Cookie-handling
tests updated to monkeypatch _cli_check instead of the old module.
.gitignore: added *.egg-info/, build/, dist/, *.egg.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 15d4a33ce9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Step 1: Compare sitemap against history to find new posts | ||
| Write-Host "`nStep 1: Finding new posts..." | ||
| python compare_posts.py $SUBSTACK_URL checked_posts.json | ||
| python -m substack_link_checker compare $SUBSTACK_URL checked_posts.json |
There was a problem hiding this comment.
run_link_checker.ps1 now invokes python -m substack_link_checker ..., but with the new src/ layout that module is not importable from a plain repo checkout unless the environment has already run pip install -e . (or sets PYTHONPATH to src). In Task Scheduler setups that only clone the repo and run this script, step 1 fails immediately with ModuleNotFoundError, so monthly automation no longer runs. Add an explicit setup/path step in this script or call commands that work directly from the checkout.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in a9443c7. Added $env:PYTHONPATH = (Join-Path $PROJECT_DIR "src") + ";" + $env:PYTHONPATH near the top of the script so python -m substack_link_checker resolves from a fresh git clone without requiring pip install -e .. Smoke-tested by uninstalling the editable package locally — PYTHONPATH=src python -m substack_link_checker compare --help now works. Comment in the script explains the why so future readers don't strip it.
Generated by Claude Code
| # so that each module's own parser handles --help correctly with its own | ||
| # epilog, examples, etc. |
There was a problem hiding this comment.
The dispatcher claims each subcommand handles its own help, but compare (and similarly fetch-archive) does not use argparse and treats --help as a positional URL. So substack-link-checker compare --help is routed here and then crashes/behaves incorrectly instead of showing help. Either add real argument parsers for these subcommands or intercept --help before delegation.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in a9443c7. Converted compare.py, fetch_archive.py, and demo.py to use argparse.ArgumentParser with the same positional argument shapes (so existing invocations don't break). --help now exits 0 with proper usage for all three. The dispatcher already sets sys.argv[0] to "substack-link-checker <name>" before calling, so argparse's auto-prog shows the right invocation in the help output.
Added a parametrised regression test in tests/test_cli_dispatch.py asserting --help works for each: 38 tests passing.
Generated by Claude Code
…archive/demo P1 (run_link_checker.ps1): the B3 refactor switched the scheduled task to `python -m substack_link_checker ...`, but that only resolves when the package is on sys.path. Users who just `git clone` and run the task (the v1.0.0 workflow) hit ModuleNotFoundError. Fix: prepend `$PROJECT_DIR\src` to PYTHONPATH near the top of the script, so a fresh clone works without `pip install -e .`. Comment explains why. P2 (cli.py dispatcher): compare/fetch-archive/demo subcommands didn't support --help correctly — they read sys.argv[1] as a positional URL, so `... compare --help` either crashed or was silently ignored. Fix: convert each main() to use argparse with the same positional argument shape, so existing invocations keep working but --help now prints proper usage. The dispatcher already sets sys.argv[0] to "substack-link-checker <name>" so argparse's auto-prog shows the right invocation. Also fix a stale post-success message in fetch_archive.py that printed `python substack_link_checker.py ...` as suggested next-step guidance — that command no longer works after B3. Tests: parametrised regression test in tests/test_cli_dispatch.py asserts `--help` exits 0 with proper usage output for all three subcommands. 38 passing (35 + 3 new).
The CI test job has been running `python substack_link_checker.py --help` as a post-install smoke test since PR #1. B3 (PR #3) deleted that file when refactoring the codebase into the src-layout package, but didn't update this step. CI has been failing on every PR since B3 merged. Replace with two invocations against the installed console script: - `substack-link-checker --help` (top-level dispatcher) - `substack-link-checker check --help` (check subcommand) This is also why the actions/checkout Dependabot PR (#7) was reported as failing — same root cause, not the action bump itself.
) * Prepare v1.1.0: bump version + fix stale troubleshooting invocations Version bump: - pyproject.toml: 1.0.0 -> 1.1.0 - src/substack_link_checker/__init__.py: __version__ -> 1.1.0 - CHANGELOG.md: rename [Unreleased] -> [1.1.0] - 2026-05-19, add a fresh empty [Unreleased] section above it Troubleshooting fix: - README.md Troubleshooting section's code blocks still used the pre-refactor `python substack_link_checker.py ...` / `python fetch_archive_urls.py ...` form because PR #4 was based on the audit branch before B3's CLI rewrite landed. Updated to the current `substack-link-checker check ...` and `substack-link-checker fetch-archive ...` invocations so a reader following the troubleshooting steps doesn't hit "command not found" on the main entry point. Once this lands, push tag v1.1.0 to trigger the release workflow (which builds the wheel/sdist and attaches them to the GitHub Release). * Fix CI smoke test (root-level substack_link_checker.py is gone) The CI test job has been running `python substack_link_checker.py --help` as a post-install smoke test since PR #1. B3 (PR #3) deleted that file when refactoring the codebase into the src-layout package, but didn't update this step. CI has been failing on every PR since B3 merged. Replace with two invocations against the installed console script: - `substack-link-checker --help` (top-level dispatcher) - `substack-link-checker check --help` (check subcommand) This is also why the actions/checkout Dependabot PR (#7) was reported as failing — same root cause, not the action bump itself. --------- Co-authored-by: Claude <noreply@anthropic.com>
2 participants
Summary
Reorganises the five top-level scripts into a
substack_link_checkerpackage undersrc/, exposed as a single CLI with subcommands:Also
python -m substack_link_checker ...works.Layout
Back-compat
The four helper scripts at the root (
compare_posts.py,import_checked_posts.py,fetch_archive_urls.py,demo_link_checker.py) remain as thin shims that prependsrc/tosys.pathand delegate. So existing user commands and the bundled PowerShell scheduled task keep working unchanged.Breaking change
python substack_link_checker.py ...no longer works — the filename collides with the new package name and the two cannot coexist on Python's import path. Migration:python substack_link_checker.py ...substack-link-checker check ...python -m substack_link_checker check ...run_link_checker.ps1already updated. README has a "Migrating from v1.0.0" section.pyproject changes
package-dir = { "" = "src" }+packages.find.where = ["src"]for src-layoutsubstack-link-checker = "substack_link_checker.cli:main"importlibimport mode so the package resolves to the installed location, not the root-level shimsTests
35 passing (was 29 in PR #1). Added
tests/test_cli_dispatch.pywith 6 new tests:--helpflagsys.argvrestored after dispatch--versionflagCookie-handling tests updated to monkeypatch
_cli_checkinstead of the old top-level module.Stacked on #1
Based on
claude/audit-github-repo-EWQDn(PR #1). Once #1 merges, GitHub should auto-rebase the diff so only this commit shows; otherwise re-target the base tomain.Test plan
pip install -e .thensubstack-link-checker --helpshows all subcommandspython -m substack_link_checker check --helpworkspython compare_posts.py https://example.substack.com history.json(back-compat shim) workssubstack-link-checker check --base-url ... --year 2024 --limit 1produces a CSVGenerated by Claude Code