Skip to content

Development Setup

Jump to bottom
Ken Tobias edited this page Oct 12, 2026 · 41 revisions

Development Setup

How to build, test, release and package retch. This page gives the rules; the incidents behind most of them are written up in NOTES.md §7 ("Hard-won lessons") in the main repo.

Prerequisites

  • Rust (latest stable toolchain)
  • just (the command runner every workflow below goes through) and Python 3 (most recipes are Python, so just check also runs on a default Windows PATH)
  • mandown ≥ 1.1.1 for just man (cargo install mandown). The version matters: older builds emit .Bl/.El mdoc macros, which makes just pr's man-page comparison fail with a ~164-line diff that looks like your change broke the page.
  • Optional: hyperfine and fastfetch for CLI benchmarks, podman for the AUR recipes.

Nix / NixOS Setup

If you use Nix, all prerequisites are provided automatically via the flake devShell:

git clone https://github.com/l1a/retch.git
cd retch
nix develop

This drops you into a shell with cargo, rustc, rust-analyzer, just, mandown, hyperfine, and python3 available. Skip to Common Just Recipes once inside.

Local Setup

  1. Clone the repository: 
    git clone https://github.com/l1a/retch.git
cd retch
  2. Run the one-time setup, which installs two git hooks: 
    just setup
    pre-push runs just check before every push, whatever tool is pushing. post-merge uploads a local benchmark run to the dashboard after you merge into main.
  3. Build: 
    cargo build --release

Common Just Recipes

Recipe Description
just dev Full development cycle: setup → fmt → lint → test → build
just check Strict fmt + clippy checks plus every offline guard (same as CI)
just fmt Auto-format code with cargo fmt
just lint Run clippy with -D warnings
just test Run all tests
just build Build release binary
just clean Remove build artifacts
just audit Run cargo audit for known security advisories
just man Regenerate docs/retch.1 from docs/retch.1.md via mandown
just install Install from this checkout: binary + committed man page + completions
just install-tag VERSION Install a released tag — binary, completions and man page all from that tag
just install-man Install the man page to the XDG man directory
just install-completions Generate and install completions for six shells
just standard-check Verify the vendored install helpers still meet the shared cross-repo standard, and refuse an @ inside a shebang recipe
just install-hooks Install git hooks (included in just setup)
just publish-check Verify packaging and dry-run publish for crates.io
just publish Publish both crates to crates.io (sysinfo first, then CLI); refuses unless HEAD is the tag
just render-check Verify scripts/render_packaging.py still renders and still refuses (offline; run by just check)
just aur-check Verify the AUR PKGBUILD is still a template recording no version (offline; run by just check)
just aur-render VERSION DIR Render the PKGBUILD + generate .SRCINFO for a released tag into DIR (needs podman)
just aur-srcinfo DIR Generate .SRCINFO beside a rendered PKGBUILD in DIR (needs podman)
just aur-publish VERSION Render for a released tag and push to the AUR (asks before publishing)
just aur-local [VERSION] Render and makepkg -si locally, without the AUR (Arch only)
just copr-check Verify the COPR spec is still a template recording no version, and run copr.yml's credentials step (offline; run by just check)
just copr-render Print the spec as .copr/Makefile will render it (offline)
just brew-check Verify the Homebrew formula is still a template recording no version (offline; run by just check)
just brew-publish VERSION Render for a released tag and push to the tap at l1a/homebrew-retch (asks first)
just metadata-check Verify every channel's description and licence agrees with packaging/metadata.toml (offline; run by just check)
just text-check Refuse control characters and carriage returns in tracked text (offline; run by just check)
just wip-check Prove update_wip.py rewrites only its marked block, and that WIP.md is still LF (offline; run by just check)
just bench-check Prove the benchmark result parser reads every source and refuses a partial result, and that the fastfetch configs still match src/fields.rs (offline; run by just check)
just fastfetch-configs Regenerate benches/fastfetch/*.jsonc from src/fields.rs; run after moving a field between modes
just github-metadata [--dry-run] Set the GitHub repo description and topics from packaging/metadata.toml, then read them back
just pr Automated Pre-PR gate — see below
just open-pr Runs just pr, then gh pr create if it passes — the only sanctioned way to open a PR
just merge-pr Squash-merge, switch to main, regenerate WIP.md's state block — see below

Test and lint with --workspace (cargo test --workspace): the root Cargo.toml is both the retch-cli package and the workspace root, so a bare cargo test skips crates/sysinfo, where most of the probing code lives.

Opening and merging a pull request

Never call gh pr create directly. Always use:

just open-pr          # any args are forwarded to gh pr create

It runs the Pre-PR gate (just pr) first and only opens the PR if that passes. gh has no hook of its own, so this recipe is the one enforcement point that works whatever is driving: a human shell, Claude Code, or any other agent. Run just pr again before every later push to an open PR. It checks, in order:

  1. you are on a feature branch, not main;
  2. Cargo.toml's version is bumped past the last tag (patch for fixes, tests and docs; minor for new user-visible features);
  3. NOTES.md has a ## Current State (v<version>) header for that version;
  4. the regenerated man page matches the committed docs/retch.1;
  5. Cargo.lock is committed;
  6. just check and cargo test --workspace pass;
  7. cargo audit (advisory only; it never blocks);
  8. a manual checklist you confirm: README, NOTES.md, the wiki, and docs/retch.md if flags changed. A non-interactive caller answers it with PR_CONFIRM=y, which is the same confirmation as typing y, not a bypass. See AGENTS.md for the full checklist.

After the PR is merged:

just merge-pr

It refuses if any check failed or none reported, then squash-merges via gh, switches to main, pulls (which fires the post-merge benchmark upload), deletes the local branch, and regenerates the state block at the top of WIP.md. It owns only the lines between <!-- BEGIN merge-pr state … --> and <!-- END merge-pr state -->; hand-written entries are yours to update.

The repo squash-merges with the branch commit's message, not the PR body, so fix a commit message by amending the commit. On a multi-commit branch the bodies are concatenated, so put trailers on the last commit only.

Guards worth knowing

Everything below runs in just check.

  • text-check reads the bytes of every tracked text file and refuses C0 control characters (except tab and newline), DEL and carriage returns. Both have shipped here: a double backslash typed into an agent's shell arrives as a single one (usr\bin became usr
    • 0x08), and with .gitattributes pinning eol=lf, git hides a CRLF working copy after the first git add. To see line endings, ask git ls-files --eol (look for i/lf w/crlf), not git status.
  • wip-check proves update_wip.py touches only its marked block, and that WIP.md is still LF. WIP.md is gitignored, so .gitattributes and text-check cannot see it. If it fails, repair the file in place, never through a temp file in /tmp: on this project's Syncthing hosts that carries an SELinux label the sync container cannot read, and the whole shared folder stops syncing. 
    python3 -c "from pathlib import Path; p=Path('WIP.md'); p.write_bytes(p.read_bytes().replace(bytes([13,10]), bytes([10])))"
  • standard-check runs the vendored install helpers' self-tests and scripts/gate_conformance.py, which refuses a leading @ in a shebang recipe. In a plain recipe just strips the @; in a shebang recipe it reaches the shell as a command named @/usr/bin/python3 and exits 127. Heredocs are skipped, since a line of data may start with @.
  • bench-check, metadata-check and the four packaging guards are covered in their own sections below.

Releasing

A release edits no files. Nothing in the repository records a released version: the packaging files are templates filled in at publish time, and main sits at the released version until the next PR bumps it.

cargo owner --list retch-cli     # proves the crates.io token first; an expired one fails at upload
git tag -a vX -F <message-file> --cleanup=verbatim && git push origin vX
                                 # CI builds the GitHub Release; copr.yml rebuilds COPR and
                                 # pushes the COPR project page text
just publish                     # crates.io; refuses unless HEAD is the tag for Cargo.toml
just aur-publish X               # renders + pushes to the AUR (needs podman)
just brew-publish X              # renders + pushes to the Homebrew tap
just github-metadata             # the repo's About box, from packaging/metadata.toml
  • --cleanup=verbatim matters. With -F, git's default strips every line starting with #, so a heading like #281 — … silently disappears. Word headings as PR #281 and compare the tag message with what you meant before pushing it.
  • The confirm variables PR_CONFIRM, AUR_CONFIRM and BREW_CONFIRM all accept y. AUR_CONFIRM=n just aur-publish X runs every preflight check without publishing.
  • If one leg fails partway, re-run just that leg (just brew-publish X), not the whole sequence, which would start again at crates.io.
  • Verify each channel from the channel, not from the publish command's exit code: install the crate into a throwaway root and run it, read the AUR's git repo (its web API lags), and check COPR per chroot through its API.
  • PUBLISH_ANY_REF=1 overrides just publish's HEAD-is-the-tag guard, deliberately.

Channel text

Every channel shows a description and a licence. packaging/metadata.toml is the one place they are written: the one-line summary, the Homebrew desc, the SPDX licence, the GitHub description and topics, and the COPR project page (packaging/copr/project-*.md). Text an artifact carries ships with that channel's publish step, and metadata-check fails when any copy drifts. The COPR page is pushed by copr.yml on each tag, and the About box by just github-metadata.

The guard proves the copies agree, not that they are true. Before tagging, read the READMEs and these files for claims that are no longer right.

Packaging

Templates, not copies

packaging/aur/PKGBUILD, packaging/copr/retch.spec and packaging/homebrew/retch.rb record no released version. Their version and checksum are @VERSION@/@SHA256@, filled in by scripts/render_packaging.py at publish time from the tag and the sha256 of the tarball it downloads. .SRCINFO is not tracked; it is generated from the rendered PKGBUILD.

This replaced in-repo copies that each recorded the version. They drifted (the PKGBUILD once sat eleven releases stale while CI was green), and because a checksum only exists after its tag, recording it forced a post-release commit and a version bump on every release. With one recording made at publish time, there is nothing to drift and nothing to bump.

The four offline guards assert that property: aur-check, copr-check and brew-check that each template still records no version, and render-check that the renderer still substitutes and still refuses (a substitution that matched nothing, a sentinel that survived, a checksum that is not a real digest).

AUR

The tag must exist first, because the checksum comes from its tarball. just aur-publish X downloads it, renders the PKGBUILD, generates .SRCINFO, checks the pair agrees, and pushes. It refuses before touching the AUR if the pair disagrees, a sentinel survived, or the AUR is in a maintenance window. To install locally without the AUR, just aur-local renders for the last released tag and runs makepkg -si (makepkg refuses the template's @ directly).

Never hand-edit .SRCINFO. It is generated by a real makepkg --printsrcinfo in an archlinux:base-devel container, since no development host here runs Arch. The AUR reads metadata from .SRCINFO but builds from the PKGBUILD, so a mismatched pair fails quietly: the AUR advertises one version and builds another.

Homebrew

just brew-publish X renders the formula and pushes it to the tap. The formula builds from source; there is deliberately no bottle. The CI brew job installs the formula from a local tap, which bypasses Homebrew 6.0's trust gate, so it proves the formula builds and works, not that brew tap l1a/retch works without brew trust (that is covered in Getting Started).

COPR

packaging/copr/retch.spec is written for a COPR project with "Enable internet access during builds" on, so cargo build resolves crates.io at build time with no vendor tarball:

  • --locked is load-bearing. It is the only thing pinning resolution to what was tested.
  • It will not build in koji. Fedora proper needs offline builds, which would mean a vendor tarball as Source1 and system sqlite instead of rusqlite's bundled feature. COPR is the endpoint.

Version: is @VERSION@ and Source0 is the local %{name}-%{version}.tar.gz: .copr/Makefile renders the version from Cargo.toml and builds the tarball from the checkout COPR cloned. So a pull request can build its own SRPM.

Rebuilds are triggered by the tag. .github/workflows/copr.yml runs on a v* tag (or by hand), and three guards keep it honest:

  • it refuses to build unless Cargo.toml equals the tag, so it cannot publish an unreleased version;
  • copr-cli buildscm --commit <tag> builds that exact commit, not whatever main is by the time COPR clones;
  • missing COPR secrets fail the run on l1a/retch (since v0.20.9); only a fork skips. copr_check.py --self-test runs the real step to hold both halves.

One-off setup. A package first built from an uploaded SRPM cannot auto-rebuild. Convert it once:

copr-cli edit-package-scm kentobias/retch --name retch \
  --clone-url https://github.com/l1a/retch.git \
  --commit main --method make_srpm --webhook-rebuild off

It is --method make_srpm, not --type (which picks the version-control tool), and there is no --subdir: COPR runs .copr/Makefile from the repository root. --webhook-rebuild off is deliberate, because copr.yml starts rebuilds on the tag, and COPR's own webhook would rebuild on every push as well. Check copr-cli edit-package-scm --help rather than a remembered invocation.

COPR runs the Makefile inside mock, which sets HOME=/builddir and moves rpm's %{_topdir} to /builddir/build. A plain container does not reproduce that, so test both environments, each in a fresh container:

# happy path
podman run --rm -v "$PWD:/src:ro,z" fedora:latest bash -c '
  dnf -y install make rpm-build rpmdevtools
  cp -r /src /work && cd /work && mkdir -p /tmp/out
  make -f .copr/Makefile srpm outdir=/tmp/out'

# mock's environment: BOTH variables
podman run --rm -v "$PWD:/src:ro,z" fedora:latest bash -c '
  dnf -y install make rpm-build rpmdevtools
  cp -r /src /work && cd /work && rm -rf .copr-sources
  mkdir -p /builddir /tmp/out
  export HOME=/builddir
  echo "%_topdir /builddir/build" > /builddir/.rpmmacros
  make -f .copr/Makefile srpm outdir=/tmp/out'

Mount with :z, never :Z: uppercase gives the directory a private SELinux category, and this repo lives in a Syncthing folder whose own container then cannot read it. CI's copr job runs both builds on every PR that can affect packaging.

Other spec details, each commented in the spec itself:

  • Build flags: it exports Fedora's own RUSTFLAGS="%{build_rustflags}". Their -Cdebuginfo=2 -Cstrip=none half lets rpm produce -debuginfo packages; the shipped binary is still an optimised, stripped release build. %cargo_prep/%cargo_build are not used, because they assume Fedora's offline, vendored workflow.
  • Install paths use %{bash_completions_dir}, %{zsh_completions_dir} and %{fish_completions_dir}.
  • Optional runtime tools are Recommends (light: chafa, curl, iproute, dmidecode) or Suggests (the desktop stack), never Requires, since a missing tool only makes a field absent. Establish package names with dnf repoquery, not memory.
  • Inspecting an RPM: rpm compresses man pages (retch.1.gz, so match retch.1*), and the fedora container image sets tsflags=nodocs, so dnf install there silently skips the man page. Read the payload with rpm2cpio instead.

What CI checks

layer proves needs
just render-check, aur-check, copr-check, brew-check the templates record no version, and the renderer still refuses bad input nothing (pure Python)
just aur-publish the checksum matches the real tarball, and the rendered PKGBUILD/.SRCINFO pair agrees network + podman
CI aur job the render for the last released tag is right, it builds with makepkg, and the packaged man page is right GitHub Actions
CI copr job the SRPM builds from the PR's own source, plainly and under mock's environment GitHub Actions
CI brew job the formula builds, installs and passes brew audit --strict on macOS GitHub Actions
COPR itself the RPM builds and %check passes on all four chroots a v* tag

"CI was green" is not "CI checked the packaging." packaging.yml runs only for PRs touching something that can affect packaging: packaging/**, .copr/**, any *.rs, Cargo.toml/Cargo.lock, the flake, the Justfile, scripts/*_check.py, scripts/render_packaging.py, and the packaging.yml and copr.yml workflows.

The install recipes, and the shared standard

cargo install on its own replaces only the binary, leaving the man page and completions at whatever version last installed them. The recipes install all three together:

binary completions man page
just install this checkout generated by that binary the committed docs/retch.1
just install-tag X cargo install --git --tag vX generated by that binary git show vX:docs/retch.1

install-tag deliberately does not reuse install-man/install-completions, which read the working tree and would pair a tag's binary with the checkout's man page.

Where completions go. scripts/install_completions.py writes six shells' completions to their per-user XDG locations, with measured exceptions:

  • nushell on Windows reads only %APPDATA%\nushell\autoload, never ~/.config/nushell.
  • PowerShell on Windows loads $PROFILE (under Documents\PowerShell, which OneDrive can redirect). The helper still writes the XDG path but reports it as NOT ACTIVE.
  • zsh reads completions only from directories on fpath. The helper asks an interactive zsh whether ~/.local/share/zsh/site-functions is on it and, if not, prints the line to add.
  • The helper takes a command name, never a path; a path used to make it overwrite the binary with its own completion script. Use just install-completions, or python3 scripts/install_completions.py retch --from-path.

The cross-repo standard. scripts/install_completions.py, scripts/install_man.py and templates/justfile-common.just are vendored byte-identically in retch, rusticprofile and etr. The Justfile carries the shared recipes between # >>> COMMON (template vN) and # <<< COMMON, with project facts (BINS, MAN_PAGES) in a header above. standard-check runs the helpers' own self-tests rather than diffing files. Editing inside the markers means changing the standard: edit the template and helpers, bump their versions, and propagate to the sibling repos in their own PRs.

Running Benchmarks

  • Criterion micro-benchmarks: just bench
  • CLI execution speed: just bench-cli
  • retch against fastfetch in all four modes: just bench-compare
  • Where one run spends its time, each probe's start offset and duration on stderr: 
    RETCH_TIMING=1 ./target/release/retch --long >/dev/null
    Capture it while retch is slow: a cost can be intermittent, and a sweep of --fields differences cannot see a slow serial section where start offsets can.

Matched comparisons. The pairs are defined once, in scripts/cli_bench.py, and CI, just bench-compare and just bench-upload all run them: retch [--<mode>] against fastfetch -c benches/fastfetch/<mode>.jsonc, a config generated from src/fields.rs by scripts/fastfetch_configs.py so both tools show the same fields. After moving a field between modes, run just fastfetch-configs; just check fails until you do. The modes run default first, then short, long and full (RUN_ORDER), because whichever runs first lands on a cold CI runner. fastfetch never loads your own config.

Uploading. just bench-upload appends a local run to the dashboard (needs hyperfine and an authenticated gh; on Windows run it from Git Bash). The post-merge hook runs it after every merge. just bench-upload --no-build --dry-run clones gh-pages, appends and stages the run, and prints the diff that would land, without pushing. If you touch how it writes dev/bench/data.js, match github-action-benchmark exactly (indent=2, ensure_ascii=False, no trailing newline), or every upload rewrites the whole file.

The dashboard at dev/bench is this repo's own page (benches/dashboard/, published with just bench-page, which asks first). It plots by the version scripts/bench_meta.py stamps into each point, marks local points AC or battery, and charts retch time ÷ fastfetch time per mode from the same run, which cancels runner speed. History before v0.20.7 is frozen at dev/bench/archive. Each suite keeps 200 runs.

Series names are an API. The dashboard keys each series by its label string, so renaming one starts a new series and orphans the old one's history. scripts/bench_labels.py produces all of them and pins the eight CLI execution - … names.

Both publishers refuse partial results. scripts/parse_criterion.py (CI) globs hyperfine_*.json, treats an expected but absent source as an error, and its self-test reads benchmark.yml to check the glob matches every file the workflow writes. (A hardcoded filename once let CI discard the CLI timings for three months while staying green.) It and just bench-upload also refuse a run missing any of the expected series (bench_labels.missing_series): in October 2026 an export the uploader could not read made it publish two empty points while reporting success.

hyperfine 1.x and 2.x both work. hyperfine 2.0 runs commands with no shell by default, splits them with POSIX rules (a backslash is an escape, so Windows paths are written with forward slashes: ./target/release/retch.exe), and writes a new JSON schema (wall-clock mean under summary.time_wall_clock). cli_bench.py and just bench-cli pass --shell=none explicitly so every version measures the same way, and bench_labels.mean_seconds reads both schemas. Which version you have depends on your package manager; cargo install hyperfine gives 2.x.

How the workflow is shaped

.github/workflows/benchmark.yml runs five platform jobs in parallel, each ending in an artifact, and one publish job appends all five to dev/bench/data.js. That shape is forced by every suite living in that one file on one branch:

  • publish runs github-action-benchmark once per suite as sequential steps of one job, each with auto-push: false, followed by a single git push;
  • every step sets skip-fetch-gh-pages: true, after one explicit git fetch. Otherwise the second step's own fetch is a non-fast-forward and the job dies;
  • publish runs on !cancelled(), and each step is guarded on its own artifact existing, so one failed platform loses only its own series;
  • a concurrency: group stops two quick merges from starting two runs that both publish.

A PR can test the measurement but not the publish. PRs touching the benchmark scripts, benches/**, src/fields.rs or the workflow run the five jobs without publishing. The publish path runs only on main, so after merging a change to it, check the first run, and remember that a failed publish loses that run's points, never history.

Clone this wiki locally