Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/update-data.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ jobs:
- name: Build tlds.json
if: steps.source-check.outputs.source_changed == 'true'
id: build
run: ./bin/build
run: ./bin/build --all

- name: Notification if build fails
if: failure() && steps.build.outcome == 'failure'
Expand Down
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ help:
@echo " bin/setup Install dependencies (uv sync, pnpm install)"
@echo " bin/lint Run ruff check, ruff format check, pyright"
@echo " bin/test Run lint then pytest"
@echo " bin/build Build output files (--preserve-asn skips the iptoasn refresh)"
@echo " bin/build --preserve-asn Build, keeping committed ASN (local/dev; no iptoasn refresh)"
@echo " bin/build --all Full build, refreshing ASN from iptoasn (CI)"
@echo " bin/fetch-coordinates Fetch geo-place lat/lon from Wikidata (--refresh to redo)"

.PHONY: download-core
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ Alongside `tlds.json`, the build ships four derived reverse-index artifacts that
- **Cultures**: ethno-linguistic communities like the Basques or Welsh (`cultures.json`).
- **Agreement types**: the ICANN registry-agreement enum (`agreements.json`).

Each TLD relates to one or more Organizations through *roles* (Sponsor, Administrative Contact, Technical Contact, and ICANN Registry Operator for gTLDs), to zero or more Places (most ccTLDs map to one country; geographic gTLDs map to a city, subdivision, country, or supranational region), to an optional Culture, and to its agreement types. Each derived artifact is a deterministic reverse index of `tlds.json`: delete it and `./bin/build` rebuilds it. Every cross-file relationship is enforced by referential-integrity tests, so a foreign key can never dangle and no record is ever orphaned.
Each TLD relates to one or more Organizations through *roles* (Sponsor, Administrative Contact, Technical Contact, and ICANN Registry Operator for gTLDs), to zero or more Places (most ccTLDs map to one country; geographic gTLDs map to a city, subdivision, country, or supranational region), to an optional Culture, and to its agreement types. Each derived artifact is a deterministic reverse index of `tlds.json`: delete it and `./bin/build --preserve-asn` rebuilds it. Every cross-file relationship is enforced by referential-integrity tests, so a foreign key can never dangle and no record is ever orphaned.

## `organizations.json`

Expand Down
36 changes: 33 additions & 3 deletions bin/build
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,37 @@
set -euo pipefail
cd "$(dirname "$0")/.."

# Build all generated data files (regenerates the IDN script mapping first).
# Pass --preserve-asn to keep committed ASN and skip the local iptoasn file.
usage() {
cat <<'EOF'
Usage: bin/build <mode>

Regenerates the data/generated artifacts. A mode is required on purpose: a bare
`bin/build` must never silently refresh ASN from a stale local iptoasn snapshot.

Modes:
--preserve-asn Build, keeping the ASN already in the committed tlds.json
(skips the local iptoasn file). Use for local/dev builds so an
unrelated data change does not churn ASN.
--all Full build, refreshing ASN from the local iptoasn file.
Used by CI and the data-update workflow.
-h, --help Show this help.
EOF
}

case "${1:-}" in
--preserve-asn) cli_args=(--build --preserve-asn) ;;
--all) cli_args=(--build) ;;
-h | --help | "")
usage
exit 0
;;
*)
printf 'bin/build: unknown option %s\n\n' "$1" >&2
usage
exit 2
;;
esac

# Regenerate the IDN script mapping, then build.
uv run python scripts/idn_unicode_scripts/generate_idn_script_mapping.py
uv run python -m src.cli --build "$@"
uv run python -m src.cli "${cli_args[@]}"
Loading