Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
code that writes the files.
- One folder under `v1/` per dataset. A new dataset also gets an entry in `src/catalog.ts`
and its sources in `SOURCES`.
- Respect every licence: values come from Wikidata (CC0) only. Wikivoyage and Wikipedia
are linked, never copied. A new source needs its licence in `SOURCES` and the README.
- Never correct data here: errors are fixed on Wikidata, at each record's `source`.
- Respect every licence: a dataset carries the licence of its sources, and a share-alike
source (ODbL, CC BY-SA) makes the dataset share-alike too. Wikivoyage and Wikipedia are
linked, never copied. A new source needs its licence and credit in `src/sources.ts` and
the README.
- Never correct data here: errors are fixed at the source (Wikidata, GeoNames, Trainline…).
40 changes: 33 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# data

The open data every Offware app reads, in one place: towns and cities worldwide with their
sights and links to free travel guides, and railway stations worldwide. Built from open
sources by GitHub Actions and served as static JSON on GitHub Pages, with no server, no key
and no cost.
sights and links to free travel guides, railway stations, cities, regions, airports, UNESCO
World Heritage sites, countries and Postcards' moments. Built from open sources by GitHub
Actions and served as static JSON on GitHub Pages, with no server, no key and no cost.

[train-api](https://github.com/offware-apps/train-api) links its stations to these places,
and [Postcards](https://github.com/offware-apps/Postcards) reads the sights and stations.
and [Postcards](https://github.com/offware-apps/Postcards) reads every other dataset.

Live at <https://offware-apps.github.io/data/v1/catalog.json> once Pages is on
(Settings → Pages → Source: GitHub Actions).
Expand All @@ -23,13 +23,39 @@ built, how many records it has and which apps read it.
| | `v1/places/landmarks.json` | Every sight except UNESCO World Heritage sites, in Postcards' landmark shape |
| | `v1/places/pack.json` | The sights as a Postcards community data pack |
| stations | `v1/stations/railways.json` | Open railway stations worldwide, in Postcards' station shape |
| all | `v1/sources.json` | Every source and its licence |
| stations-europe | `v1/stations-europe/railways.json` | About 70,000 European stations from Trainline, same shape |
| cities | `v1/cities/top.json`, `v1/cities/all.json` | The 10,000 most populous cities, and all 135,000 |
| regions | `v1/regions/regions.json` | First-level regions (states, provinces, régions) with their names |
| airports | `v1/airports/airports.json` | Every airport with an IATA code |
| heritage | `v1/heritage/unesco.json` | Every UNESCO World Heritage site, once per country |
| countries | `v1/countries/*.json` | Common names, continents, languages and UN membership |
| moments | `v1/moments/moments.json` | Postcards' travel experiences, each citing its sources |
| all | `v1/sources.json` | Every source, its licence and the credit it asks for |

Towns are kept from 5,000 people in Europe and from 15,000 elsewhere. The format is in
[docs/format.md](docs/format.md).

## Sources and licences

Each dataset keeps the licence of what it is built from, given as `license` in
`catalog.json`. A share-alike licence (ODbL, CC BY-SA) carries over to the dataset, and
an app showing the data shows the source's `attribution` from `sources.json`.

| Dataset | Licence | Source |
| --- | --- | --- |
| places, stations | CC0 1.0 | Wikidata |
| stations-europe | ODbL 1.0 | [Trainline EU stations](https://github.com/trainline-eu/stations) |
| cities | CC BY 4.0 | [GeoNames](https://www.geonames.org), by way of all-the-cities |
| regions | ODbL 1.0 | GeoNames, and [dr5hn countries-states-cities](https://github.com/dr5hn/countries-states-cities-database) for a few names |
| airports | ODbL 1.0 | [OpenFlights](https://openflights.org/data.html) (from OurAirports, public domain) |
| heritage | CC BY-SA 3.0 IGO | [UNESCO World Heritage Centre](https://whc.unesco.org/en/list/) |
| countries | ODbL 1.0 | [mledoze/countries](https://github.com/mledoze/countries), ISO 3166-1 codes |
| moments | Each item cites its sources | Written for Postcards (CC0), drawn from Wikidata, GeoNames, UNESCO and Wikivoyage |

The vendored source files are in `data/reference/`: GeoNames' admin-1 names and city codes
(export of 2026-09-30, refreshed with `npm run city-admin1`, see `scripts/city-admin1.ts`),
UNESCO's list (retrieved 2026-07-11) and the moments.

- **Wikidata (CC0).** Every name, population, coordinate, category and UNESCO flag comes
from Wikidata. CC0 asks for nothing, but every record still names its source: its
`source` field is its Wikidata item.
Expand All @@ -45,9 +71,9 @@ build. Corrections are never made in this repository.
```sh
npm ci
npm run check # typecheck + tests
npm run build -- tests/fixtures/raw.json # build from the fixture
npm run build -- tests/fixtures/raw.json # build from the fixture (add --offline to skip Trainline)
npm run fetch -- LU MC # fetch two countries (needs query.wikidata.org)
npm run build
```

Code: AGPL-3.0. Data: CC0 1.0, from Wikidata.
Code: AGPL-3.0. Data: each dataset under its source's licence, as above.
Binary file added data/reference/admin1CodesASCII.txt.gz
Binary file not shown.
Binary file added data/reference/city-admin1.tsv.gz
Binary file not shown.
Loading
Loading