mo-arbor

OS2mo integration that syncs organisation units from CSV files, such as SOR.

Usage

docker compose up -d --build

Configuration is done through environment variables. Integration-specific options are defined in mo_arbor/config.py. FastRAMQPI's options (MO URL, authentication, logging, etc.) are nested under FASTRAMQPI and set using __ as delimiter, e.g. FASTRAMQPI__MO_URL. See docker-compose.yml for a working example.

Code systems

A code system, such as SOR, is a tree of codes, and each code becomes an org unit in MO. Code systems are configured in CODE_SYSTEMS as a JSON list, and name is one the integration knows: sor or okosted.

The supplier uploads dumps to MO's EXPORTS file store, with the time of the dump in the file name (file_pattern). A new dump needs a new time in its name, as a dump uploaded again under the same name is not read again, unless it was invalid. A dump is a CSV file as described in the supplier's spec "Kodesystemfiler til OS2mo" (#69230):

• The rows without a kode_parent are roots, which become the top units in MO.
• Every row must have a type and a niveau.
• A unit is valid from gyldig_fra to gyldig_til, cut to its parent's validity. A unit that is gone from the dump is terminated.
• aktiv, dw_is_deleted and the timestamps are not used, and a file with other columns is rejected.
• SOR codes must be valid SOR IDs, which catches codes damaged by Excel.

Ingest

A cron job calls POST /ingest. For each code system, it compares the newest dump with the last ingested one, and sends an org_unit event for every unit that was added, changed or removed. Dumps in between are skipped, and the first ingest sends every unit.

Ingest skips a code system and logs an error when:

• the newest dump is invalid,
• it would terminate more than max_terminations units (override with ?force=true),
• the last ingested dump is gone or invalid, as the removed codes cannot be found without it.

The other code systems are still ingested, but the request fails, so the cron job fails.

?dry_run=true shows what would be sent without sending it, and ?code_system=<name> ingests only that code system.

Sundhedsvæsenets Organisationsregister (SOR)

SOR registers organisation and address data, as well as location numbers, for the Danish healthcare sector. It is owned and operated by Sundhedsdatastyrelsen.

Development

The integration is a FastRAMQPI application using OS2mo's GraphQL event system. FastRAMQPI declares event listeners against MO; MO's GraphQL fetcher polls events and HTTP-POSTs them to FastAPI routes on this integration. Listeners are configured in mo_arbor/app.py and the handlers live in mo_arbor/events.py.

The development environment connects to the os2mo_default network, so OS2mo must be running locally.

Running the tests

Unit tests:

poetry run pytest -m "not integration_test"

Integration tests run against the local OS2mo, from inside the container:

docker compose run --rm mo-arbor pytest -m integration_test

Updating the GraphQL client

The GraphQL client in mo_arbor/autogenerated_graphql_client MUST NOT be edited manually. It is generated from schema.graphql and the operations in queries/, using ariadne-codegen. Operations used only by the tests are in queries/testing.graphql. To regenerate after changing queries:

curl -O http://localhost:5000/graphql/v30/schema.graphql
rm -rf mo_arbor/autogenerated_graphql_client/
poetry run ariadne-codegen client
poetry run pre-commit run --files mo_arbor/autogenerated_graphql_client/*

Versioning

This project uses Semantic Versioning with the following strategy:

• MAJOR: Incompatible API changes.
• MINOR: Backwards-compatible updates and functionality.
• PATCH: Backwards-compatible bug fixes.

Authors

Magenta ApS https://magenta.dk

License

• This project: MPL-2.0

This project uses REUSE for licensing. All licenses can be found in the LICENSES folder of the project.