Aphia LDES mirror

This repository contains the code and configuration to produce and publish a mirrored LDES for the Aphia dataset also known as the World Register of Marine Species (WoRMS).

The Aphia dataset is maintained and already published as a Linked Data Event Stream (LDES) by VLIZ on https://aphia.org/feed. However, our republishing of this dataset is only intended as an academic exercise to study alternative LDES data structures and their impact on replication efficiency. Also as a demonstrator of how a derived LDES may be generated and published.

To republish the Aphia LDES, we define a data pipeline using the RDF-Connect framework.

The Aphia LDES mirror is available online at https://ldes.ilabt.imec.be/aphia-mirror.

What is different from the original?

• LDES structure: TODO: explain that we use a B+Tree-based fragmentation for higher replication efficiency and better traversability

System components and architecture

TODO: Diagram and description of pipeline components.

How to run it?

This pipeline and the necessary data storage and interface components are containerized using Docker and can be executed altogether using docker-compose as follows:

$ docker-compose up --build 

The .env file contains the main configuration variables to be set.