Unexpected interruptions combined with the saved state of the ldes-client, can lead to the loss of members due to the interruptions happening after the members have been labeled as emitted in the ldes-client, but before they have been written down into the target graph store by the sparql-ingest processor.
A solution for this could be to adjust the ldes-client to only label a member as emitted after getting a reception ACK by the next processor, plus adding a state management mechanism in sparql-ingest that upon interruption, records received members that weren't written yet.
Unexpected interruptions combined with the saved state of the
ldes-client, can lead to the loss of members due to the interruptions happening after the members have been labeled as emitted in theldes-client, but before they have been written down into the target graph store by thesparql-ingestprocessor.A solution for this could be to adjust the
ldes-clientto only label a member as emitted after getting a reception ACK by the next processor, plus adding a state management mechanism insparql-ingestthat upon interruption, records received members that weren't written yet.