fix(pipeline): reaped task-flow jobs no longer vanish silently#951
Merged
Conversation
Vervolg op PR #950. De orphan-reaper faalde terminale taak-flow-jobs zonder job_failed-taak: de aanvrager zag de 'Bezig'-regel verdwijnen en hoorde nooit dat het misging. De reaper geeft nu de geraakte jobs terug (inclusief payload) en notify_reaped_task_jobs maakt voor terminaal gefaalde taak-flow-jobs alsnog een job_failed-taak aan, met per jobtype de juiste titel, en ruimt de job-blobs op. Daarnaast krijgt de document-convert-verwerking dezelfde begrenzing onder de reaper-window als law-convert, zodat ook een trage agent-fallback zichtbaar faalt in plaats van gereaped te worden.
tdjager
marked this pull request as ready for review
July 18, 2026 07:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #950, addressing the robustness gap found during its smoke test.
Problem
The orphan reaper fails any job stuck in 'processing' past
orphan_timeoutwithout a heartbeat check, and it never created ajob_failedtask. For task-flow jobs (deliver: "task") that meant the requester saw the running item disappear from the Taken panel with no trace — observed repeatedly with slowlaw_convertruns on the pr950 preview, but the gap applied to all task-flow job types (enrich, document_convert, law_convert).Changes
reap_orphaned_jobsnow returns the reaped rows (id, law_id, job_type, NEW status, payload) instead of a bare count. The atomicUPDATE … RETURNINGhands each row to exactly one concurrent reaper.tasks::notify_reaped_task_jobs: for reaped jobs that ended TERMINAL and are task-flow with arequested_by, create ajob_failedtask with the job-type-appropriate title ("Verrijking mislukt", "Wet aanmaken mislukt" fornew_law, "Conversie mislukt", "Conversie naar wet mislukt") and clean up the job blobs. Retried jobs (reaped back to pending) and non-task-flow jobs are skipped. Best-effort per job so one failed insert never stops the reaper loop.document_convertprocessing gets the same below-the-reaper-window bound that feat(editor): add a law or regulation from an uploaded document in a traject #950 gavelaw_convert, so a slow agent-fallback conversion fails visibly (job_failed task) instead of being reaped.Tests
new_lawenrich variant gets the "Wet aanmaken mislukt" title.