Skip to content

fix(pipeline): reaped task-flow jobs no longer vanish silently#951

Merged
tdjager merged 1 commit into
mainfrom
fix/taakflow-reaper-zichtbaarheid
Jul 20, 2026
Merged

fix(pipeline): reaped task-flow jobs no longer vanish silently#951
tdjager merged 1 commit into
mainfrom
fix/taakflow-reaper-zichtbaarheid

Conversation

@tdjager

@tdjager tdjager commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #950, addressing the robustness gap found during its smoke test.

Problem

The orphan reaper fails any job stuck in 'processing' past orphan_timeout without a heartbeat check, and it never created a job_failed task. For task-flow jobs (deliver: "task") that meant the requester saw the running item disappear from the Taken panel with no trace — observed repeatedly with slow law_convert runs on the pr950 preview, but the gap applied to all task-flow job types (enrich, document_convert, law_convert).

Changes

  • reap_orphaned_jobs now returns the reaped rows (id, law_id, job_type, NEW status, payload) instead of a bare count. The atomic UPDATE … RETURNING hands each row to exactly one concurrent reaper.
  • New tasks::notify_reaped_task_jobs: for reaped jobs that ended TERMINAL and are task-flow with a requested_by, create a job_failed task with the job-type-appropriate title ("Verrijking mislukt", "Wet aanmaken mislukt" for new_law, "Conversie mislukt", "Conversie naar wet mislukt") and clean up the job blobs. Retried jobs (reaped back to pending) and non-task-flow jobs are skipped. Best-effort per job so one failed insert never stops the reaper loop.
  • document_convert processing gets the same below-the-reaper-window bound that feat(editor): add a law or regulation from an uploaded document in a traject #950 gave law_convert, so a slow agent-fallback conversion fails visibly (job_failed task) instead of being reaped.

Tests

  • Adapted the existing reaper tests to the new return type (and asserting the returned rows).
  • New DB tests: terminal task-flow law_convert gets a task while a retryable enrich and a non-task-flow harvest do not; the new_law enrich variant gets the "Wet aanmaken mislukt" title.

Vervolg op PR #950. De orphan-reaper faalde terminale taak-flow-jobs
zonder job_failed-taak: de aanvrager zag de 'Bezig'-regel verdwijnen en
hoorde nooit dat het misging. De reaper geeft nu de geraakte jobs terug
(inclusief payload) en notify_reaped_task_jobs maakt voor terminaal
gefaalde taak-flow-jobs alsnog een job_failed-taak aan, met per jobtype
de juiste titel, en ruimt de job-blobs op. Daarnaast krijgt de
document-convert-verwerking dezelfde begrenzing onder de reaper-window
als law-convert, zodat ook een trage agent-fallback zichtbaar faalt in
plaats van gereaped te worden.
@tdjager
tdjager marked this pull request as ready for review July 18, 2026 07:54
@tdjager
tdjager merged commit c9afae4 into main Jul 20, 2026
32 checks passed
@tdjager
tdjager deleted the fix/taakflow-reaper-zichtbaarheid branch July 20, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant