Repository navigation
Tell failed work apart by cause, dismiss it, and retry plan refusals on upgrade - #298
Merged
Merged
Conversation
…on upgrade Every project or clip that fails now records why: platform (the instance's fault, worth a Retry after the fix), input (the video, link or settings, which fail the same way again) or quota (the owner's plan). One helper, failedRow, is the only way a row becomes failed, and a NonRetryableError names its own cause. A migration fills in the cause of existing failures. Admins can dismiss failures from Settings > System > Queue (and restore them), without touching the owner's project. Bulk retry by filter takes platform failures unless a cause is named. Plan refusals from the last 30 days are retried when the owner subscribes, changes plan, gets a complimentary plan or has their limits raised. Same actions over MCP (dismiss_failures, cause filters) and the CLI (dismiss-failures, retry-failures --cause). Signed-off-by: dillonstreator <dillonstreator@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…pgrade retry - An expired source and "no usable clips" are platform failures: the project's Retry, or a model fix, can clear them. NonRetryableError takes its cause as a constructor argument. Retrying a failed clip of a project whose source expired goes through the project, which fetches it again. - The plan-change retry follows only changes to the minutes limits, and does nothing when even a one-minute video would be refused now. - retryProject takes a per-project retry lock and, for admin and automatic retries, only resumes a project that is still failed, so two retries never both start a run. - Issue-report import and the failed dev scenario record a cause; a test fails when a status "failed" write leaves the cause out. - No failures.dismissed event when a dismiss changes nothing; the MCP schemas use FAILURE_CAUSES. Signed-off-by: dillonstreator <dillonstreator@gmail.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The admin Failed work list mixed failures a Retry can fix (disk full, model down) with ones it cannot (a dead link, a video with no audio, nothing worth clipping, a plan refusal), offered Retry on all of them, and had no way to clear the ones nobody will retry. The pipeline already knew which errors were non-retryable (
NonRetryableError) but only used that to skip pg-boss retries; the row kept just the message. And a video refused for its length stayed failed after the owner upgraded.What
projects/clipsgetfailure_cause(platform|input|quota,FailureCausein@reelsaw/shared).failedRow(err)inpipeline/common.tsis now the only way a row becomes failed;NonRetryableErrorcarriesfailureCause(input by default, quota forQuotaExceededError), anything else is platform. Migration 0059 backfills existing failures from their recorded wording (the minutes refusal, no audio, no usable clips, dead/private links, live streams); the rest read as platform.failure_dismissed_athides a failure from the admin list only; the owner's project, error and Retry are untouched, and a new failure clears it. Restore brings it back.dismissFailuresinops/failures.ts, eventfailures.dismissed.retryRefusedWorkonuser.limits_changed(open) andbilling.subscribed/billing.changed/billing.complimentary(ee) resumes the owner's quota failures from the last 30 days throughretryProject; ingest re-checks the minutes, so a change that still doesn't cover the video fails it once more. The handler's key is null when the account has none, so most billing events enqueue nothing.dismiss_failures,causeonlist_failures/retry_failures,cause/retryable/dismissedAton items. CLI:dismiss-failures,retry-failures --cause. README, TROUBLESHOOTING and a changeset updated.Testing
ops-failures.test.ts: cause classification, cause filter, bulk retry skipping input/quota, dismiss/restore/re-fail, and the quota auto-retry (recent only, not other causes); handler map and MCP definition tests updated. Run against a scratch Postgres with this branch's migrations; server and web typecheck, lint clean.input; dismiss, show dismissed and restore behaved; raising the owner's monthly minutes resumed only the plan-limit failure.