Skip to content

A failed estate Job stays failed until what names it changes #461

Open

Description

@ExtraToast

What happened

The Vault policy job and the catalog method's Job are named by a digest of what they run with (#454, #459). On 2026-10-16 the first postgres-catalog-7b66ab65da64 failed, because the Vault policy job had not yet written its role. Once a fixed policy job had run, the catalog Job's inputs were unchanged, so it kept its name and stayed Failed. Flux's health check kept estate-vso-secrets not Ready, which held apps-delivery, apps-tribelt and apps-estate-dashboard, until the Job was deleted by hand. The hourly CronJob runs the same pod and succeeded, but the unit's health check reads the hash-named Job.

What should happen

A Job that failed for a reason outside its own inputs runs again without a person deleting it. Options: the unit does not health-check the one-shot Job and relies on the CronJob, the Job carries a backoffLimit and deadline long enough to outlast the policy job writing its role, or the catalog Job is ordered after the policy job.

Acceptance criteria

  • A catalog Job that fails because its Vault role does not exist yet succeeds on its own once the role exists, with no manual delete.
  • Both implementations and the oracle trees agree.

Activity

  1. added
    type: bugSomething is broken or behaving incorrectly.
    area: deployhomelab-deploy, deploy-v2, manifests, and rollout flow.
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: deployhomelab-deploy, deploy-v2, manifests, and rollout flow.status: ready-for-agentFully specified; an agent may take it with nobody watching.type: bugSomething is broken or behaving incorrectly.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions