Skip to content

ProviderStatus workflow fails when any single provider activity fails #286

@jorgecuesta

Description

@jorgecuesta

Problem

ProviderStatus uses Promise.all to fan out per-provider activities. If any one of the processProviderStatus activities exhausts its retries and rejects, the whole workflow execution is marked Failed, even when the other providers' activities completed successfully.

In practice this happens every minute on mainnet — a few external provider endpoints regularly return HTML or "Bad Gateway" instead of JSON (vido.info, weaversnodes.com, poktpool.com, nodefleet.org, kleomed.es). Result: ~100% of ProviderStatus workflow runs end as Failed despite most provider activities succeeding.

Source

apps/middleman-workflows/src/workflows/ProviderStatus.ts:17

await Promise.all(
  providers.map((provider) => processProviderStatus(provider.identity))
);

Suggested fix

Swap Promise.all for Promise.allSettled so providers fail independently:

await Promise.allSettled(
  providers.map((provider) => processProviderStatus(provider.identity))
);

Workflow ends as Completed; individual provider failures remain visible as ActivityTaskFailed events in the workflow history.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions