Skip to content

Add team_name tags to Amazon executor metrics#69072

Open
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:multi-team-amazon-executor-metrics
Open

Add team_name tags to Amazon executor metrics#69072
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:multi-team-amazon-executor-metrics

Conversation

@justinpakzad

@justinpakzad justinpakzad commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Description:

This PR adds team_name as a tag on the task adoption duration metrics for Amazon executors in multi-team deployments.

Metrics that now include team_name:

  • batch_executor.adopt_task_instances.duration
  • ecs_executor.adopt_task_instances.duration
  • lambda_executor.adopt_task_instances.duration

When there is no team (global executors, or older Airflow versions without multi-team support) the tag is omitted (prune_dict) and the metric behaves exactly as it does today.

Tests:

Added a parametrized test for each executor that checks the timer is called with the team name tag when a team is set, and with no tag when it is not. The tests are guarded with the AIRFLOW_V_3_1_PLUS constant for compatibility.

related: #68996

Was generative AI tooling used to co-author this PR?
  • Yes (Claude Sonnet 4.6)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Jun 27, 2026
@justinpakzad justinpakzad force-pushed the multi-team-amazon-executor-metrics branch from ba92729 to 1d227f1 Compare June 27, 2026 20:09
@justinpakzad justinpakzad marked this pull request as ready for review June 28, 2026 14:18
@justinpakzad justinpakzad requested a review from o-nikolas as a code owner June 28, 2026 14:18
@justinpakzad

Copy link
Copy Markdown
Contributor Author

cc @ferruzzi

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants