Skip to content

airflow db clean fails with --dag-ids / --exclude-dag-ids #69030

Description

@devgonvarun

Apache Airflow version

3.2.2

What happened and how to reproduce it?

When running airflow db clean with either --dag-ids or --exclude-dag-ids, the command fails with RuntimeError: airflow db clean encountered errors on the following tables and did not clean them: ['asset_event', 'deadline', 'task_reschedule'].

Steps to reproduce

  1. Run the cleanup command without DAG filtering:
airflow db clean \
  --clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
  --error-on-cleanup-failure \
  --skip-archive \
  --tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
  -y

This completes successfully.

  1. Run the same command with --dag-ids:
airflow db clean \
  --clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
  --dag-ids "your_dag_id" \
  --error-on-cleanup-failure \
  --skip-archive \
  --tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
  -y
  1. The command fails while cleaning asset_event, deadline, and task_reschedule.

  2. The same happens when using --exclude-dag-ids.

What you think should happen instead?

airflow db clean should successfully respect --dag-ids / --exclude-dag-ids.

In particular, it should not fail on asset_event, deadline, or task_reschedule when the command is otherwise valid and the same run works without DAG filtering.

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Official Apache Airflow Helm Chart

Official Helm Chart version

1.21.0

Anything else?

Notes:

  • --dag-ids and --exclude-dag-ids are both documented by airflow db clean --help.
  • The failure only occurs when those flags are used.
  • The command works without DAG filtering.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:CLIarea:MetaDBMeta Database related issues.kind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions