Airflow DAG scheduler fails to trigger tasks on version 2.4.1

[Umesh7987]

Apache Airflow version

2.10.4

If "Other Airflow 2 version" selected, which one?

Apache Airflow 2.4.1

What happened?

The scheduler fails to trigger tasks on certain DAGs. The tasks remain in the ‘queued’ state indefinitely without executing. This started after upgrading to Airflow 2.4.1.

What you think should happen instead?

The tasks should transition from the ‘queued’ state to the ‘running’ state, and be executed as per the DAG schedule.

How to reproduce

Provide clear, step-by-step instructions to reproduce the problem. Example:

Install Apache Airflow 2.4.1.
Create a simple DAG with two tasks.
Trigger the DAG and observe that the tasks stay in the ‘queued’ state without execution.

Operating System

Ubuntu 20.04

Versions of Apache Airflow Providers

Ubuntu 20.04

Deployment

Official Apache Airflow Helm Chart

Deployment details

apache-airflow-providers-google 2.0.1

Anything else?

Deployed using Helm chart on Kubernetes with 3 replicas
This issue occurs every time the DAG is triggered after the upgrade to 2.4.1. Logs indicate the scheduler is not picking up the task in the ‘queued’ state

[2025-01-22 12:34:56] ERROR task_scheduler: Task stuck in 'queued' state

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Without more detailed logs and explanation, there is no way to help you. Converting to a discussion, but you should provide more details in a new issue, enough to be able to see related logs and what happens.

[potiuk]

Also we are flooded by AI generated spam issues, so you need to provide all those details to prove that you are actual human.