TriggerDagRunOperator Skip Paused DAGs

Airflow
1

Hi,
I would like to trigger multiple child DAGs through a master DAG. Once all children are done, a final DAG should run.
The problem is that some child DAGs may be paused. The Master DAG triggers these child DAGs anyway, but they never finish.
image
Is there a possibility that the TriggerDagRunOperator skips paused DAGs?
If I have to write a custom operator, how do I find out if a DAG is paused or active?

Thanks :slight_smile:

2

Did you ever find a solution? I’m running into the same issue.

3

Hi @kenfu1010, thanks for reaching out.

You can check whether a DAG is paused or not using Airflow REST API (see Get basic information about a DAG section).

An example Python task below is based on Astro Deployment. As a result, you’d get a True or False.

@task
def check_dag_info(deployment_url, dag_id, token):
    import requests

    response = requests.get(
        url=f"{deployment_url}/api/v1/dags/{dag_id}",
        headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
    )
    return response.json()["is_paused"]

This guide goes into more details on how to make requests to the Airflow REST API on Astro in case you’re interested.

Alternatively, if you’re developing locally using Astro CLI, this should be helpful (check this guide for more):

@task
def check_dag_info(dag_id):
    import requests

    response = requests.get(
        url=f"http://host.docker.internal:8080/api/v1/dags/{dag_id}",
        auth=("admin", "admin")
    )
    return response.json()["is_paused"]
1 Like
4

Thanks @magdagultekin!