Use a master DAG to trigger and sense completion of other dags

Airflow
1

Was trying out a combination of TriggerDagRunOperator and ExternalTaskSensor to get this to work.

The scenario has 3 dags:

dag1 : task1 >> task2
dag2 : task1
masterdag : [ trigger_dag1 , sense_dag1_task_2] >> trigger_dag2

But the sensor is not being able to pick up on the dag1_task2 being completed. I have used a reschedule mode and tried running it both manually and on a schedule (the masterdag on schedule, dag1 dag2 have schedule_interval=None.

2

In the log emitted by ExternalTaskSensor, which dag id and task id is it looking for?

It doesn’t matter whether it is triggered manually or by schedule. Since you are sensing on the DAGs triggered by the DAG doing the sensing, the execution date should be the same, which means you don’t need to set a execution delta.

It would be helpful if you can provide your DAG with information redacted if need be.