I have an airflow dag that is suppose to run daily. I am currently backfilling i.e with catchup=True, However, I want my subsequent day’s run to start only if previous dag run is completed successfully and If previous run fails I don’t want any subsequent day’s run to trigger at all.
I have max_active_runs=1 in-place but somehow I am not sure how to achieve this. In nutshell I want next days dag to run only if todays dag run is successful.
I would really appreciate your inputs. Thank you
You can set depends_on_past=True for your first task and also, on that same task, set wait_for_downstream=True. Thus, your first task will not move forward unless the previous DAG completed. This makes some assumptions about the tree structure of your DAG, but is correct for most DAGs.
@collinmcnulty
Thank you for your response. One QQ: I am under the impression that wait_for_downstream only wait for immediate child task(s) so in this case, the next day’s dag run would only wait untill the child task is completed and not the entire downstream. Please correct if I am wrong.