Hi We are going to use Airflow soon so i am doing a POC on scheduling an dag to run 3 times a day first - 3:30 am CST second - 11:30 am CST and Third - 4:00PM CST could you please help me to schedule or create a custom calendar in Airflow.
Hi @kumar.8ds- welcome to our community! Are you familiar with the concept of DAG Runs in Airflow? You can schedule DAG runs by pasing a cron expression as a string to your schedule_interval DAG argument. If you’d like some help writing cron expressions, there are some nifty tools out there like Crontab Guru that will do the trick.
cron can do stuff like “every X hours”, say every 8 hours = 3 times a day but since @kumar.8ds’s use case requires uneven intervals, i did this in my DAG (using his example, run at 3:30am CST)
task1
delay_7_hours >> task2
delay_11.5_hours >> task3
task1~3 are the same task with different task ids
also, you don’t want your tasks to hang around doing nothing for 11 hours, so don’t just use sleep, instead, use the TimeDeltaSensor
2 Likes
Unfortunately, cronstrings are unable to represent a schedule with differing hours and minutes.
@zachliu’s solution works great! He is leveraging the TimeDeltaSensor to wait a certain time before executing the task, which is essentially a makeshift schedule. The only “downside” is that you will have three tasks that does the same thing.
Thanks for your response i am able to schedule it now.
1 Like