Customer Question:
On daily tasks, using
ds(an Airflow Variable that allows you to specify execution date) makes sense because we need to process the data of the previous day.
When we use weekly scheduling,dsprocesses data that was completed 1 week ago from today, so in truth data that we received the prior week. In reality, we need to process data from the past 7 days.
Would you know what Airflow macro to use or scheduling strategy to apply on this cases? Should we be using{{ next_ds }}?
Answer:
Try: dt BETWEEN '{{ ds }}' AND '{{ next_ds }}'
You could otherwise do something like, {{ (execution_date - macros.timedelta(days=7) }} but that’d still give you data from 2 weeks ago.