We have few Airflow DAGs for our tests and we don’t want them to be deployed to the Cloud (so the Cloud will contain only users DAGs) how can we make the deploy ignore them?
Yes! You should be able to create a folder in your directory titled .airflowignore and throw those files in there to specify the directories or files in dags that Airflow should intentionally ignore. The file behaves similarly to .gitignore.
Leveraging Version Control
It’s important to note that adding an .airflowignore folder will cause any files within that folder to be ignored both if you’re developing locally and when you deploy to Astronomer Cloud - the exact same image is built and deployed in both scenarios.
In other words, there’s no easy out-of-the-box way to say “Allow these DAGs to show up locally, but only a subset to be deployed.” Or, “Deploy this directory to Cloud (with .airflowignore folder), but this directory when developing locally (without .airflowignore)” if you’re staying within the same project.
With that said, we’d encourage you to leverage version control here. You could always separate the two (or more) among git branches/CICD. Something like, “Committing to a master branch triggers a deploy to Cloud via astro airflow deploy, while my Test DAGs stay in a different branch and don’t trigger that deploy.”
Official Airflow documentation here: https://airflow.apache.org/concepts.html#airflowignore