I am moving my DAGs from Airflow hosted on a single GCP vm to Astronomer
We have this maintenance DAG that cleans up Airflow’s backend database with some retention period
This is the DAG we use
Is this something that I no longer need using Astronomer? does the airflow backend has a retention policy by default? if yes, where can I find out what is the retention period?
I just looked into this a couple months ago and couldn’t find anything saying that Astronomer cleans up the meta database automatically, so i actually implemented that same DAG in my deployments.
Hey @gilboa-reif and @Tgoad! Thanks for reaching out on this. Astronomer products currently do not do any automatic metadata cleanup, but it’s definitely on our roadmap.
We’ve looked at those DAGs from Google – they’re super nice. I’d recommend sticking to those for now until it’s native to our products.
Is the 30-day mark reasonable for your team? Would you like that to be configurable?
Hey @Paola, I will always vote for configurable, but 30 sounds like a solid default value.