Does anyone have thoughts on how to best maintain DAG release and versioning?
This Stackoverflow post (https://stackoverflow.com/questions/48378712/efficient-way-to-deploy-dag-files-on-airflow) looks like a reasonable approach, but involves having to adjust the version of the DAG manually but updating the dag_id in the DAG file. If you version control the code using git or something similar, this approach may lead to inconsistencies in git versions and DAG versions.
Keeping DAGs version-less is a good option, but we’ve noticed that deploying a DAG file to our system’s airflow/dags/ folder will impact both the contents and the shape of currently running DAGs. In an ideal world, currently running DAGs would not be modified once they started.
As this likely affects most of us in the community, would love to hear any thoughts, opinions or best practices you’ve learned.