Hey, I am using astro cli for local development and the start up time of airflow is starting to become an issue because it takes several minutes before the scheduler becomes available.
Looking at the logs it looks like this is due to airflow db upgrade being executed everytime I run astro dev start or astro dev restart. After db migrate is done, the scheduler is healthy and continuous parsing of DAGs doesn’t seem to cause any problems.
I initially thought that this was due to astonomer-cosmos inefficiently parsing our dbt project, however, I switched it over to using a preparsed Manifest with only slightly improved the situation.
Here’s my questions:
Is this intended behavior? I was of the impression, that db upgrade/db migrate is only needed when initializing the metadata db or upgrading to a new airflow version. If my assumption is true, I would prefer that it is only executed when running astro dev start.
In case this is the intended behavior, is there a way to debug/benchmark this to identify where there is optimization potential?
Many thanks