Hi @jonpie! We ended up sorting this out via a private support ticket with our team, but for anyone else following - the issue here was rooted in an underprovisioned Webserver.
In this case,
- Webserver for a deployment on Astronomer Cloud was configured to 2AU
- An underprovisioned Webserver caused unexpected behavior in the Airflow UI, including a message that erroneously signaled a missing DAG (that was confirmed to be in the right directory)
- Scaling the Webserver to 5AU via the Astronomer UI (and pushing a deploy via the CLI) resolved the issue
What we’re doing to improve the user experience as it related to managing Airflow’s Webserver on Astronomer:
- Adjusting the default Webserver configuration on Astronomer Cloud to 5AU
- Exposing Webserver/Scheduler/Worker logs in the Astronomer UI (previously unavailable)
- Improved documentation on recommended resource configurations on Astronomer at scale
These changes will be incorporated in our upcoming upgrade to Astronomer Cloud. If you’re seeing related issues, feel free to also reference this forum post for more detail on symptoms and other potential fixes.
To reach our technical support team, please reach out to us at support@astronomer.io or submit a request at support.astronomer.io