The Astronomer CLI runs docker-compose under the hood (and uses this template to start up).
When you initialize an Airflow project on Astronomer, 3 Docker containers are spun up on your machine, each responsible for a component:
- Postgres: Airflow’s Metadata Database
- Scheduler: The Airflow component responsible for rendering the Airflow UI
- Webserver: The Airflow component responsible for monitoring and triggering tasks
To verify these containers on your machine, you can always run: docker ps
Access on Astronomer Cloud
Currently, all Docker container access on an image pushed to an Astronomer Cloud deployment is purely local. For security reasons, they’ll be inaccessible once you’ve pushed up the image to Astronomer Cloud.
Note: On Astronomer Cloud’s coming release, all Worker/Scheduler/Webserver logs will be directly exposed in the Astronomer UI - giving you the transparency into core processes that bashing into containers would otherwise give you.
Local Access
For more detail on what you can run locally, check out the “Logs and Docker Containers” section in our CLI Getting Started guide.
To see scheduler logs on your local docker container, for example, you can run:
docker logs $(docker ps | grep scheduler | awk '{print $1}')
OR
To enter one of these containers directly, you can run:
docker exec -it <containerid> /bin/bash