How do I terraform airflow connections?

Astronomer Astronomer Nebula
1

We have a terraformed setup, mostly. But the IAM user, group, and policies for airflow to connect to aws, e.g. for the EcsOperator, are not terraformed.

A desirable state for me would be: terraformed IAM user, where the aws connection information, including access key ID and secret, are seeded into the airflow connections. Ideally where secrets remain encrypted, e.g. not injected into the airflow container image as part of the docker build process.

Currently: Our CI tools builds and deploys our astronomer airflow container to astronomer cloud, and I have manually added aws connection info through the airflow connections UI. I manually maintain the user and group policies to enable containers started by airflow to have the correct permissions.

2

Hey @mplovepop, what would be the ideal way for you to get the terraformed IAM user secret injected into your airflow environment in Astronomer Cloud? Is this something you’d like to see the Airflow API support, or are you thinking something else?

3

I don’t know enough to answer your question, sadly. I am looking to see if others have solved this already, and if so, how? Failing that, any ideas are welcome. Our current setup works, but it’s more manual setup than I’d like and doesn’t fit in well with our attempts to have automatically created infrastructure.