Is there a consensus on the most secure way to store AWS Auth information (keys, etc.)?


Most AWS services in Airflow (such as the S3Hook) inherit from and build off of the common AWSHook to make connections, which just relies on typical access_keys/secrets.We generally see the aws_access_key_id and aws_secret_access_key used to access AWS resources.

These can either be stored together in a dict in the extras field of your connection where they will be encrypted at REST, or you can put the aws_access_key_id in the login field and then the aws_secret_access_key in the password field where your aws_secret_access_key will be encrypted at REST and also inaccessible from the UI (still present but not shown after save).

We haven’t personally encountered anyone using KMS but it might be worth to check out the modified AWSkmsHook and AWS Secret Manager Hook to do so. Note that you’ll then need to also modify the S3, EMR, etc. hooks to inherit from here rather than the typical AWS Hook.