Securing sensitive values in Airflow

We are masking variables that has sensitive values using Airflow masking using following code.

from airflow.utils.log.secrets_masker import mask_secret


We are then passing these masked variables as op_kwargs to callable function in PythonOperator.

This secures (masked as ***) the value in “op_kwargs” in “Rendered Template” in Airflow UI. However the values are clear in “Task Instance Details”.

Please suggest how can we mask the value in Task instance.

Hi @krishanu, thanks for reaching out!

A good way to go about it would be to leverage an environment variable AIRFLOW__CORE__SENSITIVE_VAR_CONN_NAMES where you can pass a comma-separated list of extra sensitive keywords to look for in variables names or connection’s extra JSON.

This way you will see *** both in the Rendered Template and Task Instance Details.

HI @magdagultekin, I am also facing a similar kind of issue.
I set the following astro env variables with the given values
But still the airflow connections extra json’s private key remains a visible text

Hi @ashaj, welcome to the community!

Did you mean that it remains a visible text in the Airflow UI? If so - unfortunately, you won’t be able to mask it, extras will be visible.

To keep it as a secret, I’d suggest looking at secrets backend.