Securing sensitive values in Airflow

We are masking variables that has sensitive values using Airflow masking using following code.

from airflow.utils.log.secrets_masker import mask_secret


We are then passing these masked variables as op_kwargs to callable function in PythonOperator.

This secures (masked as ***) the value in “op_kwargs” in “Rendered Template” in Airflow UI. However the values are clear in “Task Instance Details”.

Please suggest how can we mask the value in Task instance.

Hi @krishanu, thanks for reaching out!

A good way to go about it would be to leverage an environment variable AIRFLOW__CORE__SENSITIVE_VAR_CONN_NAMES where you can pass a comma-separated list of extra sensitive keywords to look for in variables names or connection’s extra JSON.

This way you will see *** both in the Rendered Template and Task Instance Details.