Hi, I use my airflow inside a docker image, provided in airflow documentation itself (apache/airflow:2.5.2), and now I need to send the airflow dags log to an external elasticsearch instance. I didn’t manage how to do it, since the documentation asks me to set some options in airflow config file and some of the options does not have a related enviroment variable linked to it.
I would like to know if anyone here have done it before to give me some light to this problem i’m facing.
Hey @DETONsith
Each of the configuration provided in Airflow is available through the environment variables. For example, based on the latest documentation on Logging, the environment variables will be:
AIRFLOW__LOGGING__REMOTE_LOGGING
AIRFLOW__ELASTICSEARCH_HOST
AIRFLOW__ELASTICSEARCH_WRITE_STDOUT
and so on…
Basically, the structure is AIRFLOW__<name of the config section>__<name of the config property> .
To learn more about Airflow Logging with examples, see the docs on Astronomer..
Thanks
Manmeet