Unable to set-up an external database backend for Airflow

Astro CLI
1

Hi all,

I am running Astronomer CLI version 0.28.1 (hosted on a VM on GCP) and would like to set-up an external database backend for Airflow. The external DB is Cloud SQL (Postgres hosted on GCP), accessible through public IP from my VM.

So far, I have added the following line in my .env file (public IP of the DB masked):
AIRFLOW__CORE__SQL_ALCHEMY_CONN="postgresql://db-user:db-password@XX.XX.XX.XX/db-instance-airflow"

I also ran astro dev stop and astro dev start --env .env. However, when I log on the scheduler container (using docker exec -it), I always see the variable unchanged as below:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql://postgres:postgres@postgres:5432

I also see that the Postgres container as running when starting Astro (instead of using my DB on GCP). Any idea of what is causing this issue?
Thanks

2

Hi @sinwise did you ever solve this? I am in the same situation and I really don’t know how to do it

3

You could try adding the following to your docker-compose.override.yml file. More information in docs here.

services:
  scheduler:
    environment:
      AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql://db-user:db-password@XX.XX.XX.XX/db-instance-airflow
  webserver:
    environment:
      AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql://db-user:db-password@XX.XX.XX.XX/db-instance-airflow