Define gcp connections in `airflow_settings.yaml`?

I did this

    ....
    - conn_id: google_cloud_default
      conn_type: google_cloud_platform
      project_id: test_project
      keyfile_json: 'xxxxx'

then astro dev start without issues. I could see the connection google_cloud_default in Admin>Connections but somehow the “Project ID” field is not populated. Anything wrong with the yaml above? I was thinking I may have used a wrong key on “project_id”, but can’t be sure - this spec here isn’t super helpful either.

1 Like

Hi @shawnx! You should be able to add a conn_extra field and then the project_id under it. Here’s an example of what your airflow_settings.yaml might look like:

airflow:
  connections:
    - conn_id:
      conn_type:
      conn_host:
      conn_schema: 
      conn_login:
      conn_password:
      conn_port:
      conn_extra:
        extra__google_cloud_platform__project: test_project

Note: conn_extra should be a string and will be JSON encoded.

Here’s an example of how you might do it via the Airflow CLI:

❯ airflow connections --add --conn_id example_gcp --conn_type google_cloud_platform --conn_extra 'extra__google_cloud_platform__key_path=%2Fkeys%2Fkey.json&extra__google_cloud_platform__scope=https%3A%2F%[2Fwww.googleapis.com](http://2fwww.googleapis.com/)%2Fauth%2Fcloud-platform&extra__google_cloud_platform__project=airflow&extra__google_cloud_platform__num_retries=5'

successfully added `conn_id`=example_gcp : google_cloud_platform://:******@:

Want to give it a shot and let us know if this works?

Thanks paola, this snippet below doesn’t work.

airflow:
  connections:
    - conn_id: google_cloud_default
      conn_type: google_cloud_platform
      keyfile_json: '{....}'
      conn_extra:
        extra__google_cloud_platform__project: test_project

No error complaints on >astro dev start, and I can see the connection in airflow UI, but when I dig into the setting in “Admin > Connections”, I see “Project Id” still empty. Anything I did wrong?

Also, (maybe a n00b question) how did you get into airflow CLI and be able to run >airflow connections --add ...? Thanks

I’m wondering this same thing. I’m working through this right now, so any guidance would be greatly appreciated!