Dependency error with dbt-core/dbt-snowflake and Airflow 2.3+ | `flask 2.2.2 requires Werkzeug>=2.2.2, but you have werkzeug 2.1.2 which is incompatible`

Astro CLI
1

Problem

I upgraded to Airflow 2.3 and started getting errors related to the dbt-snowflake and dbt-core pip packages in my requirements.txt file. The Airflow webserver/scheduler seems to fail with the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
flask 2.2.2 requires Jinja2>=3.0, but you have jinja2 2.11.3 which is incompatible.
flask 2.2.2 requires Werkzeug>=2.2.2, but you have werkzeug 2.1.2 which is incompatible.

Root Cause

The latest stable versions of dbt-core and dbt-snowflake pip packages are currently incompatible with Airflow 2.3+.

Fix

We encourage all users to continue to upgrade to the latest versions of Airflow. To upgrade to Airflow 2.3 or later with dbt-core or dbt-snowflake, you can do one of the following:

  • Install dbt-core==1.3.0b2 or dbt-snowflake==1.3.0b1 instead in your requirements.txt file. These versions of the packages are in beta and have not been tested by Astronomer.
  • Install the dbt Cloud provider package by adding apache-airflow-providers-dbt-cloud to your Astro project. This will work only if you use dbt Cloud.
  • Use the KubernetesPodOperator or the ExternalPythonOperator to isolate dbt-core from the rest of your dependencies.

If you’re running on Astro, see Runtime Upgrade Documentation and Release Notes.

More information coming soon as this problem is resolved in the open source Airflow project.

1 Like