I tried to create a basic astronomer project, as per: Create an Astronomer Project | Astronomer Cloud
Dockerfile has this line: FROM quay.io/astronomer/ap-airflow:2.2.3-onbuild
astro dev start and local instance has the example dag running!
When I try to deploy this very project to our production installation I get these errors and airflow doesn’t startup:
02/01 10:00:58Traceback (most recent call last): File “/usr/local/lib/python3.9/site-packages/airflow/settings.py”, line 450, in import_local_settings import airflow_local_settings File “/usr/local/airflow/config/airflow_local_settings.py”, line 2, in from airflow.contrib.kubernetes.pod import Pod ModuleNotFoundError: No module named ‘airflow.contrib.kubernetes’
Has anyone have any idea what might be wrong?
I’m getting the exact same error but with another DAG I’m wroking on… Were you able to solve it by any chance?
Broken DAG: [/usr/local/airflow/dags/cells_polygon_stores/cells_polygon_stores.py] Traceback (most recent call last):
File "/usr/local/airflow/dags/cells_polygon_stores/cells_polygon_stores.py", line 34, in <module>
from dags.utils.ml_operator_sup import MLOperator
File "/usr/local/airflow/dags/utils/ml_operator_sup.py", line 5, in <module>
from airflow.contrib.kubernetes.pod import Resources
ModuleNotFoundError: No module named 'airflow.contrib.kubernetes'
Might not be your problem.
In my case the problem was that our Astronomer installation had not supported Airflow 2.x. Once we created a new instance that does, everything works as it should.
Hi @alainmore & @biku did you add
apache-airflow[kubernetes] to your
requirements.txt before deploying?
This will pip install the package
I tried that but it did not help because my deployment environment wasn’t 2.x. With 2.x I did not need to add it to requirements.txt