I’m learning AirFlow and when running the following import codes on my py. file i’m having the “Broken DAG: [/usr/local/airflow/dags/file.py] No module named ‘pyodbc’” error on my Airflow webserver.
I dont know what is happening since i have pyodbc installed on my python.
Here are my imports:
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator, BranchPythonOperator
from datetime import datetime, timedelta
import pandas as pd
import pyodbc
import sqlalchemy
import zipfile
And the error on airflow webserver
[2021-03-16 15:28:14,316] {{taskinstance.py:887}} INFO - Executing <Task(PythonOperator): escreve_dw> on 2021-03-16T15:27:03.248408+00:00
[2021-03-16 15:28:14,321] {{standard_task_runner.py:53}} INFO - Started process 84 to run task
[2021-03-16 15:28:14,491] {{logging_mixin.py:112}} INFO - Running %s on host %s <TaskInstance: treino-05.escreve_dw 2021-03-16T15:27:03.248408+00:00 [running]> 8ab4027ba82f
[2021-03-16 15:28:15,437] {{taskinstance.py:1128}} ERROR - No module named 'pyodbc'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 966, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 113, in execute
return_value = self.execute_callable()
File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 118, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/usr/local/airflow/dags/treino05.py", line 168, in escreve_dw
"mssql+pyodbc://SA:FLorestas#12@127.0.0.1/ENADE?driver=ODBC+Driver+17+for+SQL+Server"
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py", line 479, in create_engine
return strategy.create(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/connectors/pyodbc.py", line 37, in dbapi
return __import__("pyodbc")
ModuleNotFoundError: No module named 'pyodbc'
[2021-03-16 15:28:15,475] {{taskinstance.py:1185}} INFO - Marking task as FAILED.dag_id=treino-05, task_id=escreve_dw, execution_date=20210316T152703, start_date=20210316T152814, end_date=20210316T152815
[2021-03-16 15:28:24,204] {{logging_mixin.py:112}} INFO - [2021-03-16 15:28:24,204] {{local_task_job.py:103}} INFO - Task exited with return code 1
If i just delete the “import pyodbc” all my tasks are running perfectly, but i need to send the results to a My SQL Server.