Astronomer Conda Packages

I’m trying to run a DAG which uses geopandas as one of its imports. I was able to install geopandas via Conda on the astronomerinc/ap-airflow:1.10.7-alpine3.10-onbuild Docker image after getting Conda installed. However, when I try to import it in the DAG, it cannot find the module.

Doing a “docker exec -it CONTAINER_ID python” then “import geopandas” does successfully run so it is installed.

When I run ps inside the webserver container I noticed, it’s using the default install of python
{/usr/bin/python} /usr/bin/python3.7 /usr/bin/airflow webserver
and not /opt/conda/bin/python which I’ve installed and has the geopandas library.

Is there any way to have airflow use conda packages? Or have conda install packages to the base python install(I’m not sure if this is even possible).

Note: Installing geopandas with pip on alpine Linux does not appear to be possible. It’s strongly recommended to use conda if possible.

Thanks,
Chris

Hey @Chrisreed278! Interesting that geopandas can’t be installed on Alpine via pip. Have you tried our debian base image? You might have more luck getting that package installed that way- you can use astronomerinc/ap-airflow:1.10.7-buster-onbuild if you’d like to give that a shot.

Let me know how it goes!

1 Like