I’ve been trying to install
spacy without success, by adding them to requirements.txt. When running
astro airflow start I get the following output and then the process seems to just hang.
Sending build context to Docker daemon 236.5kB
Step 1/13 : FROM astronomerinc/ap-airflow:0.9.0-1.10.3-onbuild
# Executing 5 build triggers
---> Using cache
---> Using cache
---> Running in a27916d656c4
Here’s the contents of packages.txt:
Ok so it turns out that the process didn’t actually freeze but rather was compiling the source distributions of the python packages, which takes a long time. Am I correct to assume that there are no pre-compiled binaries of
spacy available for Linux Alpine, and that this is what causes the long installations?
I think that’s correct. And yes, it takes quite a long time. We are testing out a Beta image on Debian instead of Alpine if that’s something you’re interested in trying.
I would love to give that a try. How do I proceed?
@miroli Hi there! If you’re interested in testing out our Debian image, you can go ahead and throw the following in your Dockerfile:
If you have any issues, feel free to reply back in here or email us directly at firstname.lastname@example.org
I also am interested in using the Debian image. I was able to build the dockerfile but when I clicked run on any of the DAGs I got this error: “AttributeError: ‘NoneType’ object has no attribute ‘create_dagrun’”
Thanks for your help,
This is usually due to an airfow version mismatch. Try
astro airflow kill and
astro airflow start and you should be good to go!
Hi @paola. Is there any official version of the Debian Image?
Hi @gonzalodiazrappi! We should have a Debian image available for testing in the next few days, we’re very excited to get this released and to get feedback on it as it should address many of the common OS-level package dependency issues encountered when installing python libraries in Alpine.
Hi all! Quick update here that the Astronomer Platform (v0.11+) now officially supports a Debian Image, alongside Alpine, as a part of our Airflow build process.
A few notes:
- At the time of this post, we support a Debian image for Airflow 1.10.5, 1.10.6 and 1.10.7 (1.10.10 coming soon)
- Alpine continues to be our default for local development + new projects created on
astro dev init, but we envisioning transitioning to Debian by default in the future
- For our list of images and instructions on how to implement, check out this doc: https://www.astronomer.io/docs/airflow-versioning/
- In addition to our docs, you can always find all available Docker images in Astronomer’s DockerHub repo