When adding python libraries to your requirements.txt file you may see an error similar to this (likely with a lot more detail above):
The command '/bin/sh -c pip install --no-cache-dir -q -r requirements.txt' returned a non-zero code: 1
Error: command 'docker build -t astro-local_a85b22/airflow:latest failed: failed to execute cmd: exit status 1
A likely cause for these errors is the Alpine distro of Linux missing some OS-level packages required to install your desired python libraries.
A logical next step might be to perform a search for “Alpine install ” to search for a list of OS-level packages that may need to be added to your image. Whether a search results in a definitive list or you end up parsing through the more extensive errors received from an attempted build, you might be left doing a little trial and error of installing OS packages and then attempting a pip install of the python library in question.
In order to test these installations you’ll need to build a container outside of the local astro environment. This can be done by looking for the name of a recent image returned by docker images (which will begin with astro-local_)
and running docker run -it <image_name> bash
You’ll now be able to run apk add <os_package> as well as pip install <python_library> to determine which OS-level packages will need to be added to your packages.txt file prior to running astro airflow start with your desired requirements.txt entries.
P.S. We’ve specifically run a new container outside of the astro environment. If you attempt to run astro airflow start, and shell into one of the containers with docker exec -it <container_name> bash to begin installing OS-level packages via apk add <os package>, the default user within the astro deployment will run into the following error right away:
bash-5.0$ apk add build-base
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied