What packages do I get by default?
You can get this list by starting an astronomer project (‘astro airflow start’ in an astro directory), jumping into the container (docker ps to get the container name, and then docker exec -it SCHEDULER_CONTAINER bash) and then running pip freeze.
The output of this is:
dvirajparekh@orbiter:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a0d14716cba airflow-code_66a665/airflow:latest "tini -- /entrypoint…" 4 hours ago Up 4 hours 5555/tcp, 8793/tcp, 0.0.0.0:8080->8080/tcp airflowcode66a665_webserver_1
9568966a0280 airflow-code_66a665/airflow:latest "tini -- /entrypoint…" 4 hours ago Up 4 hours 5555/tcp, 8080/tcp, 8793/tcp airflowcode66a665_scheduler_1
0d706b4e973c postgres:10.1-alpine "docker-entrypoint.s…" 4 hours ago Up 4 hours 0.0.0.0:5432->5432/tcp airflowcode66a665_postgres_1
virajparekh@orbiter:~$ docker exec -it airflowcode66a665_scheduler_1 /bin/bash
bash-4.4$ pip freeze
adal==1.2.1
alabaster==0.7.12
alembic==0.8.10
amqp==2.4.2
analytics-python==1.2.9
apache-airflow==1.10.2
asn1crypto==0.24.0
atlasclient==0.1.2
aws-xray-sdk==0.95
azure-common==1.1.18
azure-cosmos==3.0.2
azure-datalake-store==0.0.19
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.4.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==1.2.2
azure-nspkg==3.0.2
azure-storage==0.36.0
azure-storage-blob==1.5.0
azure-storage-common==1.4.0
Babel==2.6.0
bcrypt==3.1.6
billiard==3.5.0.4
bleach==2.1.4
boto==2.49.0
boto3==1.7.84
botocore==1.10.84
cachetools==3.1.0
cassandra-driver==3.17.0
celery==4.1.1
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4
click==6.7
cloudant==0.5.10
colorama==0.4.1
configparser==3.5.3
cookies==2.2.1
croniter==0.3.27
cryptography==2.6.1
cx-Oracle==7.1.2
Cython==0.29.6
datadog==0.27.0
decorator==4.3.2
defusedxml==0.5.0
dill==0.2.9
docker==3.7.0
docker-pycreds==0.4.0
docopt==0.6.2
docutils==0.14
ecdsa==0.13
elasticsearch==5.5.3
elasticsearch-dsl==5.4.0
entrypoints==0.3
enum34==1.1.6
fastavro==0.9.6
Flask==0.12.4
Flask-Admin==1.5.2
Flask-AppBuilder==1.12.1
Flask-Babel==0.12.2
Flask-Bcrypt==0.7.1
Flask-Caching==1.3.3
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.3.2
flask-swagger==0.2.13
Flask-WTF==0.14.2
flower==0.9.2
freezegun==0.3.11
funcsigs==1.0.0
future==0.16.0
gitdb2==2.0.5
GitPython==2.1.11
google-api-core==1.8.1
google-api-python-client==1.7.8
google-auth==1.6.3
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.2.0
google-cloud-bigquery==1.10.0
google-cloud-bigtable==0.31.0
google-cloud-container==0.2.1
google-cloud-core==0.28.1
google-cloud-spanner==1.8.0
google-resumable-media==0.3.2
googleapis-common-protos==1.5.8
grpc-google-iam-v1==0.11.4
grpcio==1.19.0
grpcio-gcp==0.2.2
gunicorn==19.9.0
hdfs==2.2.2
hmsclient==0.1.1
html5lib==1.0.1
httplib2==0.12.1
idna==2.8
ijson==2.3
imagesize==1.1.0
inflection==0.3.1
iso8601==0.1.12
isodate==0.6.0
itsdangerous==1.1.0
JayDeBeApi==1.1.1
jeepney==0.4
Jinja2==2.10
jira==2.0.0
jmespath==0.9.4
JPype1==0.6.3
json-merge-patch==0.2
jsondiff==1.1.1
jsonpickle==1.1
keyring==18.0.0
kombu==4.3.0
kubernetes==8.0.1
ldap3==2.5.2
lockfile==0.12.2
lxml==4.3.2
Mako==1.0.7
Markdown==2.6.11
MarkupSafe==1.1.1
mock==2.0.0
mongomock==3.15.0
monotonic==1.5
moto==1.3.5
msrest==0.4.29
msrestazure==0.4.34
multi-key-dict==2.0.3
mysqlclient==1.4.2.post1
nose==1.3.7
nose-ignore-docstring==0.2
nose-timer==0.7.5
ntlm-auth==1.2.0
numpy==1.16.2
oauthlib==3.0.1
ordereddict==1.1
packaging==19.0
pandas==0.24.2
pandas-gbq==0.9.0
parameterized==0.7.0
paramiko==2.4.2
pbr==5.1.3
pendulum==1.4.4
pinotdb==0.2.2
protobuf==3.7.0
psutil==5.6.1
psycopg2==2.7.7
pyaml==18.11.0
pyasn1==0.4.5
pyasn1-modules==0.2.4
pycparser==2.19
pycryptodome==3.7.3
pycryptodomex==3.7.3
pydata-google-auth==0.1.3
pydruid==0.5.2
Pygments==2.3.1
PyHive==0.6.1
PyJWT==1.7.1
pykerberos==1.2.1
pymongo==3.7.2
pymssql==2.1.4
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.3.1
pysftp==0.2.9
PySmbClient==0.1.5
python-daemon==2.1.2
python-dateutil==2.8.0
python-editor==1.0.4
python-http-client==3.1.0
python-jenkins==1.4.0
python-jose==2.0.2
python-nvd3==0.15.0
python-slugify==3.0.0
python3-openid==3.1.0
pytz==2018.9
pytzdata==2018.9
pywinrm==0.3.0
PyYAML==5.1
qds-sdk==1.11.0
redis==2.10.6
rednose==1.3.0
requests==2.21.0
requests-futures==0.9.4
requests-kerberos==0.12.0
requests-mock==1.5.2
requests-ntlm==1.1.0
requests-oauthlib==1.2.0
requests-toolbelt==0.9.1
responses==0.10.5
rsa==4.0
s3transfer==0.1.13
sasl==0.2.1
SecretStorage==3.1.1
sendgrid==5.6.0
sentinels==1.0.0
setproctitle==1.1.10
six==1.12.0
slackclient==1.3.1
smmap2==2.0.5
snakebite==2.11.0
snowballstemmer==1.2.1
snowflake-connector-python==1.7.8
snowflake-sqlalchemy==1.1.10
Sphinx==1.8.5
sphinx-argparse==0.2.5
Sphinx-PyPI-upload==0.2.1
sphinx-rtd-theme==0.4.3
sphinxcontrib-httpdomain==1.7.0
sphinxcontrib-websupport==1.1.0
SQLAlchemy==1.2.18
sshtunnel==0.1.4
statsd==3.3.0
tabulate==0.8.2
tenacity==4.8.0
termstyle==0.1.11
text-unidecode==1.2
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
tzlocal==1.5.1
unicodecsv==0.14.1
uritemplate==3.0.0
urllib3==1.24.1
vertica-python==0.9.0
vine==1.2.0
webencodings==0.5.1
websocket-client==0.54.0
Werkzeug==0.14.1
wrapt==1.11.1
WTForms==2.2.1
xmltodict==0.12.0
zdesk==2.7.1
zope.deprecation==4.4.0
Note that if you have packages in requirements.txt that overlap or conflict with the default packages, you may run into errors. It is best practice to make sure that nothing in requirements.txt is already installed in the base image
1 Like