How do I run Airflow 1.10 on Astronomer v0.7?

Astronomer
1

I’m an Astronomer customer, and would love access to Airflow 1.10 features on Astronomer v0.7.

What do I need to do?

1 Like
2

Yes, using airflow 1.10.1 is a simple change.

  1. Make sure you are running v0.7.5 version of the Astronomer CLI. If you’re not sure, run astro version. To install, run:

curl https://install.astronomer.io | sudo bash -s -- v0.7.5

  1. Change the FROM link in your Dockerfile to:
FROM astronomerinc/ap-airflow:0.7.5-1.10.1-onbuild

This will pull down our 1.10.1 image instead of the 1.9 image.

  1. Confirm this works by running astro airflow start and you should see the 1.10.1 UI at localhost:8080/admin.

image
image.png1276×502 33.2 KB

Notice that there’s the little Red X to delete DAGs from the UI, confirming that you are running Airflow 1.10.1 (or go to About -> Version

Note: Astronomer’s 1.10.1 image does not support the Airflow RBAC on v0.7 - this will be a part of our 0.9 release.

Note: The 1.10.1 Webserver is more CPU intensive than the 1.9 version. If you’re seeing your browser load slowly or not picking up the latest Airflow version, you might need to scale up the Webserver a bit via the Astronomer UI (don’t worry about running a deploy once you do so - the config will save as soon as you hit Save).

4

I like the ability to clear out old DAGs from the UI, that’s handy. Are there other improvements I should upgrade for? Anything that 1.9 does badly that 1.10 will improve?

The changelog lists new support for SageMaker and IMAP, both of which are interesting and I can see being useful for me in the future, but not immediately necessary.

Thanks for this topic!

5

Hey @mplovepop!

This is a really good writeup of some of the changes:

The biggest takeaways are additions of AWS hooks/operators and adding a few docs.

Please note that we don’t support the K8s executor or the RBAC UI - we’ll be adding those shortly.

Also, we’ve found that the new webserver is a little more CPU greedy than the 1.9 webserver - if you are seeing the UI lag a bit, I’d recommend giving it some more AUs (Astronomer units) on that sweet new slider in the Configure tab :slight_smile: