How do I manage multiple projects within a single Astronomer Enterprise environment?


#1

Can I have multiple projects with different dependencies, python versions, Airflow versions, etc. - all within a single deployment of Astronomer Enterprise?


#2

Absolutely. Astronomer Enterprise is built precisely for concurrent projects.

Here’s a quick diagram that should give you an idea of how projects, workspaces, and deployments relate to each other and to the devs working with them.

A few additional notes:

  • Each project can have its own dependencies, python version, airflow version, etc. That entire image will get shipped out and run in an airflow deployment. (Note: We’d recommend that each Airflow deployment only run 1 version of Python, but you could technically have one instance running python2 and another on python3 without any platform changes).

  • You can also specify your version of Airflow; 1.9 or 1.10

  • Resources are given at the deployment level, not the workspace level (user groups exist for workspace level). You can have different sized environments for different projects/environments.

  • Deployments run in their own Kubernetes namespace, which means they’re fully isolated from one another.