Transitioning to the Celery executer

Airflow
1

My organization seems like it’s hitting the limits of the LocalExecuter on Astronomer cloud: lots of killed tasks and we’re near the max of our resources.

I’ve never used the Celery executer in airflow before, or celery at all for that matter. Do I need any knowledge to migration our deployment? Is it as simple as flipping the switch? What should I expect. I need to avoid downtime.

Thanks

2

Yes it should be as simple as flipping the switch. You may wnat to wait until you don’t have any tasks running. If you do, those tasks will fail and should be retried by the CeleryExecutor when it comes online. (assuming you have retires enabled on that task).