Issues with launching a worker

Airflow
1

Hi,

I am getting the following error running
airflow worker -q this_queue

Traceback (most recent call last):
File “/home/test/3.7.6/bin/airflow”, line 37, in
args.func(args)
File “/home/test/3.7.6/lib/python3.7/site-packages/airflow/utils/cli.py”, line 76, in wrapper
return f(*args, **kwargs)
File “/home/test/3.7.6/lib/python3.7/site-packages/airflow/bin/cli.py”, line 1271, in worker
worker = worker.worker(app=celery_app)
File “/home/test/3.7.6/lib/python3.7/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/home/test/3.7.6/lib/python3.7/site-packages/click/core.py”, line 781, in main
with self.make_context(prog_name, args, **extra) as ctx:
File “/home/test/3.7.6/lib/python3.7/site-packages/click/core.py”, line 698, in make_context
ctx = Context(self, info_name=info_name, parent=parent, **extra)
TypeError: init() got an unexpected keyword argument ‘app’

2

Something seems wrong with celery 5.0.0 version and airflow version 1.10.12.

Celery:
def worker(ctx, hostname=None, pool_cls=None, app=None, uid=None, gid=None,
loglevel=None, logfile=None, pidfile=None, statedb=None,
**kwargs):
“”"Start worker instance.

Examples
--------
$ celery --app=proj worker -l INFO
$ celery -A proj worker -l INFO -Q hipri,lopri
$ celery -A proj worker --concurrency=4
$ celery -A proj worker --concurrency=1000 -P eventlet
$ celery worker --autoscale=10,0

"""
app = ctx.obj.app

Airflow

worker = worker.worker(app=celery_app)

as you can see, airflow isn’t calling worker with ctx…

Thoughts?

3

I’m seeing the same thing. Have you figured out a workaround? Maybe an older celery works. UPDATE yes 4.4.7 seems to work fine, yay.