As a general best practice when using the Celery Executor, we recommend avoiding writing a file to a worker (and retrieving it). Given the temporary nature of workers/their resources, all storage on workers should be treated as ephemeral.
If you’re looking to retrieve a file from that worker, we’d recommend considering defining an on_success
or on_failure
callback in your DAGs that writes your file to an external system (e.g. S3, GCS). Related forum post here.