[Errno 13] Permission denied: '/usr/local/airflow/logs/scheduler'

Airflow
1

If you’re seeing this error when working locally on Astronomer, it may be that a logs folder was somehow built into your image with the wrong permissions.

A few things:

  • Do you have a logs folder somewhere in your astro project directory?
  • If so, you can try either:
    1. Removing the logs folder by running rm -rf logs/ and then pushing up a deploy OR
    2. Try running echo logs/ > .dockerignore so that that folder stays, but doesn’t get built into your image
2 Likes
2

The solution makes sense to me but it is not working for me locally. :frowning:


Error

Waiting for host: postgres 5432
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/logging/config.py", line 563, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/local/lib/python3.7/logging/config.py", line 736, in configure_handler
    result = factory(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/airflow/utils/log/file_processor_handler.py", line 50, in __init__
    os.makedirs(self._get_log_directory())
  File "/usr/local/lib/python3.7/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/local/airflow/logs/scheduler/2020-05-13'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 45, in <module>
    settings.initialize()
  File "/usr/local/lib/python3.7/site-packages/airflow/settings.py", line 374, in initialize
    LOGGING_CLASS_PATH = configure_logging()
  File "/usr/local/lib/python3.7/site-packages/airflow/logging_config.py", line 68, in configure_logging
    raise e
  File "/usr/local/lib/python3.7/site-packages/airflow/logging_config.py", line 63, in configure_logging
    dictConfig(logging_config)
  File "/usr/local/lib/python3.7/logging/config.py", line 800, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/local/lib/python3.7/logging/config.py", line 571, in configure
    '%r' % name) from e
ValueError: Unable to configure handler 'processor'
Waiting for host: postgres 5432
Unable to load the config, contains a configuration error.

Things I checked and tried:

  • rm -rf logs/
  • chmod 777 -r logs
  • echo logs/ > .dockerignore
  • delete all images and astro start
3

Hi Nehil,

Hm, yeah something is up with postgres there.

Few things to try:

  • Can you create a new directory somewhere and run astro dev init and astro dev start ? Lets see if that works.
  • Can you try running chmodd 777 -R {directory_name} Will apply it to the entire directory
    -If that doesn’t work, can you remove the volumes for your postgres container? This will remove the data inside of that postgres but should free up those permissions.

docker volume ls
`docker volume rm <volume_name>

Let us know if this helps

1 Like
4

Ran into this exact same issue today using latest 1.10.12 buster image + custom adds of backport-providers airflow packages.

astro dev kill was invaluable to help clear out old/bad data and logs volumes (actually does docker volume rm <volume_name> for you behind the scenes) and resolved all my issues for me.

1 Like