Docker Hub Rate Limit Error: `toomanyrequests: You have reached your pull rate limit`

Astro CLI
1

Astronomer recently migrated from Docker Hub to Quay.io as our platform’s primary Docker Registry in light of Docker Hub’s new rate-limit policy, effective Nov 2nd, 2020.

Problem

Given the new rate limit, you may see the following error via the Astronomer CLI when pulling an Astronomer Docker image from Docker Hub:

FROM astronomerinc/ap-airflow:1.10.12-buster-onbuild
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

If you see this error, we recommend that you either:

  1. Replace your existing astronomerinc image to a corresponding quay.io/astronomer image as referenced in our “Manage Airflow Versions” doc, or
  2. Follow Docker’s instructions to increase your own limit as a user by authenticating and upgrading, or
  3. Wait an hour for the rate limit to lower and try again

While it is not recommended given the intermittent error above, images that pull from Docker Hub are not officially deprecated. If you happen to not run into the rate limit error, successfully running astronomerinc/ap-airflow images will not impact your DAGs in any way.

Reference

To query how many Docker Hub pulls you have left, run this script. Unfortunately, this does consume one API request (which counts towards your limit).

For more information, reference:

Note: If you’re running Astronomer Enterprise, the Quay.io change will affect all Docker Images you run on our platform, not just ap-airflow (e.g. ap-registry, ap-pgbouncer).