Monitoring with datadog

We use datadog to monitor much of our infrastructure. Often a sidecar datadog agent is able to provide a statsd daemon to receive metrics and forward them on.

Is there a way to do similar with our astronomer cloud deployment? Could we add another container to our deployment?

The alternative I am planning to go with if there is not already a known better approach is to use a custom entrypoint script that will spawn a datadog agent in the background. Something along these lines:
https://docs.docker.com/config/containers/multi-service_container.

However, I generally avoid running two processes in a single container and hope there is a good alternative.

Thanks!

There is an official datadog integration for airflow:

I raised a support ticket to ask when this would be available for Astronomer and the reply (May 2020):

Unfortunately at this moment we do not have support for Datadog integration with Astronomer platform, but this is something we are looking to have in near future.

Perhaps with more interest they will raise the priority of this request.
Cheers

Definitely still on our roadmap!

I have taken note of this request and adding it to our issue tracker!

1 Like