Forward Astronomer Logs to Amazon S3 | Astronomer Docs

We have a question around shipping our logs to Splunk.

We will be remote logging our logs to the S3 bucket (or forward from Fluentd as suggested in overview), from Astronomer overview we understand that Fluentd will clean up the log and send to elasticsearch, preferred output will be elasticsearch.

Our question is what is the best practices around shipping the logs to Splunk

  • from Elasticsearch
  • Local folder before fluentD cleans them up
  • S3, to make is symmetric