Elasticsearch won't work on EKS

1

I’m running into an issue when installing the platform on EKS - elasticsearch keeps crashing with

changes documentation for the next major version.
[2019-03-22T18:02:31,525][INFO ][o.e.d.DiscoveryModule    ] [dusty-opossum-elasticsearch-master-0] using discovery type [zen] and host providers [settings]
[2019-03-22T18:02:32,011][INFO ][o.e.n.Node               ] [dusty-opossum-elasticsearch-master-0] initialized
[2019-03-22T18:02:32,012][INFO ][o.e.n.Node               ] [dusty-opossum-elasticsearch-master-0] starting ...
[2019-03-22T18:02:32,219][INFO ][o.e.t.TransportService   ] [dusty-opossum-elasticsearch-master-0] publish_address {10.182.8.213:9300}, bound_addresses {0.0.0.0:9300}
[2019-03-22T18:02:32,230][INFO ][o.e.b.BootstrapChecks    ] [dusty-opossum-elasticsearch-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max file descriptors [8192] for elasticsearch process is too low, increase to at least [65536]
2

This is caused by an update Amazon made to the EKS AMI for Kubernetes 1.11

Until AWS releases a new AMI, our recommended workaround is to use the CloudFormation yaml provided here instead of the one provided in the AWS doc.

This includes a bash script that fixes the issue:

sed -i '/"nofile": {/,/}/d' /etc/docker/daemon.json

The above sed command turns:

{
  "bridge": "none",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "10"
  },
  "live-restore": true,
  "max-concurrent-downloads": 10,
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Soft": 2048,
      "Hard": 8192
    }
  }
}

to

{
  "bridge": "none",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "10"
  },
  "live-restore": true,
  "max-concurrent-downloads": 10,
  "default-ulimits": {
  }
}

Further information can be found here:
https://github.com/bitnami/kube-prod-runtime/blob/master/docs/troubleshooting.md#troubleshooting-elasticsearch