Fix AWS Elasticsearch cluster yellow status

This article covers a mismatch of index and AWS Elasticsearch service domain configurations. It could be a reason for the “yellow” status of the AWS ES cluster if you used default shard and replicas configs without updating ES nodes.

ES is superior technology. It allows you to implement an instant search engine for your project efficiently. Meanwhile, many implementation details and configurations could affect the quality of search results and performance.

The index is created with the default amount of shards and replicas.

Here you can read a good article about ES basics concepts.

Let’s have a look at defaults as you are familiar with the concepts.

Index with default configs has index.number_of_shards equals to 5. Meanwhile, replicas property index.number_of_replicas equals 1.

Stats request returns such payload for this case.

The response could be confusing. Ten total shards, but only 5 are successful.

There is a reason. Index’s replica requires a separated node for itself and its shards. But there is no separated data node for default local or AWS ES instance.

Five shards for the master node are successfully set up. Meanwhile, the total requested by configs amount of shards is 10. This is why the yellow AWS ES cluster used default ES settings but created an AWS ES domain with only one node.

Add node on the AWS

AWS makes this process easy and smooth. You can do it without any downtime.

Go to Elasticsearch service domain dashboard.

Click “Edit domain” button

Update number of nodes on Data nodes section

“Submit” changes.

Don’t be confused with extra data nodes on the dashboard. AWS needs some time to balance instances. A redundant data node exists around an hour for a 9GB ES domain with migration from 1 to 2 nodes.

P.S. Set amount of shards and replicas on index creation.

You can configure a number of shards and replicas at index creation. Add these fields to the payload on the index creation.

Be free to ask us any questions. Code with Shardik.