Shulou(Shulou.com)06/01 Report--

This article introduces the knowledge of "how to remove elasticsearch nodes safely". Many people will encounter such a dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Why remove a node? there are many situations.

The physical machine where the node is located needs to be destroyed.

Node instances need to be upgraded and restarted

Es document

Elasticsearch safely removes nodes

If you want to safely remove an es node without changing the number of fragments, 100% will not cause data loss, that is, to ensure that all data of this node is received by other nodes. Then stop the instance of this node.

1. Data-node node

Step 1: exclude nodes from the cluster routing policy

Curl-XPUT http://0.0.0.0:9200/_cluster/settings?pretty-d'{"transient": {"cluster.routing.allocation.exclude._ip": "10.10.10.11"}} 'PUT _ cluster/settings {"transient": {"cluster.routing.allocation.exclude._ip": "10.1.22.129"}} execution result: {"acknowledged" >

Step 2: wait for all the fragments on the node to be migrated

Curl http://0.0.0.0:9200/_cluster/health?prettycurl http://0.0.0.0:9200/_cluster/pending_tasks?prettycurl http://0.0.0.0:9200/_cluster/allocation/explain?pretty1. Check cluster status http://10.1.34.146:9200/_cluster/health?pretty{ cluster_name: "my-es6-test", status: "green", timed_out: false, number_of_nodes: 4, number_of_data_nodes: 4, active_primary_shards: 150, active_shards: 272, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0 Number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100} 2. If pening_tasks occurs, there is a risk that the cluster will not be able to create a new index when the pending_tasks level is > = HIGH. Http://10.1.34.146:9200/_cluster/pending_tasks?pretty{ "tasks": []} 3. If UNASSIGNED shards appears in the cluster, check the reason to see if the allocation policy causes the shard http://10.1.22.129:9200/_cluster/allocation/explain?pretty4. not to be migrated. Check whether the node data has been migrated. 0 means that the data has also been migrated http://10.1.34.146:9200/_nodes/%7Bnode-6%7D/stats/indices?pretty{ _ nodes: {total: 0, successful: 0, failed: 0}, cluster_name: "my-es6-test", nodes: {}}

Step 3: offline node

Kill {pid}

Step 4: cancel the node disable policy

Curl-XPUT http://0.0.0.0:9200/_cluster/settings?pretty-d'{"transient": {"cluster.routing.allocation.exclude._ip": null}} 'PUT _ cluster/settings {"transient": {"cluster.routing.allocation.exclude._ip": null}}

Two nodes disable the policy:

Curl-XPUT http://0.0.0.0:9200/_cluster/settings?pretty-d'{"transient": {"cluster.routing.allocation.exclude._ip": "10.10.10.11pr 10.10.10.12"}'

2. Master-node node # offline non-master node # start# step 1: stop the ES instance specified by IP # # Note: because the ES cluster configuration file specifies that the ES cluster must have two master eligible nodes before the master node can be elected Therefore, the number of nodes in the mater-node group should be maintained at least 3 # end# offline master nodes # start# step 1: stop the ES instance specified by IP # # Note: the default time for master nodes to conduct elections is 3s, which may be set in the configuration file as 30s. Cluster features are not available during the master election (indexing, lookup, various API functions) # end3. Client-node Node # unverified # start## step Delete the specified IP## in 1:LB step 2: stop the ES instance specified by IP # endElasticsearch Security shutdown

Shut down Elasticsearch in an orderly manner to ensure that Elasticsearch has the opportunity to clean up and close unfinished resources. For example, after the node is shut down, it is removed from the cluster in an orderly manner, and logs are transferred to disk synchronously as well as other related cleaning activities. You can make sure that Elasticsearch stops in an orderly manner to help Elasticsearch stop correctly.

If Elasticsearch runs as a service, you can stop Elasticsearch through the service management features you install.

If you are running Elasticsearch directly from the console, you can stop it by sending conrtol + C, or send a SIGTERM signal to the Elasticsearch process on the POSIX system. You can get PID through a variety of tools to send signals (such as ps or jps):

[localhost~] $ps aux | grep Elasticsearch [localhost~] $ps-ef | grep Elasticsearch

Or by starting the log:

[2016-07-07 12 I8hydUG 26 version [5.0.0-alpha4], pid [15399], build [3f5b994/2016-06-27T16:23:46.861Z], OS [Mac OS X/10.11.5/x86_64], JVM [Oracle Corporation/Java HotSpot (TM) 64-Bit Server VM/1.8.0_92/25.92-b14]

Or obtain it from the PID file specified at startup:

$. / bin/elasticsearch-p / tmp/elasticsearch-pid-d $cat / tmp/elasticsearch-pid & & echo15516 $kill-SIGTERM 155161. Fatal error downtime

While the Elasticsearch virtual machine is running, some fatal errors may occur to mark the virtual machine as suspicious. These fatal errors may include virtual machine internal errors and serious Ihop O errors.

When Elasticsearch detects that the virtual machine is experiencing such a fatal error, Elasticsearch will try to log the error and then stop the virtual machine. When Elasticsearch initiates such a shutdown, it does not go through the orderly shutdown described above. Elasticsearch will return a specific status code to identify the error.

Error cause error code JVM internal error 128memory overflow 127heap overflow 126unknown virtual machine error 125serious Iamp O error 124unknown fatal error 1 "how to safely remove elasticsearch nodes" is here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.