Shulou(Shulou.com)06/03 Report--

[TOC]

0 description

The es version used is 5.6 and the Linux version is CentOs 6.5.

1 Elasticsearch bootstrap checks1.1 development environment

If network.host is not configured in the configuration of es to specify an available IP address, by default, it is bound to localhost. In this case, es will think that users are only using es in the development environment, based on the principle of using it out of the box, although es also performs bootstrap checks at this time to check whether the user's configuration matches the security value set by es, as follows:

If there is a match, there will be no warnning information, and es will start normally; if there is no match, there will be warnning information, but because it is a development environment, es will still start normally.

Once the user configures network.host to specify an available non-loopback address, es will assume that the user starts es in the production environment and will also check it, but if the check fails, it will directly promote the previous warnning to error, so es will fail to start.

2 bootstrap checks analysis when the development environment is started

If you do not configure network.host, start es directly and you will have the following warnning:

[2018-12-07T04:15:44735] [INFO] [o.e.d.DiscoveryModule] [PQ85ukj] using discovery type [zen] [2018-12-07T04:15:45702] [INFO] [o.e.n.Node] initialized [2018-12-07T04:15:45703] [INFO] [o.e.n.Node] [PQ85ukj] starting. [2018-12-07T04:15:46071] [INFO] [o .e.t.TransportService] [PQ85ukj] publish_address {127.0.0.1:9300} Bound_addresses {[:: 1]: 9300}, {127.0.0.1 o.e.b.BootstrapChecks 9300} [2018-12-07T04:15:46090] [WARN] [o.e.b.BootstrapChecks] [PQ85ukj] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2018-12-07T04:15:46090] [WARN] [o.e.b.BootstrapChecks] [PQ85ukj] max number of threads [1024] for user [hadoop] is too low Increase to at least [2048] [2018-12-07T04:15:46090] [WARN] [o.e.b.BootstrapChecks] [PQ85ukj] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] [2018-12-07T04:15:46090] [WARN] [o.e.b.BootstrapChecks] [PQ85ukj] system call filters failed to install Check the logs and fix your configuration or disable system call filters at your own risk [2018-12-07T04:15:49269] [INFO] [o.e.c.s.ClusterService] [PQ85ukj] new_master {PQ85ukj} {PQ85ukjdSoeVEpSpByAjMw} {Dbb3lzTWTN-eUEKXO8z-sw} {127.0.0.1} Reason: zen-disco-elected-as-master ([0] nodes joined) [2018-12-07T04:15:49313] [INFO] [o.e.h.n.Netty4HttpServerTransport] [PQ85ukj] publish_address {127.0.0.1 nodes joined 9200}, bound_addresses {[:: 1]: 9200} {127.0.0.1 PQ85ukj 9200} [2018-12-07T04:15:49313] [INFO] [o.e.n.Node] [PQ85ukj] started [2018-12-07T04:15:49553] [INFO] [o.e.g.GatewayService] [PQ85ukj] recovered [0] indices into cluster_state

Extract its waarnning information as follows:

File descriptor: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] Threads: max number of threads [1024] for user [hadoop] is too low, increase to at least [2048] Virtual memory: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] system call filters: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

You can see that there are four problems, namely: file descriptor, number of threads, virtual memory and system call filters.

Although there is warnning, because es itself will be thought to be running in a development environment, it can still be started normally based on the out-of-the-box feature.

3 bootstrap checks analysis when the production environment is started

After binding the IP address and then starting it, the following error message is found:

ERROR: [4] bootstrap checks failed [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2]: max number of threads [1024] for user [hadoop] is too low, increase to at least [2048] [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] [4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

Error directly, so startup will fail unless the above settings meet the security requirements.

4 normal startup configuration of production environment

To solve the problems above, you need to make the following configuration.

4.1 temporary modification of file descriptor: ulimit-n 65536

But after logging back in, it will return to the default value.

Permanent modification

Modify the / etc/security/limits.conf configuration as follows:

Hadoop soft nofile 65536 # soft means that if you exceed this value, you will have warnninghadoop hadr nofile 100000 # hard, but you cannot exceed this value.

After that, log in again and use ulimit-n to verify.

4.2 number of threads

Modify the / etc/security/limits.conf configuration as follows:

Hadoop soft nproc 2048hadoop hard nproc 4096

In fact, the profile's description of nproc is the number of processes, not the number of threads:

# 39 an user name# 1 41% # / etc/security/limits.conf##Each line describes a limit for a user in the form:## Where:# can be:#-an user name#-a group name With @ group syntax#-the wildcard *, for default entry#-the wildcard%, can be also used with% group syntax # for maxlogin limit## can have the two values:#-soft "for enforcing the soft limits# -" hard "for enforcing hard limits## can be one of the following:#-core-limits the core filesize (KB) #-data-max data size (KB) #-fsize-maximum filesize (KB) #-memlock-max locked-in-memory address space (KB) #-nofile-max number of open files#-rss-max resident set size (KB) #-stack-max stack size (KB) #-cpu-max CPU time (MIN) #-nproc-max number of processes#-as-address space limit (KB) #-maxlogins-max number of logins for this user#-maxsyslogins-max number of logins on the system#-priority The priority to run user process with#-locks-max number of file locks the user can hold#-sigpending-max number of pending signals#-msgqueue-max memory used by POSIX message queues (bytes) # nice-max nice priority allowed to raise to values: [- 20 19] #-rtprio-max realtime priority## * soft core 0 memory * hard rss 100004.3 Virtual memory View current value sysctl vm.max_map_count temporary setting sysctl-w vm.max_map_count=262144

But it will fail when the system is rebooted.

Permanent setting

Modify the configuration file / etc/sysctl.conf as follows:

Vm.max_map_count=262144

It will not take effect until it is rebooted.

4.4 system call filters

Reason

This is because Centos6 does not support SecComp, and the default bootstrap.system_call_filter of ES5.4.0 is true for detection, so the detection fails. After the failure, the ES cannot be started directly.

Solve

Configure bootstrap.system_call_filter as false in elasticsearch.yml, and note that under Memory:

Bootstrap.memory_lock: false

Bootstrap.system_call_filter: false

Reference: https://www.jianshu.com/p/89f8099a6d09

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.