Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "the construction of ELK cluster and the simple use of Elasticsearchd api". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Elasticsearchd installation, the specific installation steps are very simple, but some configurations still need to pay attention to, the following is a brief description of the installation steps

Step 1: download the latest version from the official website: elasticsearch-5.4.1_1, install jdk8,elasticsearch developed by java before installation, extract it to the location you want to put: modify the configuration file, the location of the configuration file: / usr/local/elasticsearch-5.4.1_1/config/elasticsearch.yml, because for the sake of security elasticsearch cannot be started with root users, you need to create new users and assign permissions to the folder to new users For example, the user's name is elasticsearch. Under root, chown-R elasticsearch:elasticsearch / usr/local/elasticsearch-5.4.1_1, and then modify the parameters of the system:

a. Add / etc/security/limits.conf

* soft nofile 65536

* hard nofile 131072

* soft nproc 2048

* hard nproc 4096

B./etc/security/limits.d/90-nproc.conf

Just make the 1024 of soft nproc 1024 bigger.

C.vi / etc/sysctl.conf

Add the following configuration:

Vm.max_map_count=262144

d. And execute the command:

Sysctl-p

e. Modify elasticsearch.yml

Cluster.name: es_test_1 bootstrap.memory_lock: falsebootstrap.system_call_filter: falsenode.name: esc_test_node_1http.enabled: truenode.master: truenode.data: true// file location path.logs: / path/to/logs// log location path.logs: / home/data/elasticsearch/log/// this must be set otherwise the public network cannot access network.host: 193.168.6.87http.port: 9201transport.tcp Port: 9301http.cors.enabled: truehttp.cors.allow-origin: /. * / http.cors.allow-credentials: true// this is the configuration of the cluster stand-alone does not need to configure discovery.zen.ping.unicast.hosts: ["x.x.x.x"] / / 2 nodes node.max_local_storage_nodes: 2

Bin directory startup service:. / elasticsearch-d, look at the log / home/data/elasticsearch/log/,. If the log is normal, it means the startup is successful.

Common mistakes

At org.elasticsearch.bootstrap.Elasticsearch.main (Elasticsearch.java:84) ~ [elasticsearch-5.4.1.jar:5.4.1] Caused by: java.lang.RuntimeException: can not run elasticsearch as root at org.elasticsearch.bootstrap.Bootstrap.initializeNatives (Bootstrap.java:106) ~ [elasticsearch-5.4.1.jar:5.4.1] at org.elasticsearch.bootstrap.Bootstrap.setup (Bootstrap.java:194) ~ [elasticsearch-5.4.1.jar 5.4.1] at org.elasticsearch.bootstrap.Bootstrap.init (Bootstrap.java:350) ~ [elasticsearch-5.4.1.jar:5.4.1] at org.elasticsearch.bootstrap.Elasticsearch.init (Elasticsearch.java:123) ~ [elasticsearch-5.4.1.jar:5.4.

Normal startup log:

[2017-08-20T15:56:07031] [o.e.n.Node] [esc_test_node_1] JVM arguments [- Xms512m,-Xmx512m,-XX:+UseConcMarkSweepGC,-XX:CMSInitiatingOccupancyFraction=75,-XX:+UseCMSInitiatingOccupancyOnly,-XX:+DisableExplicitGC,-XX:+AlwaysPreTouch,-Xss1m,-Djava.awt.headless=true,-Dfile.encoding=UTF-8,-Djna.nosys=true,-Djdk.io.permissionsUseCanonicalPath=true,-Dio.netty.noUnsafe=true,-Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0,-Dlog4j.shutdownHookEnabled=false,-Dlog4j2.disable.jmx=true,-Dlog4j.skipJansi=true,-XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/local/elasticsearch-5.4.1_1] [2017-08-20T15:56:07889] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [aggs-matrix-stats] [2017-08-20T15:56:07889] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [ingest-common] [2017-08-20T15:56:07889] [ INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [lang-expression] [2017-08-20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [lang-groovy] [2017-08-20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [lang-mustache] [2017-08-20T15 56 esc_test_node_1 07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [lang-painless] [2017-08-20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [percolator] [2017-08-20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [reindex] [2017-08 -20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [transport-netty3] [2017-08-20T15:56:07890] [INFO] [o.e.p.PluginsService] [esc_test_node_1] loaded module [transport-netty4] [2017-08-20T15:56:07891] [INFO] [o.e.p.PluginsService] [esc_test_node_1] no plugins loaded [2017 -08-20T15:56:09451] [INFO] [o.e.d.DiscoveryModule] [esc_test_node_1] using discovery type [zen] [2017-08-20T15:56:10205] [INFO] [o.e.n.Node] [esc_test_node_1] initialized [2017-08-20T15:56:10205] [INFO] [o.e.n.Node] [esc_test_node_1] Starting... [2017-08-20T15:56:10387] [INFO] [o.e.t.TransportService] [esc_test_node_1] publish_address {192.168.6.87 INFO} Bound_addresses {192.168.6.87 INFO 9300} [2017-08-20T15:56:10394] [INFO] [o.e.b.BootstrapChecks] [esc_test_node_1] bound or publishing to a non-loopback or non-link-local address Enforcing bootstrap checks [2017-08-20T15:56:13446] [INFO] [o.e.c.s.ClusterService] [esc_test_node_1] new_master {esc_test_node_1} {xSFrf7feRD25YuwgtBeSLw} {G4byr3dMRc6T1QN_PRxxDQ} {192.168.6.87} {192.168.6.87 Reason: zen-disco-elected-as-master ([0] nodes joined) [2017-08-20T15:56:13465] [INFO] [o.e.h.n.Netty4HttpServerTransport] [esc_test_node_1] publish_address {192.168.6.87 virtual 9200} Bound_addresses {192.168.6.87 INFO 9200} [2017-08-20T15:56:13468] [INFO] [o.e.n.Node] [esc_test_node_1] started [2017-08-20T15:56:14573] [INFO] [o.e.g.GatewayService] [esc_test_node_1] recovered [11] indices into cluster_state

Step 2: install logstash, which is easy to install. Download the latest version from the official website and extract it.

Create an input and output file sample.conf

Input {stdin {} output {elasticsearch {hosts = > ["192.168.6.87 output 9201"]} stdout {codec = > rubydebug}}

Start the service:

The third step is to install kibana-5.1.1-linux-x86_64, modify the configuration file / usr/local/kibana-5.1.1-linux-x86_64/config/kibana.yml, and modify server.host: "192.168.x". Otherwise, you can start the service through the public network if you cannot access it.

[root@localhost bin] #. / kibana log [08info] [status] [plugin:kibana@5.1.1] Status changed from uninitialized to green-Ready log [08info] [status] [plugin:kibana@5.1.1] Status changed from uninitialized to green-Ready log] [info] [status] [plugin:elasticsearch@5.1.1] Status changed from uninitialized to yellow-Waiting for Elasticsearch log [08info] [status] [plugin:console@5.1.1] Status changed from uninitialized To green-Ready log [08 warning 09 warning 06.648] You're running Kibana 5.1.1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v5.4.1 @ 192.168.6.87 plugin:elasticsearch@5.1.1 9200 (192.168.6.87) log [08V09purl 06.657] [error] [plugin:elasticsearch@5.1.1] Status changed from yellow to red-Elasticsearch is still initializing the kibana index. Log [08info] [status] [plugin:timelion@5.1.1] Status changed from uninitialized to green-Ready log [08info] [status] [plugin:timelion@5.1.1] Status changed from uninitialized to green-Ready log [info] [listening] Server running at error-Elasticsearch plugin is red [error] [status] [ui settings] Status changed from uninitialized to red-Elasticsearch plugin is red

Open kibana in the browser and type "Xing er"

The use of Elasticsearchd is actually very simple, everything is resful, use postman as a simple example, insert a message into elsearch

What can be obtained

This is the end of the content of "the Construction of ELK Cluster and the simple use of Elasticsearchd api". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.