Shulou(Shulou.com)06/02 Report--

Note: in the 2-node es used in the demonstration, some parameters are not configured properly, and more stringent parameter settings are required in the production environment (for example, password authentication must be enabled).

System version: CentOS7

Node planning:

Hot data Node: 192.168.2.4

Temperature data node: 192.168.2.190

PS: there is no such three-tier storage as hot warm cold. We usually use two kinds of hot warm.

We use tar.gz universal binaries here.

Useradd es

# cat / etc/security/limits.d/elasticsearch.conf

Es-nofile 65535es-noproc 65535es-memlock unlimited

Cd / usr/local/elasticsearch-7.4.2/

Mkdir data logs

Chown es.es / usr/local/elasticsearch-7.4.2/-R

Su-es

Cd / usr/local/elasticsearch-7.4.2/

Hot data node: 192.168.2.4 is configured as follows:

Cat config/elasticsearch.yml

Cluster.name: my-applicationnode.name: node-1node.attr.rack: r1node.attr.temperature: hotpath.data:. / data/path.logs:. / logsnode.master: truenode.data: truenode.ingest: truebootstrap.memory_lock: truenetwork.host: 0.0.0.0http.port: 9200cluster.initial_master_nodes:-192.168.2.4 0.0.0.0http.port 9300 discovery.seed_hosts:-192.168.2.4-192.168.2.4- 192.168.2.190 configuration item # xpack.security.enabled: true#xpack.security.transport.ssl.enabled: truexpack.security.audit.enabled: truexpack.security.audit.logfile.events.include: access_denied for gateway.recover_after_nodes: 1#action.destructive_requires_name: true#xpack Access_granted, anonymous_access_denied, authentication_failed,connection_denied, tampered_request, run_as_denied, run_as_grantedxpack.security.audit.logfile.emit_node_host_address: truexpack.security.audit.logfile.emit_node_host_name: truexpack.sql.enabled: truexpack.ilm.enabled: true

Temperature data node: 192.168.2.190

Cat config/elasticsearch.yml

Cluster.name: my-applicationnode.name: es11node.attr.rack: r1node.attr.temperature: warmpath.data:. / data/path.logs:. / logsnode.master: truenode.data: truenode.ingest: truebootstrap.memory_lock: truenetwork.host: 0.0.0.0http.port: 9200cluster.initial_master_nodes:-192.168.2.4 r1node.attr.temperature 9300 discovery.seed_hosts:-192.168.2.4-192. Configuration item # xpack.security.enabled: true#xpack.security.transport.ssl.enabled: truexpack.security.audit.enabled: truexpack.security.audit.logfile.events.include: access_denied for 168.2.190gateway.recover_after_nodes: 1#action.destructive_requires_name: true#xpack Access_granted, anonymous_access_denied, authentication_failed,connection_denied, tampered_request, run_as_denied, run_as_grantedxpack.security.audit.logfile.emit_node_host_address: truexpack.security.audit.logfile.emit_node_host_name: truexpack.sql.enabled: truexpack.ilm.enabled: true

Start the script:

# cat start.sh

# es needs to export export JAVA_HOME=./jdk/nohup. / bin/elasticsearch > / dev/null 2 > & 1 & if it depends on a higher version of jdk.

Stop the script:

# cat stop.sh

Kill $(jps | grep Elasticsearch | awk'{print $1}')

Create the index and move the data to the hot node:

Curl-H 'Content-Type: application/json'-X PUT http://localhost:9200/index-2019.10.19?prettycurl-H 'Content-Type: application/json'-X PUT http://localhost:9200/index-2019.10.19/_settings-d'{"index.routing.allocation.require.temperature": "hot"}'

If we want to move the index-2019.10.19 data to the warm node, we can use the following command

Curl-H 'Content-Type: application/json'-X PUT http://localhost:9200/index-2019.10.19/_settings-d'{"index.routing.allocation.require.temperature": "warm"}'

After the process runs through, we can write a script that does two things:

1. Create the index of the following day ahead of time and make sure it falls to the hot node

2. Index, label, and store the indexes of 7 days ago in the warm node of es (large capacity HDD disk):

#! / bin/bash# creates a new index with a statement similar to the following five lines You need to modify it according to your situation (it is recommended to use for loop to create index) # curl-H 'Content-Type: application/json'-X PUT http://localhost:9200/index-2019.10.19?pretty# curl-H' Content-Type: application/json'-X PUT http://localhost:9200/index-2019.10.19/_settings-d'# {# "index.routing.allocation.require.temperature": "hot" #}' Day=$ (date + "% Y.%m.%d"-d-7day) curl-H 'Content-Type: application/json'-X PUT http://192.168.2.4:9200/*-${day}/_settings-d' {"index.routing.allocation.require.temperature": "warm"}'

In addition, the index-lifecycle-management function is provided in es7, and we can configure it in the kibana interface. For details, you can check out the official documentation of es (individuals prefer to use the script above).

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.