Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the method of Elasticsearch cluster data backup and recovery". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next, let the editor take you to learn "what is the method of Elasticsearch cluster data backup and recovery"?

Brief introduction

Elasticsearch has a replica mechanism to ensure the high availability of the cluster, but it cannot solve the data loss in the following situations:

The machine storage where the master copy is located is all damaged.

Mistakenly delete index data.

The upgrade failed and the data cannot be rolled back.

Backup the data regularly and restore on demand can solve the above problems.

Elasticsearch provides Snapshot and Restore API for backup and recovery of cluster data.

The process of data backup can be understood simply as the process of synchronizing local data files to a remote warehouse (repository).

Full and incremental backups are supported.

Repository has the following common types: fs / S3 / HDFS / Azure / Google Cloud Storage.

Practice 1 create the original index and import data PUT _ template/blog-template {"index_patterns": ["blog-*"], "settings": {"number_of_shards": 3, "number_of_replicas": 1}, "mappings": {"dynamic_templates": [{"integers": {"match": "int_*" "mapping": {"type": "integer"}, {"strings": {"match_mapping_type": "string", "mapping": {"type": "keyword", "ignore_above": 256}] "properties": {"title": {"type": "text"} PUT blog-testPOST blog-test/_bulk {"index": {}} {"title": "elasticsearch best practice", "author_firstname": "tony", "author_lastname": "song", "tags": "elasticsearch, logstash", "int_age": 18, "locale": "zh" En "{" index ": {} {" title ":" elastic stack release V5 "," author_firstname ":" shay "," author_lastname ":" banon "," tags ":" elk, elasticsearch,release "," int_age ": 28," locale ":" zhc,en "} {" index ": {} {" title ":" kibana tutorial "," author_firstname ":" zen "," author_lastname ":" wei "," tags ":"," int_age "38: "locale": "zhc,en"} GET blog-testGET blog-test/_search2 modifies elasticsearch configuration file

Modify the elasticsearch.yml configuration file by adding the following configuration:

Path.repo: ["/ home/elastic/backup"]

Restart the elasticsearch process to view the created repo:

GET _ cluster/settings?include_defaults&filter_path=*.path.repo# output result: {"defaults": {"path": {"repo": ["/ home/elastic/backup"]} 3 create an association repository# create an association repositoryPUT / _ snapshot/my_fs_backup {"type": "fs" "settings": {"location": "/ home/elastic/backup/my_fs_backup", "compress": true}}

View the associations created:

GET _ snapshot/my_fs_backup# output result: {"my_fs_backup": {"type": "fs", "settings": {"compress": "true", "location": "/ home/elastic/backup/my_fs_backup"}

See on which node the association was created:

POST _ snapshot/my_fs_backup/_verify# output result {"nodes": {"pMrJwVGSQcSgeTZdh71QRw": {"name": "node1"} 4 execute snapshot

Indices: index the snapshot.

Wait_for_completion=true: whether to wait for the snapshot to be completed before responding; if true, it will wait for the snapshot to complete before responding. (default is false, and respond immediately without waiting for the snapshot to complete)

Ignore_unavailable: when set to true, indexes that do not exist are ignored when snapshots are created.

Include_global_state: when set to false, snapshots can be completed when not all primary shards of an index are available.

# create snapshot_1 to snapshot the blog-test index PUT / _ snapshot/my_fs_backup/snapshot_1?wait_for_completion=true {"indices": "blog-test", "ignore_unavailable": true, "include_global_state": false}

View the snapshot created:

GET / _ snapshot/my_fs_backup/snapshot_*#GET / _ snapshot/my_fs_backup/_all look at all # output results: {"snapshots": [{"snapshot": "snapshot_1", "uuid": "FEbAt3BiR1SAiBkO7pfoZg", "version_id": 7020199, "version": "7.2.1", "indices": ["blog-test"] "include_global_state": false, "state": "SUCCESS", "start_time": "2021-02-06T03:28:40.001Z", "start_time_in_millis": 1612582120001, "end_time": "2021-02-06T03:28:40.213Z", "end_time_in_millis": 1612582120213, "duration_in_millis": 212 "failures": [], "shards": {"total": 3, "failed": 0, "successful": 3}]}

View snapshot status:

GET _ snapshot/my_fs_backup/snapshot_1/_status# output result: {"snapshots": [{"snapshot": "snapshot_1", "repository": "my_fs_backup", "uuid": "FEbAt3BiR1SAiBkO7pfoZg", "state": "SUCCESS", "include_global_state": false, "shards_stats": {"initializing": 0 "started": 0, "finalizing": 0, "done": 3, "failed": 0, "total": 3}, "stats": {"incremental": {"file_count": 12, "size_in_bytes": 15608} "total": {"file_count": 12, "size_in_bytes": 15608}, "start_time_in_millis": 1612582120074, "time_in_millis": 15608}, "indices": {"blog-test": {"shards_stats": {"initializing": 0 "started": 0, "finalizing": 0, "done": 3, "failed": 0, "total": 3}, "stats": {"incremental": {"file_count": 12 "size_in_bytes": 15608}, "total": {"file_count": 12, "size_in_bytes": 15608}, "start_time_in_millis": 1612582120074, "time_in_millis": 15608} "shards": {"0": {"stage": "DONE", "stats": {"incremental": {"file_count": 4, "size_in_bytes": 5104} "total": {"file_count": 4, "size_in_bytes": 5104}, "start_time_in_millis": 1612582120143, "time_in_millis": 39}} "1": {"stage": "DONE", "stats": {"incremental": {"file_count": 4, "size_in_bytes": 5265} "total": {"file_count": 4, "size_in_bytes": 5265}, "start_time_in_millis": 1612582120074, "time_in_millis": 25}} "2": {"stage": "DONE", "stats": {"incremental": {"file_count": 4, "size_in_bytes": 5239} "total": {"file_count": 4, "size_in_bytes": 5239}, "start_time_in_millis": 1612582120113 "time_in_millis": 21}}]} 5 restore snapshot

Rename_pattern: regular match original index name

Rename_replacement: use the matched field to rename the new index

POST _ snapshot/my_fs_backup/snapshot_1/_restore {"indices": "blog-test", "ignore_unavailable": true, "rename_pattern": "(. +)", "rename_replacement": "restored_$1"}

Query the data for the new index:

GET restored_blog-test/_search so far, I believe you have a deeper understanding of "what is the method of Elasticsearch cluster data backup and recovery". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.