Shulou(Shulou.com)06/01 Report--

What this article shares with you is what is the basic concept of Elasticsearch? the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Subject to Elasticsearch 7.9.2.

Documents and indexes are logical concepts for developers, while nodes, fragments and clusters are physical concepts for operation and maintenance.

Document

A document can be understood as a JSON object. A document is the smallest unit stored and searched in ES.

Documents have their own id in the form of an integer that can be specified or generated by ES.

A common look of a document:

{"_ index": "myindex", "_ type": "doc", / / es7 starts. It must be doc "_ id": "YmYJdj888", "_ score": 0.2876333, "_ source": {"foo": "astr", "bar": "bstr", "biz": ["book", "cat"]}}.

Where the _ source field holds the original contents of the document, and the other fields are document metadata.

_ index indicates that it belongs to that index, and _ id represents the id of the document. We'll talk about it later on the other fields.

Indexes

So it's documents and collections. Similar to tables in a relational database.

Polysemy, index:

(in ES) a noun indicating a collection of documents

(in ES) verb, which means to put a document into ES (for example, to index a document into ES)

(put aside ES) noun, denote lookup mark (for example, this database is indexed by B + tree)

The documents in the index are scattered over shard.

Node

An ES node refers to a running instance of an ES program, which is actually a Java process.

In a production environment, a machine runs only one instance.

Each node has a name, specified by the configuration file, or with parameters added at startup

-E node.name=foo

After the node is started, the UID is assigned and saved in the data directory.

Cluster

ES is distributed software in which nodes of different roles are clustered to provide the following features:

Availability:

Service available: allow a small number of nodes to be unavailable

Data available: data is not lost when a small number of nodes are damaged

Scalable: machines can be added continuously.

Cluster:

Different clusters are distinguished by name, and the default cluster name is elasticsearch. The cluster name is specified through the configuration file or on the command line-E cluster.name=myc. There are > = 1 nodes in the cluster.

Roles of nodes in the cluster:

Master eligible: can be elected as master. Master is mainly responsible for creating and deleting indexes, deciding where to split, and maintaining and updating the status of the cluster.

Data: stores data.

Ingest: filter the data before it comes in.

Coordinating: the node that deals with the client. All the client requests go to the coordinator first, then the coordinator is busy behind, and then the result is returned to the client.

Machine learning: responsible for machine learning. The X-Pack plug-in needs to be enabled.

Node role parameter default value master eligiblenode.mastertruedatanode.datatrueingestnode.ingesttrue coordinator all involve machine learning node.mltrue

Any node must be a coordinator, and if you want to set a node to act only as a coordinator, you only need and must set the other four parameters to false.

In a production environment, each node should play only one role. If the amount of data is small, let some nodes work part-time as appropriate.

In a production environment, only the address of the coordinator should be retained.

Fragmentation and copy

The index has > = 1 shard, each shard has 1 main shard and > = 0 replica shard.

For example, for index myindex, if the number of shards is set to 3 and the number of copies is set to 1, the total number of data slices is 3 * (1x1) = 6. Generally speaking, if the number of fragments of an index is n and the number of copies is m, then the total number of slices is n (masks 1).

After a document is placed in an index, the document is placed into all the copy fragments corresponding to the main fragment. The main shard and its corresponding replica shards must be on different nodes, but this does not have to be the case between the main shards.

For example, index myindex, the number of shards is set to 3, and the number of copies is set to 1. The three main fragments are marked as P0, P1 and P2, and the corresponding replicas are marked as R0Percience R1 and R2. For three machines, the distribution of data slices may be:

(P0 R2) (P1 R0) (P2 R1)

If only 2 machines are given, the distribution of data slices may be:

(P0 P1 R2) (P2 R0 R1)

If only one machine is given, the distribution of data slices may be:

(P0 P1 P2)

At this time, ES cannot create replica shards, and the cluster is available but not highly available.

The number of fragments of the index cannot be modified after it is set, and the index must be rebuilt if it needs to be modified. The number of copies can be adjusted dynamically.

Increasing the number of shards appropriately can avoid the excessive size of a single data slice and increase the scalability of the machine. Increasing the number of replicas appropriately can improve performance and throughput.

These are the basic concepts of Elasticsearch. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.