Shulou(Shulou.com)06/02 Report--

Common terminology

Document Document

The data document stored by the user in es is equivalent to a row of data index in mysql data table. Index consists of a list of documents with the same fields, which is equivalent to a table in mysql database. Table node Node, a running instance of elasticsearch, is the constituent unit of the cluster. The cluster Cluster consists of one or more nodes that provide service Document.

Json Object is composed of fields (field). Common data types are as follows:

String: text,keyword numeric type: long,integer,short,byte,double,float,scaled_ float Boolean: Boolean date: date binary: binary range type: integer_range,float_range,long_range,double_range,date_range

Each document has a unique ID logo

Self-specified es automatic generation

Metadata (MetaData), used to annotate document-related information

_ index: index name of the document _ type: type name of the document _ id: document unique id_uid: combination id, composed of _ type and _ id (6.x _ type no longer works, just like _ id) _ source: the original Json data of the document, from which you can get the content of each field _ all: integrate all field contents into this field, and by default disable vertical indexing of document ID to document content, word association.

The relationship between inverted index words and document ID

Inverted index-the query process obtains the documents corresponding to the "search engine" through the inverted index. Id has 1 and 3 through the full contents of the forward index query 1 and 3 returns the user's final result inverted index-word dictionary

Word dictionary (Term Dictionary) is an important component of inverted index.

Record the words of all documents, generally large. Record the associated information of the word inverted list. Inverted index-inverted list.

An inverted list (Posting List) records a collection of documents corresponding to words and consists of inverted index items (Posting)

The inverted index entry (Posting) mainly contains the following information:

Document Id, which is used to obtain the original information word frequency (TF, Term Frequency), to record the number of times the word appears in the document, to record the subsequent correlation calculation position (Position), to record the word segmentation position in the document (multiple), to do word search offset (Offset), to record the word at the beginning and end of the document, to highlight the Index.

Documents with the same structure are stored in the index (Document)

Each index has its own mapping definition, which defines the field name and type

A cluster can have multiple indexes, such as:

When nginx logs are stored, an index can be generated every day according to the date to store them.

Nginx-log-2019-01-01

Nginx-log-2019-01-02

Nginx-log-2019-01-03 create index and write data Rest API ()

Elasticsearch clusters provide RESTful API to others.

REST (REpresentational State Transfer) presentation layer state transfer (the state of operation on resources will change) URL specifies resources, such as Index,DocumentHttp Method specifies resource operation types, such as GET,POST,PUT,DELETE two interactive modes Curl command line

Kibana DevTools

Index API

Es has a special Index API for creating, updating, deleting index configurations, etc.

Create index APIPUT / test_index

View existing index GET_cat/indices

Document Document API

Es has a special Document API.

Create documents, query documents, update documents, delete documents, create documents:

Specify the document ID to create the document:

PUT / test_index/doc/1 {"username": "kibana", "version": 6.1}

Do not specify ID to create the document: POST / test1_index/doc {"username": "kibana", "version": 6.1}

Query document specifies the IDGET / test_index/doc/1 to query the document

Search all documents: _ searchGET / test_index/doc/_search / / find GET / test_index/doc/_search {"query": {"term": {"_ id": "1" / / specify criteria, find documents with ID 1}

Create document API in batch

Es allows you to manipulate more than one document at a time (add, delete, modify, query, create to create a document, and an error will be reported if the document already exists. Index creates the document and overwrites it if it exists. )

Endpoint is _ bulk As follows: POST _ bulk {"index": {"_ index": "test_index", "_ type": "doc", "_ id": "3"} {"username": "zabbix", "version": 4} {"delete": {"_ index": "test_index", "_ type": "doc", "_ id": "1"} {"update": {"_ id": "4", "_ index": "test_index" "_ type": "doc"} {"doc": {"es": "5.0"}}

Output:

{"took": 979, / / the query takes time, in ms "errors": false, / / returns the result Correct or wrong "items": [/ / the result returned by each operation {"index": {"_ index": "test_index", "_ type": "doc", "_ id": "3", "_ version": 1, "result": "created", / / create "_ shards": {"total": 2, "successful": 2 "failed": 0}, "_ seq_no": 0, "_ primary_term": 1, "status": 201}}, {"delete": {"_ index": "test_index", "_ type": "doc", "_ id": "1", "_ version": 2, "result": "deleted" / delete "_ shards": {"total": 2, "successful": 2, "failed": 0}, "_ seq_no": 1, "_ primary_term": 1, "status": 200}}, {"update": {"_ index": "test_index", "_ type": "doc", "_ id": "4" "_ version": 2, "result": "updated", / / change "_ shards": {"total": 2, "successful": 2, "failed": 0}, "_ seq_no": 3, "_ primary_term": 1, "status": 200} batch query document endpoint is _ mget As follows: (and documents with different indexes can be obtained under a _ mget) GET / _ mget / / look at the test_index index Documents with id 4 and 1. {"docs": [/ / indicate the document to be queried id {"_ index": "test_index", "_ type": "doc", "_ id": "4"}, {"_ index": "test_index", "_ type": "doc" "_ id": "2"}]} returns {"docs": [{"_ index": "test_index", "_ type": "doc", "_ id": "4", "_ version": 2, "found": true, "_ source": {"username": "es", "version": 6.1 "es": "5.0"}, {" _ index ":" test_index "," _ type ":" doc "," _ id ":" 2 "," _ version ": 1," found ": true," _ source ": {" username ":" zabbix "," version ": 4.2}]}}

Welcome to join us.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.