Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what is the difference between store field and non-store field in Elasticsearch". In daily operation, I believe that many people have doubts about the difference between store field and non-store field in Elasticsearch. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the question of "what is the difference between store field and non-store field in Elasticsearch?" Next, please follow the editor to study!

The difference between store field and non-store field in Elasticsearch

When defining the mapping of index, we can specify whether some fields want store (default is not store)

So what's the difference between them? PUT / my_index {"mappings": {"properties": {"title": {"type": "string", "store": true}, "date": {"type": "date", "store": true} Content: {"type": "string"}

In fact, whether you set store to ture or false, elasticsearch will store these field for us. The difference is:

When store is false (default configuration), these field are only stored in the "_ source" field.

When store is true, the value of these field is stored in a separate field at the same level as _ source. It is also stored in _ source, so there are two copies.

So when do you need to set up store field? In general, there are two situations:

_ source field is disable in the mapping of the index. In this case, those who do not define a field as store=true will not be able to see the field in the returned query results.

The content of _ source is very large. At this point, if we want to explain the value of a field in the returned _ source document, it will be very expensive (of course, you can also define that source filtering will reduce the network overhead), in proportion to a book saved in a document, so there may be these field: title, date, content in the document. If we just want to query the book's title and date information without explaining the entire _ source (very large), we can consider setting the field of title and date to store=true at this time.

It is important to note that field store seems to reduce query overhead, but it also increases the frequency of disk visits. If you define store for all 10 field in _ source, there will be 10 disk seek operations when you query these field. When _ source is returned, there is only one disk seek operation. So this is what we need blance when we define it.

The store attribute of elasticsearch is followed by the _ source field

It is well known that the _ source field stores the original contents of the index, so what is the setting of the store property? Why does es set the default value of store to no? Is it duplicate storage when set to yes?

We write the value of a field to es, either to perform a search operation on this field (we don't know the specific id), or to perform a retrieve operation (retrieved according to id). However, if you do not explicitly set the store property of the field to yes and the _ source field enabled, you can still get the value of the field. This means that in some cases it still makes sense to keep a field from being index or store.

When you set the store property of a field to true, this is handled at the lucene level. Lucene is an inverted index that can perform a fast full-text search and return an id list of documents that meet the retrieval criteria. In addition to full-text indexing, lucene also provides the ability to store the values of fields to support queries that provide id (raw information is obtained based on id). Usually the value of field that we store at the lucene level is returned along with the search request (the value of id+field). Es does not need to store the value of every field you want to return, because the complete information for each document is stored by default, so you can follow the query structure to return all the field values you want.

In some cases, it is necessary to explicitly store certain field values: when _ source is disabled, or you do not want to parser from the source to get the field value (even if the process is automatic). Remember: getting values from each stored field requires one disk io, and if you want to get multiple field values, you need multiple disk io, but if you get multiple field values from _ source, you only need disk io once, because _ source is just a field. So in most cases, getting from _ source is fast and efficient.

The default setting in es, _ source, is enable, which stores the value of the entire document. This means that the information for the entire document can be returned when the search operation is performed. If you do not want to return the complete information of this document, you can also specify that the field,es that is required to be returned will automatically extract the value of the specified field from _ source and return it (for example, the requirements of highlighting).

You can specify some fields store as true, which means that the data for this field will be stored separately. At this point, if you ask to return field1 (store:yes), es will tell that the field1 has been stored, so it will not be loaded from _ source, but from the storage block of field1.

In which cases do you need to specify the store attribute explicitly? In most cases, it is not necessary. Getting values from _ source is fast and efficient. If your document is long and it is expensive to store _ source or get field from _ source, you can explicitly set the store property of some field to yes. The disadvantage is as mentioned above: suppose you have stored 10 field, but if you want to get the value of these 10 field, you need multiple io, if you get it from _ source, you only need to get it once, and _ source is compressed.

There is another situation: reindex from some field, when some fields are re-indexed. Read data from source and then reindex, which is obviously cheaper than reading data from some field. It is appropriate to set these fields store to yes.

At this point, the study on "what is the difference between store field and non-store field in Elasticsearch" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.