Shulou(Shulou.com)06/01 Report--

This article mainly talks about "what are the Mapping parameters". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn what are the Mapping parameters.

The status of Mapping in Elasticsearch is equivalent to schema in relational database. It can be used to define the name of the field in the index, to define the data type of the field, and to configure some fields. Since Elasticsearch 7. 0, Mapping does not care about the need to define type information, the specific reasons can be seen in the official explanation.

Data type of the field

We just mentioned that the data types of fields can be defined in Mapping, which is probably the most common feature of Mapping, so let's take a look at which data types are supported by Elasticsearch.

Simple types: text, keyword, date, long, double, boolean, ip complex types: object types, nested types Special types: geo_point, geo_shape used to describe geographical location

The data types supported by Elasticsearch are much more than these, and they will not be listed here because of the space. Let me find some common ones in my job to introduce you.

The first is the string. There are two kinds of strings in Elasticsearch: text and keyword. Among them, the string of text type can be searched by full text, and it will be acted by the word splitter.

PUT my_index

{

"mappings": {

"properties": {

"full_name": {

"type": "text"

}

}

}

}

When you set the field type to text, you can further customize the field with some parameters.

Index: marks whether this field can be searched. The default is true.

Search_analyzer: the word splitter used when being searched. The word splitter set in setting is used by default.

Fielddata: whether fields are allowed to sort and aggregate in memory. Default is false.

Meta: some metadata about fields

For fields like id, mailbox, domain name, we need to use the keyword type. Because the keyword type can support sorting, aggregation, and only precise queries.

Some students may set ID as a numeric type, which is no problem. Numeric types and keyword have their own advantages. Numerical types can be used for range search, while keyword types are more efficient. Which one to use depends on the usage scene.

Date types can be expressed in three forms in Elasticsearch

A string that can be formatted into a date type. Millisecond timestamps such as "2020-07-26" and "12:10:30 on 2015-01-01" are represented by the long type and represented by the integer type.

Within Elasticsearch, the date type is stored in a millisecond timestamp of type long, and the time zone uses the 0 time zone.

We can customize the time format. The default is strict_date_optional_time | | epoch_millis

"strict_date_optional_time_nanos" is a common date format parsing that includes at least the year, and if you want to include time, it is separated by T, such as yyyy-MM-dd'T'HH:mm:ss.SSSSSSZ or yyyy-MM-dd.

If you want to support multiple date formats simultaneously, you can use the format field

PUT my_index

{

"mappings": {

"properties": {

"date": {

"type": "date"

"format": "yyyy-MM-dd HH:mm:ss | | yyyy-MM-dd | | epoch_millis"

}

}

}

}

Mapping parameter

We just mentioned that the parameter format,Mapping, which configures Mapping's date format, provides a number of other parameters.

Analyzerboostcoercecopy_todoc_valuesdynamiceager_global_ordinalsenabledfielddatafieldsformatignore_aboveignore_malformedindex_optionsindex_phrasesindex_prefixesindexmetanormalizernormsnull_valueposition_increment_gappropertiessearch_analyzersimilaritystoreterm_vector

Let's introduce a few commonly used fields.

Fields

The first is fields, which enables the same field to achieve different purposes in different ways.

For example, we can set a string field to type text for full-text retrieval, and we can use fields to set it to type keyword for sorting and aggregation.

PUT my-index-000001

{

"mappings": {

"properties": {

"city": {

"type": "text"

"fields": {

"raw": {

"type": "keyword"

}

}

}

}

}

}

When querying, we can use city for full-text retrieval and city.raw for sorting and aggregation.

GET my-index-000001/_search

{

"query": {

"match": {

"city": "york"

}

}

"sort": {

"city.raw": "asc"

}

"aggs": {

"Cities": {

"terms": {

"field": "city.raw"

}

}

}

}

Enabled

Sometimes, we just want to use a field as a data store and do not need to be used for search, so we can disable this field, and after the field is disabled, the value it stores is not controlled by the type specified by mapping.

PUT my-index-000001

{

"mappings": {

"properties": {

"user_id": {

"type": "keyword"

}

"last_updated": {

"type": "date"

}

"session_data": {

"type": "object"

"enabled": false

}

}

}

}

In the above example, we disabled the session_data field, so you can store data in either JSON format or non-JSON format in the session_data field.

In addition to disabling individual fields, we can also disable the entire mapping directly. Let's recreate an index

PUT my-index-000002

{

"mappings": {

"enabled": false

}

}

At this point, all the fields of the document are not indexed, but are just used for storage.

It should be noted that the enabled attribute of either the specific field or the entire mapping cannot be modified, because once it is set to false,Elasticsearch, the field will not be indexed, nor will the validity of the data be verified. If dirty data is generated and then set to true, it will cause a program error.

Null_value

Null cannot be indexed or searched in Elasticsearch. The null we are talking about here is not the null of a language in a narrow sense, but all null values. For example, all values are arrays of null. In short, the definition here is that there are no values.

What about businesses that need to search for null values? Elasticsearch provides us with the parameter null_value, which can specify a value that is used instead of a null value when searching.

Take a chestnut.

PUT my-index-000001

{

"mappings": {

"properties": {

"status_code": {

"type": "keyword"

"null_value": "NULL"

}

}

}

}

We set null_value to "NULL" for the status_code field. Note here that the type of null_value must be the same as the data type you are looking for, and if the type of status_code in this example is long, then null_value cannot be set to "NULL".

Dynamic

For newly added fields:

When dynamic is set to true, once a document with new fields is written, Mapping will also be updated. When dynamic is set to false, Mapping will not be updated and the new fields cannot be indexed, but the information will appear in _ source when dynamic is set to strict, and the document writing fails.

For existing fields, once data has been written, modifying the field definition is no longer supported.

Dynamic Mapping

When we create an index, we don't have to write Mappings manually, and Elasticsearch will help us automatically identify the type of field. We call it Dynamic Mapping. But sometimes the projections may not be very accurate.

Elasticsearch automatically recognizes types based on JSON. The corresponding relationship of data types is as follows (the table is from the official website of elastic)

"JSON data type" Elasticsearch data type "nullNo field is added.true or falseboolean fieldfloating point numberfloat fieldintegerlong fieldobjectobject fieldarrayDepends on the first non-null value in the array.stringEither a date field (if the value passes date detection), a double or long field (if the value passes numeric detection) or a text field, with a keyword sub-field.

The data types of field mappings supported by Elasticsearch are specified in this document, except for these, all other type mappings need to be displayed.

Date types can be mapped by default, but Elasticsearch can only recognize dates in several formats yyyy/MM/dd HH:mm:ss | | yyyy/MM/dd | | epoch_millis. If the date_detection switch is turned off, it can only be recognized as a string.

PUT my-index-000001

{

"mappings": {

"date_detection": false

}

}

Of course, you can also specify the date format to be identified as needed, just use the dynamic_date_formats parameter.

PUT my-index-000001

{

"mappings": {

"dynamic_date_formats": ["MM/dd/yyyy"]

}

}

Elasticsearch also provides the ability to recognize string numbers as numbers, which is controlled by the numeric_detection switch.

PUT my-index-000005

{

"mappings": {

"numeric_detection": true

}

}

PUT my-index-000005/_doc/1

{

My_float: 1. 0

"my_integer": "1"

}

In this example, my_float is recognized as the float type, and my_integer is recognized as the long type.

Dynamic template

Dynamic template allows us to customize mapping and apply it to specific indexes. The definition of dynamic template is generally like this.

"dynamic_templates": [

{

"my_template_name": {

... Match conditions...

Mapping: {...}

}

}

...

]

My_template_name can be any string.

Match conditions includes match_mapping_type, match, match_pattern, unmatch, path_match, path_unmatch.

Mapping refers to what kind of mapping should be used for the matching field. Let's introduce several kinds of match conditions

Match_mapping_type

Let's start with a simple example.

PUT my-index-000001

{

"mappings": {

"dynamic_templates": [

{

"integers": {

"match_mapping_type": "long"

"mapping": {

"type": "integer"

}

}

}

{

"strings": {

"match_mapping_type": "string"

"mapping": {

"type": "text"

"fields": {

"raw": {

"type": "keyword"

"ignore_above": 256

}

}

}

}

}

]

}

}

Here we have two templates, one is to use the integer type instead of the long type, and the other is to map the string type to keyword.

Match and unmatch

These two are relatively simple. Match is the field that matches the pattern, and unmatch is the field that does not match.

PUT my-index-000001

{

"mappings": {

"dynamic_templates": [

{

"longs_as_strings": {

"match_mapping_type": "string"

"match": "long_*"

"unmatch": "* _ text"

"mapping": {

"type": "long"

}

}

}

]

}

}

In this example, what we need is a string that begins with long_, not a string field that ends with _ text.

In addition to the above three, match_pattern is used for regular matching, and path_match and path_unmatch indicate whether the path of the field matches or not.

In addition, dynamic template supports two kinds of variable substitution, which are {name} and {dynamic_type}. In fact, name is the field name, and dynamic_type is the field type detected.

At this point, I believe you have a deeper understanding of "what are the Mapping parameters?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.