Shulou(Shulou.com)05/31 Report--

Editor to share with you the example analysis of MongoDB fragmentation keys. I hope you will get something after reading this article. Let's discuss it together.

MongoDB version: 3.6

1. Category of sharding key

1. Ascending chip key

Ascending sequence key for example: date-time field, self-increment field.

two。 Randomly distribute chip keys

Randomly distribute slice keys such as user name, email name, UUID, MD5 value, or other columns with irregular values.

3. Location-based chip key

Location-based chip keys such as IP, latitude and longitude, address of residence, etc.

2. Slicing strategy

1. Range slicing

When you create a shard, only one block {"username": {"$minKey": 1}}-> {"username": {"$maxKey": 1}} on: rs-a Timestamp (1,0) is created on the main shard.

There must be at least 3 different values before the block is split, and the same value will only be in one chunk. For example, if you partition a name field by inserting "a" into the name field, it will store the main shard in {"username": {"$minKey": 1}-- > {"username": {"$maxKey": 1}} until name has three different values, such as "a", "b" and "c". Of course, this is just a test, and in reality, such coarse-grained fields will not be shredded separately.

2.hashed fragmentation

When you create a shard, two data blocks are created on each shard by default. But at present, there is no data on each block.

3. Combined slicing

Combinatorial slicing is a better choice for slicing, and a good combinatorial slicing can solve the hot spot and random reading IO problems at the same time. For example:

Sh.shardCollection ("test.bbbb", {"username": 1, "_ id": 1})

4. Label fragmentation

For example, for some log non-query documents, you can insert them only into a fragment through a tag. For example

Sh.addTagRange ("test.log", {"_ id": {"$minKey": 1}}, {"_ id": {"$maxKey": 1}}, "tag_rs-a")

You can view the set tag information in the tag document in the config library.

Use configdb.tags.find ()

III. Label

A specific range of data can be placed in a specified shard through a label.

Save the data in the range of {"_ id": 18000}-> {"_ id": 26000} to the rs-a shard, which spans two data blocks.

1. Specify tag for shards

Sh.addShardTag ("rs-a", "tag_rs-a"); sh.addShardTag ("rs-b", "tag_rs-b"); sh.addShardTag ("rs-c", "tag_rs-c")

two。 Create a rule

Sh.addTagRange ("test.person", {"_ id": 18000}, {"_ id": 26000}, "tag_rs-a")

The data {"_ id": 18000}-> {"_ id": 26000} has been moved to the rs-a shard.

IV. Slicing cases

The sharding strategy is not absolutely good or bad, so choose different slicing strategies for different business scenarios.

1. Fragment scenario

1. All the pieces are read and written evenly.

two。 Data access is uniform rather than random; since new data is first created in memory, try to avoid the need to access new data from disk.

3. Try to avoid the hot data being cleaned out of memory due to the data movement of the data block, which causes the data to be loaded into memory from the disk.

4. Combined field sharding may be an ideal sharding scheme.

Sharding key formula: {coarseLocality:1,search:1}

CoarseLocality: should be a large-grained local field. For example, the Mont month ascending field.

Search: is a field that is often used to find.

two。 Slicing case

Case 1. Problems with using date fields, self-increment fields, and timestamp slicing

There is a site browsing record table with a createtime field to record the insertion time of each day's records.

It is not appropriate to use the createtime field as a shard field for this type of document, because both reading and writing may be focused on the latest shard. There is the same problem with using self-increment fields.

Case 2. Large granularity field slicing problem

There is a table of user documents on five continents with a continent field that stores the state where the user is located.

If you use continent as the sharding field, there will be the following problems:

1. The granularity of the shard is too large, which will cause the data of each shard to be very large and impossible to divide again. It may also lead to insufficient disk space.

two。 It may cause the traffic of one shard to be much more than that of other shards at a certain point in time.

Case 3: combined sharding using month and user name

There is a collection of user operation records, and the business needs to query the user's operation records for the last month. The collection has month and username keys

Using {month:1,userName:1} sharding scenarios is as follows:

Month ensures that hot data is superior to memory.

UserName: ensure the randomness of the data and avoid the problem of centralized overheating.

The existing problem: because the new document does not exist for many months, it will cause the new data to be inserted into the last shard, there is a hot read and write problem, and finally the data block is moved through the equalizer.

Data testing

Sh.shardCollection ("test.news", {"month": 1, "username": 1})

-insert January data 100000 record

For (var iTuno month: {"$maxKey": 1}, "username": {"$maxKey": 1}} on: rs-a Timestamp (3,1)

After the data is inserted, the equalizer distributes a block on the rs-c to the rs-a

-insert all month data.

For (var axi1tera)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.