Shulou(Shulou.com)05/31 Report--

This article mainly shows you the "MongoDB fragments in the deployment and maintenance management of common matters are", the content is easy to understand, clear, hope to help you solve doubts, the following let the editor lead you to study and learn "MongoDB fragments in the deployment and maintenance management of what are common matters" this article.

Sharding is a method used by MongoDB to divide large collections into different servers (or clusters). It mainly provides a method to deal with application scenarios with high throughput and large amount of data.

By distributing the data to different machines, you don't need a powerful server to store more data and handle larger loads. The basic idea is to cut the set into small pieces, which are scattered into several pieces, each of which is only responsible for a part of the total data, and finally equalizes each slice (data migration) through an equalizer. Through a routing process called mongos, mongos knows the correspondence between the data and the slice (by configuring the server). Most usage scenarios are to solve the problem of disk space. For writes that may become worse (+ +), queries try to avoid cross-shard queries.

Timing of using slicing:

1. The machine does not have enough disks. Use slicing to solve disk space problems.

2. A single mongod can no longer meet the performance requirements for writing data. Through sharding, the writing pressure is distributed to each shard, and the resources of the sharding server are used.

3. Want to put a lot of data in memory to improve performance. As above, the sharding server's own resources are used through sharding.

Compared with the existing sub-database sub-table and partition scheme, the biggest difference of MongoDB is that it can accomplish almost everything automatically. As long as it tells MongoDB to allocate data, it can automatically maintain the balance of data between different servers.

one。 Fragmented cluster component

1.Mongos [routing]

As the access entry for requests, all requests are routed, distributed, and merged by mongos. These actions are transparent to the client driver, and users connect to mongos as if they were connected to mongod. Mongos routes the request to the corresponding Shard based on the request type and shard key.

2.Config Server [configure Server]

Store all Sharding Cluster metadata, all metadata is stored in config database

* Save the information about the chunk on each slice * Save the range of chip keys on the chunk.

3. Shard [sharding]

Store application data records.

two。 Fragmentation advantage

1. Abstract the cluster, make the cluster "invisible", and the fragmentation is transparent to the application system.

Mongos is a proprietary routing process that accurately routes requests from clients to one or a group of servers in the cluster and assembles the received responses and sends them back to the client.

two。 Ensure that the cluster is always readable and writable

The combination of MongoDB sharding and replication set features not only ensures that data is shredded to multiple servers, but also ensures that each piece of data has a corresponding backup, ensuring that when a server goes down, other slave libraries can immediately replace the broken part to continue to work. The availability and reliability of the cluster are improved.

3. Make the cluster easy to expand

When the system needs more space and resources, MongoDB enables us to expand the system capacity conveniently as needed.

three。 Considerations for sharding deployment (common errors)

1. Configuring a replicable set as a shard node is basically the same as configuring a replicable set for separate use. However, the-shardsvr parameter needs to be specified in the startup parameters.

Otherwise, error in starting database fragmentation Times: {"code": 193, "ok": 0, "errmsg": "Cannot accept sharding commands if not started with-- shardsvr"}.

two。 When creating a configuration server cluster, the witness node cannot be set.

Otherwise, error "errmsg": "Arbiters are not allowed in replica set configurations being used for config servers" is reported.

3. When configuring an Mongos instance, do not configure the dbpath parameter.

Otherwise, the service cannot be started normally if the dbpath parameter is set. Error: Error parsing INI config file: unrecognised option 'dbpath'.

4. When configuring an Mongos instance, you need to set up Keyfile.

Otherwise, you cannot start normally without setting Keyfile,Service. Error: 2018-05-10T15:30:26.791+0800 W SHARDING [mongosMain] Error initializing sharding state, sleeping for 2 seconds and trying again:: caused by:: Unauthorized: Error loading clusterID:: caused by:: not authorized on config to execute command {find: "version", readConcern: {level: "majority", afterOpTime: {ts: Timestamp 1525937413000 | 2, t: 1}}, maxTimeMS: 30000

5. Sharding collection settings.

Sharding will not be generated by default. You need to start sharding (sh.enableSharding ("DBName")) in the database before setting the collection shard (sh.shardCollection ("Collection" {chip key})).

four。 Considerations for sharding management (common commands)

1. Check shards configuration and status

Db.runCommand ({listshards:1})

two。 Check the address of the database master and whether it is partitioned

Db.getSiblingDB (config) .databases.find ()

3. Check the number of blocks

Db.chunks.count ()-need to switch to the configuration database (config)

4. View the details of the shard, including database information and scope information

Sh.status ()

5. Index is an important means to optimize query performance. When you declare an index on a shard collection, each shard defines a separate index for its own part of the collection. The sharding collection only allows unique indexes to be created on the _ id field and the shard key.

6. The underlying mechanism of split and migration MongoDB relies on two mechanisms to maintain the balance of the cluster: split and migration.

Segmentation is the process of dividing a large block of data into two smaller blocks. Migration is the process of moving data blocks between shards. The process of migration is triggered when some sharding servers contain much more block data than others. This trigger is called migration round.

6.1 Migration trigger condition

6.2 check whether the Balancer process is enabled with sh.getBalancerState ()

6.3 stop the Balancer process sh.stopBalancer () and start the Balancer process

6.4. By default, the Balancer process is running all the time. In order to reduce the running of the Balancer process to the system, you can set the run time window for the Balancer process to let the Balancer process operate in the specified time window.

6.4.1 for example, set the Balancer process to execute within the time window from 23:00 to 6:00.

Db.settings.update ({_ id: "balancer"}, {$set: {activeWindow: {start: "23:00", stop: "6:00"}, true)

6.4.2 remove the Balancer process run time window

6.5. View the range of blocks

6.5.1 if the amount of collected data is small, you can view it directly through sh.status ().

6.5.2 if the amount of data in the collection is large, sh.status () cannot reflect the chunked information of the collection. At this point, you can view the printShardingStatus (db.getSisterDB ("config"), 1) by executing the following command

6.5.3 you can also switch the command to the config database and perform db.chunks.find () view. You can enter formulation parameters, such as viewing the sharded repsms2 and collecting the blocks of the cloud-docs.PushMessageRecord (cloud-docs is the database name)

Db.chunks.find ({"shard": "repsms2", "ns": "cloud-docs.PushMessageRecord"}) .pretty ()

five。 Remarks

The amount of data managed by the sharding cluster is relatively large, and the sharding architecture is relatively complex. Therefore, must be in the business needs on the slicing, and then on the slicing, and can not be required to "dazzle" technology and slicing. In addition, after the launch, the relevant monitoring must be deployed and gradually improved.

These are all the contents of this article entitled "what are the common matters in the deployment and maintenance management of MongoDB fragments?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.