Shulou(Shulou.com)06/01 Report--

I. on slicing

1. What is fragmentation

Database applications with high data volume and throughput will cause great pressure on the performance of the stand-alone machine, large queries will exhaust the CPU of the stand-alone machine, and the large amount of data will cause greater storage pressure on the stand-alone machine, which will eventually deplete the memory of the system and transfer the pressure to the disk IO.

MongoDB sharding is a method of using multiple servers to store data to support huge data storage and data manipulation. Slicing technology can meet the needs of the massive growth of MongoDB data. When one MongoDB server is not enough to store massive data or provide acceptable read and write throughput, we can split the data on multiple servers so that the database system can store and process more data.

2. Advantages of MongoDB sharding

(1) allocation provides a way to deal with high throughput and large amount of data.

(2) the use of shards reduces the number of requests to be processed per shard, so through horizontal scaling, the cluster can increase its own storage capacity and throughput. For example, when inserting a piece of data, the application only needs to access the shard that stores the data.

(3) the use of sharding reduces the data stored in each shard.

(4) the advantage of sharding is to provide an architecture similar to linear growth, improve data availability and improve the performance of large database query servers. Sharding can be used when MongoDB single point database server storage becomes a bottleneck, when the performance of a single point database server becomes a bottleneck, or when large applications need to be deployed to make full use of memory. The structure of MongoDB shards is as follows:

3. The composition of MongoDB fragmentation cluster.

The MongoDB sharding cluster has the following three main components:

Shard: sharding server, which is used to store actual data blocks. In the actual production environment, a shard server role can be assumed by a Replica Set composed of several servers to prevent a single point of failure of the host.

Config Server: configuration server that stores configuration information for the entire sharded cluster, including chunk information.

Router: front-end routing, which allows clients to access, and makes the whole cluster look like a single database, so that front-end applications can be used transparently.

The composition of the MongoDB sharding cluster is shown below:

II. Simple deployment of MongoDB clusters

System environment:

1 route real queue (port 27017)

1 configuration real column (port 37017)

2 sets of Shard real train (port 47017 dint 47018)

1. Compile and install MongoDB3.2 with source code

Yum install openssl-devel-y / / install the dependency package tar zxvf mongodb-linux-x86_64-3.2.1.tgz-C / opt/ extract the source package mv mongodb-linux-x86_64-3.2.1 / / usr/local/mongodb / / move the source package to the local installation software directory mkdir-p / data/mongodb/mongodb {1mlgml3} / / create a multi-implementation library mkdir / data/mongodb/logstouch / data/mongodb/logs/mongodb {1jin2 3} .log / / create the corresponding log file chmod-R777 / data/mongodb/logs/*.log / / ulimit-n 25000 / / set the number of files that can be opened by each process ulimit-u 25000 / / set the maximum number of processes

2. Deploy the configuration server (config server)

Vim mongodb1.conf

Add the following

Port=37017 / / configure the port number of the server dbpath=/data/mongodb/mongodb1 / / the path of the factual library logpath=/data/mongodb/logs/mongodb1.log / / log path logappend=true / / enable the log function fork=true / / enable the daemon process maxConns=5000 / / the maximum number of port connections storageEngine=mmapv1 / / specify the storage engine configsvr=true / / enable the configuration service function

Optimize the configuration server:

When one node runs out of memory, allocate memory from other nodes

Sysctl-w vm.zone_reclaim_mode=0echo never > / sys/kernel/mm/transparent_hugepage/enabledecho never > / sys/kernel/mm/transparent_hugepage/defrag

Make a software link to the common commands under mongodb/bin

Ln-s / usr/local/mongodb/bin/mongo / usr/bin/mongoln-s / usr/local/mongodb/bin/mongod / usr/bin/mongod

Enable the real column of the configuration server

Mongod-f / usr/local/mongodb/bin/mongodb1.conf

3. Deploy sharding server

Cp-p mongodb1.conf mongodb2.conf / / copy master configuration file corresponding to the first shard cp-p mongodb2.conf mongodb3.conf / / corresponding to the second vi mongodb2.conf / / modify the corresponding port number and the corresponding path

Mongod-f mongodb2.conf / / start the sharding server mongod-f mongodb3.conf respectively

4. Start the routing server

. / mongos-- port 27017-- fork-- logpath=/usr/local/mongodb/bin/route.log-- configdb 192.168.80.100 fork 37017-- chunkSize 1

5. Start the sharding server in the routing instance

Mongo / / enter the real column, the default port 27017show dbs / / to view the configured size

Sh.status () / / View sharding status. There is no sharding server.

Sh.addShard ("192.168.80.100 47017") / / add sharding server sh.addShard ("192.168.80.100 47018")

Sh.status ()

6. Verify the data slicing function

Use kgcfor (var iTunes 1)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.