Shulou(Shulou.com)06/01 Report--

Brief introduction:

Sharding refers to the process of splitting a database and distributing it on different machines. By distributing the data to different machines, you don't need a powerful server to store more data and handle larger loads. The basic idea is to cut the set into small pieces, which are scattered into several pieces, each of which is only responsible for a part of the total data, and finally equalizes each slice (data migration) through an equalizer. Through a routing process called mongos, mongos knows the correspondence between the data and the slice (by configuring the server). Most usage scenarios solve the problem of disk space, which may get worse for writes, and queries try to avoid cross-shard queries.

Timing of using slicing:

1. The machine does not have enough disks. Use slicing to solve disk space problems.

2. A single mongod can no longer meet the performance requirements for writing data. Through sharding, the writing pressure is distributed to each shard, and the resources of the sharding server are used.

3. Want to put a lot of data in memory to improve performance. As above, the sharding server's own resources are used through sharding.

Sharding architecture diagram:

Specific operations:

-- install MongoDB 3.2 music-

[root@localhost ~] # mkdir / abc

[root@localhost ~] # mount.cifs / / 192.168.100.1/rhel7 / abc

Password for root@//192.168.100.1/rhel7:

[root@localhost ~] # cd / abc

[root@localhost abc] # cd MongoDB/

[root@localhost MongoDB] # ls

Mongodb-linux-x86_64- 3.2.1.tgz mongodb-linux-x86_64-rhel70-4.0.0.tgz

[root@localhost MongoDB] # tar zxvf mongodb-linux-x86_64-3.2.1.tgz-C / opt/

[root@localhost MongoDB] # cd / opt/

[root@localhost opt] # mv mongodb-linux-x86_64-3.2.1 / / usr/local/mongodb

[root@localhost opt] # cd / usr/local/mongodb/bin/

[root@localhost bin] # ln-s / usr/local/mongodb/bin/mongo / usr/bin/mongo

[root@localhost bin] # ln-s / usr/local/mongodb/bin/mongod / usr/bin/mongod

[root@localhost bin] # mongo

[root@localhost bin] # mkdir-p / data/mongodb/mongodb {1pm 2pm 3pm 4}

[root@localhost bin] # cd / data/mongodb/

[root@localhost mongodb] # mkdir logs

[root@localhost mongodb] # cd logs/

[root@localhost logs] # touch mongodb {1,2,3,4} .log

[root@localhost logs] # chmod 777 * .log

[root@localhost logs] # ulimit-u 25000

[root@localhost logs] # ulimit-n 25000

[root@localhost logs] # cd / usr/local/mongodb/bin/

-- configure server--

[root@localhost bin] # vim mongodb1.conf

Port=37017

Dbpath=/data/mongodb/mongodb1

Logpath=/data/mongodb/logs/mongodb1.log

Logappend=true

Fork=true

MaxConns=5000

StorageEngine=mmapv1

Configsvr=true

-when one node runs out of memory, allocate memory from other nodes-

[root@localhost bin] # sysctl-w vm.zone_reclaim_mode=0

Vm.zone_reclaim_mode = 0

[root@localhost bin] # echo never > / sys/kernel/mm/transparent_hugepage/enabled

[root@localhost bin] # echo never > / sys/kernel/mm/transparent_hugepage/defrag

[root@localhost bin] # mongod-f mongodb1.conf

[root@localhost bin] # mongo-- port 37017

[root@localhost bin] # cp-p mongodb1.conf mongodb2.conf

[root@localhost bin] # vim mongodb2.conf

Port=47017

Dbpath=/data/mongodb/mongodb2

Logpath=/data/mongodb/logs/mongodb2.log

Logappend=true

Fork=true

MaxConns=5000

StorageEngine=mmapv1

Shardsvr=true

-sharding server-

[root@localhost bin] # cp-p mongodb2.conf mongodb3.conf

[root@localhost bin] # vim mongodb3.conf

Port=47018

Dbpath=/data/mongodb/mongodb3

Logpath=/data/mongodb/logs/mongodb3.log

Logappend=true

Fork=true

MaxConns=5000

StorageEngine=mmapv1

Shardsvr=true

[root@localhost bin] # mongod-f mongodb2.conf

[root@localhost bin] # mongod-f mongodb3.conf

[root@localhost bin] #. / mongos-help

-- start routing server--

[root@localhost bin] #. / mongos-- port 27017-- fork-- logpath=/usr/local/mongodb/bin/route.log-- configdb 192.168.120.129 port 37017-- chunkSize 1

-- enable sharding server--

[root@localhost bin] # mongo

MongoDB shell version: 3.2.1

Connecting to: test

Server has startup warnings:

2018-09-14T14:47:11.104+0800 I CONTROL [main] * * WARNING: You are running this process as the root user, which is not recommended.

2018-09-14T14:47:11.104+0800 I CONTROL [main]

Mongos > show dbs

Config 0.031GB

Empty under mongos > sh.status () # shards and no sharding server

Mongos > sh.addShard ("192.168.120.129 purl 47017")

{"shardAdded": "shard0000", "ok": 1}

Mongos > sh.addShard ("192.168.120.129 purl 47018")

{"shardAdded": "shard0001", "ok": 1}

Mongos > sh.status () # two fragments have been added

Shards:

{"_ id": "shard0000", "host": "192.168.120.129 virtual 47017"}

{"_ id": "shard0001", "host": "192.168.120.129 virtual 47018"}

-- sharding function--

# create a kgc library and write data

Mongos > show dbs

Config 0.031GB

Mongos > use kgc

Switched to db kgc

Mongos > db.users.insert ({"id": 1, "name": "zhangsan"})

WriteResult ({"nInserted": 1})

Mongos > for (var iTunes 2 witch I show dbs

Config 0.031GB

Kgc 0.078GB

# View the first five lines of data

Mongos > db.users.find () limit (5)

# View database sharding information

Mongos > sh.status ()

Databases:

{"_ id": "kgc", "primary": "shard0000", "partitioned": false}

# enable database sharding

Mongos > sh.enableSharding ("kgc")

{"ok": 1}

Mongos > sh.status ()

Databases:

{"_ id": "kgc", "primary": "shard0000", "partitioned": true}

# create an index on users table

Mongos > db.users.createIndex ({"id": 1})

# Table fragmentation

Mongos > sh.shardCollection ("kgc.users", {"id": 1})

{"collectionsharded": "kgc.users", "ok": 1}

Mongos > sh.status ()

-- sharding management--

Mongos > use kgc

Switched to db kgc

Mongos > for (var iTune1witi db.users2.createIndex ({"id": 1})

# sharding

Mongos > sh.shardCollection ("kgc.users2", {"id": 1})

{"collectionsharded": "kgc.users2", "ok": 1}

Mongos > db.users2.stats ()

# add tags

Mongos > sh.addShardTag ("shard0000", "sales00")

Mongos > sh.addShardTag ("shard0001", "sales01")

Mongos > sh.status ()

Shards:

{"_ id": "shard0000", "host": "192.168.120.129 host", "tags": ["sales00"]}

{"_ id": "shard0001", "host": "192.168.120.129 host", "tags": ["sales01"]}

# connecting to the configuration server

[root@localhost bin] # mongo-- port 37017

Configsvr > use config

Switched to db config

Configsvr > show collections

[root@localhost bin] # cd / usr/local/mongodb/bin/

[root@localhost bin] # cp-p mongodb3.conf mongodb4.conf

[root@localhost bin] # vim mongodb4.conf

Port=47019

Dbpath=/data/mongodb/mongodb4

Logpath=/data/mongodb/logs/mongodb4.log

Logappend=true

Fork=true

MaxConns=5000

StorageEngine=mmapv1

Shardsvr=true

# start

[root@localhost bin] # mongod-f mongodb4.conf

[root@localhost bin] # mongo

# add shards

Mongos > sh.addShard ("192.168.120.129 purl 47019")

# View status

Mongos > sh.status ()

Chunks:

Shard0000 4

Shard0001 4

Shard0002 3

# Delete shards

Mongos > use admin

Switched to db admin

Mongos > db.runCommand ({"removeshard": "192.168.120.129 purl 47019"})

# View status

Mongos > sh.status ()

Chunks:

Shard0000 6

Shard0001 5

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.