Shulou(Shulou.com)06/01 Report--

Introduction

Nowadays, in the era of big data, the database application of massive data and throughput has caused great pressure on the performance of the single machine, and some problems will occur, such as CPU exhaustion, storage pressure, depletion of available resources and so on.

Then there is a new technology, slicing technology. It is the method used by MongoDB to split large collections into different servers. It can accomplish almost everything automatically. As long as it tells MongoDB the data to be allocated, it can automatically maintain a balanced storage of data between different servers.

The sharding mechanism provides three advantages: abstracting the cluster, making the cluster "invisible" to ensure that the cluster is always readable and writable, making it easy for the cluster to expand the sharding cluster architecture:

Config server: stores all nodes in the cluster and shard data routing information. 3 config server nodes need to be configured by default.

Mongos: provides access to external applications. All operations are performed through mongos. Generally, there are multiple mongos nodes, data migration and automatic data balancing.

Mongod: stores application data records, usually with multiple Mongod nodes, to achieve the purpose of data fragmentation.

System environment:

1. The three servers are configured as follows:

IP address routing server configuration server shard1shard2shard3192.168.96.102000021000270012700227003192.168.96.112000021000270012700227003192.168.96.122000021000270012700227003

two。 Operating system: CentOS 7

3. Turn off firewall and Selinux features

4. Software package: mongodb-linux-x86_64-rhel70-3.6.6.tgz password: 4h77

Begin to deploy the first part of the three servers installation and configuration 1. Extract the installation package

Tar-zxvf mongodb-linux-x86_64-rhel70-3.6.6.tgz-C / usr/local

two。 Switch to the software directory

Cd / usr/local

3. Rename directory name

Mv mongodb-linux-x86_64-rhel70-3.6.6 mongodb

4. Create routing, configuration, configuration of sharding server, data, log management directory mkdir-p / data/mongodb/confmkdir-p / data/mongodb/mongos/logmkdir-p / data/mongodb/config/datamkdir-p / data/mongodb/config/log# cycle create sharding server directory for i in 12 3do mkdir-p / data/mongodb/shard$i/data mkdir-p / data/mongodb/shard$i/logdone5. Create an administrative user

Useradd-M-s / sbin/nologin mongo

6. Modify directory permissions

Chown-R mongo.mongo / usr/local/mongodb

Chown-R mongo.mongo / data/mongodb

7. Add to the system environment variable

Echo "PATH=/usr/local/mongodb/bin:$PATH" > > / etc/profile

8. Reload environment variabl

Source / etc/profile

9. System parameter optimization (the current user is permanently effective) 1. Edit the current user .bash _ profile file vim ~ / .bash_profile2. Append the following optimization parameter ulimit-n 25000 / / the maximum number of files that can be opened ulimit-u 25000 / / the maximum number of processes available by the user sysctl-w vm.zone_reclaim_mode=0 / / when memory is insufficient, allocate memory from other nodes. Reload source ~ / .bash_profile starting with the Centos7 version, MongoDB recommends that you turn off the THP feature of the system, otherwise it may cause performance degradation. 1. Edit the rc.local file vim / etc/rc.d/rc.local2. Add the following at the end of the article. / / optimize a pair of large memory pages if test-f / sys/kernel/mm/transparent_hugepage/enabled; then echo never > / sys/kernel/mm/transparent_hugepage/enabled fi if test-f / sys/kernel/mm/transparent_hugepage/defrag; then echo never > / sys/kernel/mm/transparent_hugepage/defrag fi3. Give the rc.local file execution permissions chmod + x / etc/rc.d/rc.local

Special reminder: please check whether all three servers have been configured and started.

The second part is to configure the server (the three configuration steps are the same) 1. Create a profile

Vim / data/mongodb/conf/config.conf

# configuration file content pidfilepath = / data/mongodb/config/log/config-srv.piddbpath = / data/mongodb/config/datalogpath = / data/mongodb/config/log/config-srv.loglogappend = truebind_ip = 0.0.0.0port = 21000fork = true#declare this is a config db of a cluster;configsvr = true# replication set name replSet=configs# sets the maximum number of connections maxConns=200002. Create a startup script:

Vim / usr/lib/systemd/system/mongo-config.service

[Unit] Description=mongodb-configAfter=network.target remote-fs.target nss- lookup.target [service] Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod-f / data/mongodb/conf/config.confExecReload=/bin/kill-s HUP $MAINPIDExecStop=/usr/local/mongodb/bin/mongod-- shutdown-f / data/mongodb/conf/config.confPrivateTmp= truth [install] WantedBy=multi-user.target3. Start the configsrv server systemctl daemon-reloadsystemctl enable mongo-configsystemctl start mongo-config

Special reminder: after all three servers have been configured, perform the following initialization steps

4. Initialize the replication set (log in to any server to initialize the replication set)

Mongo-port 21000

/ / configuration > config= {"_ id": "configs", members: [{"_ id": 0, "host": "192.168.96.10 members 21000"}, {"_ id": 1, "host": "192.168.96.11 configs"}, {"_ id": 2 "host": "192.168.96.12 config 21000"}} / / initialize > rs.initiate (config) / / View status > rs.status () part III sharding server 1, shard1 sharding server 1. Create a profile

Vim / data/mongodb/conf/shard1.conf

Pidfilepath = / data/mongodb/shard1/log/shard1.piddbpath = / data/mongodb/shard1/datalogpath = / data/mongodb/shard1/log/shard1.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27001fork = true# replication set name replSet=shard1#declare this is a shard db of a cluster;shardsvr = true# sets the maximum number of connections maxConns=200002. Create a startup script

Vim / usr/lib/systemd/system/mongo-shard1.service

[Unit] Description=mongodb-shard1After= mongo-config.target network.target [service] Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod-f / data/mongodb/conf/shard1.confExecReload=/bin/kill-s HUP $MAINPIDExecStop=/usr/local/mongodb/bin/mongod-- shutdown-f / data/mongodb/conf/shard1.confPrivateTmp= truth [install] WantedBy=multi-user.target3. Start the service systemctl daemon-reloadsystemctl enable mongo-shard1systemctl start mongo-shard1

Special reminder: after all three servers have been configured, perform the following initialization steps

4. Initialize the replication set (log in to any server to initialize the replication set)

Mongo-port 27001

> use admin > config= {"_ id": "shard1", members: [{"_ id": 0, "host": "192.168.96.10 host 27001", arbiterOnly:true}, {"_ id": 1, "host": "192.168.96.11 id 27001"}, {"_ id": 2, "host": "192.168.96.12 id 27001"}]} > rs.initiate (config) > rs.status () II, shard2 sharding server 1. Create a profile

Vim / data/mongodb/conf/shard2.conf

Pidfilepath = / data/mongodb/shard2/log/shard2.piddbpath = / data/mongodb/shard2/datalogpath = / data/mongodb/shard2/log/shard2.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27002fork = true# replication set name replSet=shard2#declare this is a shard db of a cluster;shardsvr = true# sets the maximum number of connections maxConns=200002. Create a startup script

Vim / usr/lib/systemd/system/mongo-shard2.service

[Unit] Description=mongodb-shard2After= mongo-config.target network.target [service] Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongod-f / data/mongodb/conf/shard2.confExecReload=/bin/kill-s HUP $MAINPIDExecStop=/usr/local/mongodb/bin/mongod-- shutdown-f / data/mongodb/conf/shard2.confPrivateTmp= truth [install] WantedBy=multi-user.target3. Start the service systemctl daemon-reloadsystemctl enable mongo-shard2systemctl start mongo-shard2

Special reminder: after all three servers have been configured, perform the following initialization steps

4. Initialize the replication set (log in to any server to initialize the replication set)

Mongo-port 27002

> use admin > config= {"_ id": "shard2", members: [{"_ id": 0, "host": "192.168.96.10 host 27002"}, {"_ id": 1, "host": "192.168.96.11 host 27002", arbiterOnly:true}, {"_ id": 2, "host": "192.168.96.12 id 27002"}]} > rs.initiate (config); > rs.status () shard3 sharding server 1. Create a profile

Vim / data/mongodb/conf/shard3.conf

Pidfilepath = / data/mongodb/shard3/log/shard3.piddbpath = / data/mongodb/shard3/datalogpath = / data/mongodb/shard3/log/shard3.loglogappend = truejournal = truequiet = truebind_ip = 0.0.0.0port = 27003fork = true# replication set name replSet=shard3#declare this is a shard db of a cluster;shardsvr = true# sets the maximum number of connections maxConns=200002. Create a startup script

Vim / usr/lib/systemd/system/mongo-shard3.service

```[Unit]

Description=mongodb-shard3

After= mongo-config.target network.target

[Service]

Type=forking

User=mongo

Group=mongo

ExecStart=/usr/local/mongodb/bin/mongod-f / data/mongodb/conf/shard3.conf

ExecReload=/bin/kill-s HUP $MAINPID

ExecStop=/usr/local/mongodb/bin/mongod-- shutdown-f / data/mongodb/conf/shard3.conf

PrivateTmp=true

[Install]

WantedBy=multi-user.target

# 3. Create a startup script

Systemctl daemon-reload

Systemctl enable mongo-shard3

Systemctl start mongo-shard3

* * Special reminder: after all three servers are configured, perform the following initialization steps * * # 4. Initialize the replication set (log in to any server to initialize the replication set) > mongo-- port 27003

Use admin

Config= {"_ id": "shard3", members: [{"_ id": 0, "host": "192.168.96.10 members 27003"}, {"_ id": 1, "host": "192.168.96.11 shard3 27003"}, {"_ id": 2, "host": "192.168.96.12 shard3 27003", arbiterOnly:true}]}

Rs.initiate (config)

Rs.status ()

The fourth part is mongos routing server 1. Create a profile

Vim / data/mongodb/conf/mongos.conf

Pidfilepath = / data/mongodb/mongos/log/mongos.pidlogpath = / data/mongodb/mongos/log/mongos.loglogappend = truebind_ip = 0.0.0.0port = 20000fork = the address of the configuration server that true# listens on: Port, (important, important) configdb = configs/192.168.96.10:21000192.168.96.11:21000192.168.96.12:21000# sets the maximum number of connections maxConns=200002. Create a startup script

Vim / usr/lib/systemd/system/mongos.service

[Unit] Description=Mongo Router ServiceAfter=mongo- config. Service [service] Type=forkingUser=mongoGroup=mongoExecStart=/usr/local/mongodb/bin/mongos-- config / data/mongodb/conf/mongos.confRestart=on-fail ure[ install] WantedBy=multi-user.target3. Start the service systemctl daemon-reloadsystemctl enable mongossystemctl start mongos

Special reminder: after all three servers have been configured, perform the following steps

4. Start the sharding function (log in to any server)

Mongo-port 20000

> use admin > sh.addShard ("shard1/192.168.96.10:27001192.168.96.11:27001192.168.96.12:27001") sh.addShard ("shard2/192.168.96.10:27002192.168.96.11:27002192.168.96.12:27002") sh.addShard ("shard3/192.168.96.10:27003192.168.96.11:27003192.168.96.12:27003") > sh.status () 5. Check whether the shard is successful > use school > for (iShop 1: "school"}}; / / specify the collection and key of the shard required by the school database > db.runCommand ({shardcollection: "school.user", key: {id:1}}) / / View status > db.user.stats ()

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.