Shulou(Shulou.com)06/01 Report--

For the basics of MongoDB, please refer to: https://blog.51cto.com/kaliarch/2044423

MongoDB (replica set) Please refer to: https://blog.51cto.com/kaliarch/2044618

I. Overview

1.1 background

In order to solve the problem that the database of mongodb on each slave node of replica set is a full copy of the database, there is a great challenge in the scenario of high concurrency and large amount of data under the node pressure. At the same time, considering the expansibility of the later mongodb cluster when the data pressure is huge, the slicing mechanism is introduced to deal with the massive data.

1.2 the concept of fragmentation

Slicing is the process of splitting the database and distributing it on different machines. It can store more data and handle larger loads without a powerful server. In the total data, the collection is divided into small pieces, and these blocks are distributed into several slices. Each slice only loads a part of the total data and operates through a routing process of a component mongos that knows the corresponding relationship between the data and the slices.

1.3 basic components

It makes use of four components: mongos,config server,shard,replica set

Mongos: the entry of database cluster requests. All requests need to be coordinated by mongos, and there is no need to use programs to select routes at the application level. Mongos itself is a request distribution center, which is responsible for distributing external requests to the corresponding shard servers. As a unified request entry, mongos is generally required to HA mongos in order to prevent single node failure.

Config server: configuration server that stores the configuration of all database metadata (sharding, routing). Mongos itself does not physically store sharding server and data routing information, but caches it in memory to read data. When mongos starts for the first time or restarts later, it will load configuration information from config server. If the configuration server information is updated, it will notify all mongos to update its status, so as to ensure accurate request routing. Multiple config server is usually required in production environment. Prevent the loss of a single node in the profile.

Shard: in the traditional sense, if there is a large amount of data, the pressure of storing 1T on a single server is very great, no matter considering the hard disk of the database, network IO, and the bottleneck of CPU and memory, if more than one server allocates 1T of data, it will be estimably small data on each platform. As long as the slicing rule is set up in the mongodb cluster, the database can be operated through mongos. The corresponding operation request can be automatically forwarded to the corresponding back-end sharding server.

Replica set: in the overall mongodb cluster architecture, for the corresponding sharding node, if a single machine goes offline, the data corresponding to the whole cluster will be partially missing, which cannot happen. Therefore, replica set is required for shard nodes to ensure the reliability of the data. In production environment, it is usually 2 copies + 1 arbitration.

1.4 Architecture Diagram

II. Installation and deployment

2.1 basic environment

In order to save servers, use multi-instance configuration, three mongos, three config server, a single server running inaccessible roles of the shard (for later data slicing evenly, the three shard will play different roles on each server. ), replica set is used in a node to ensure high availability, and the corresponding CVM and port information is as follows:

Hostnam

IP address

Component mongos

Component config server

Shard

Mongodb-1

172.20.6.10

Port: 20000

Port: 21000

Master node: 22001

Replica node: 22002

Arbitration node: 22003

Mongodb-2

172.20.6.11

Port: 20000

Port: 21000

Arbitration node: 22001

Master node: 22002

Replica node: 22003

Mongodb-3

172.20.6.12

Port: 20000

Port: 21000

Replica node: 22001

Arbitration node: 22002

Master node: 22003

2.2. Installation and deployment

2.2.1 Software download directory creation

Wget-c https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel62-3.4.10.tgztar-zxvf mongodb-linux-x86_64-rhel62-3.4.10.tgzln-sv mongodb-linux-x86_64-rhel62-3.4.10 mongodbecho "PATH=$PAHT:/usr/local/mongodb/bin" > / etc/profile.d/mongodb.shsource / etc/profile.d/mongodb.sh

2.2.2 create a directory

Create directories and log files in mongodb-1/mongodb-2/mongodb-3

Mkdir-p / data/mongodb/mongos/ {log,conf} mkdir-p / data/mongodb/mongoconf/ {data,log,conf} mkdir-p / data/mongodb/shard1/ {data,log,conf} mkdir-p / data/mongodb/shard2/ {data,log,conf} mkdir-p / data/mongodb/shard3/ {data,log Conf} touch / data/mongodb/mongos/log/mongos.logtouch / data/mongodb/mongoconf/log/mongoconf.logtouch / data/mongodb/shard1/log/shard1.logtouch / data/mongodb/shard2/log/shard2.logtouch / data/mongodb/shard3/log/shard3.log

2.2.3 configuring config server replica set

After the mongodb3.4 version, the configuration server is required to also be created as a replica set, where the replica set name: replconf

Configure the config server replica set profile on the three servers and start the service

Cat > / data/mongodb/mongoconf/conf/mongoconf.conf/data/mongodb/shard1/conf/shard.conf

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.