Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the introduction of mongodb and the principle of distributed application". In daily operation, I believe many people have doubts about the introduction of mongodb and the principle of distributed application. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "introduction of mongodb and distributed application principle". Next, please follow the editor to study!

1. About mongodb

MongoDB is a database based on distributed file storage. Written in C++ language. Designed to provide scalable high-performance data storage solutions for WEB applications. MongoDB is a product between relational database and non-relational database, which is the most abundant and most similar to relational database. The data structure he supports is very loose, which is similar to json's bjson format, so it can store more complex data types. The most important feature of Mongo is that the query language it supports is very powerful, and its syntax is somewhat similar to the object-oriented query language. It can almost achieve most of the functions similar to the single table query of relational database, and also supports the establishment of data indexing.

Second, the principle of mongodb distributed application

MongoDB cluster includes a certain number of mongod (sharded storage data), mongos (routing processing), config server (configuration node), clients (client) and arbiter (arbitration node: to elect the master node of a sharded storage data node).

1. Shards: a shard is a group of mongod, usually a group of two, master-slave or each other master-slave, the data in this group of mongod is the same, specifically see "mongodb distributed data replication". The data is segmented in an ordered manner, and the data on each shard is a certain range of data blocks, so it can support the range query of specified shards, which is similar to the BigTable of google. A data block has a specified maximum capacity, and once the capacity of a data block grows to the maximum capacity, the data block will be split into two; when there is too much data in the shard, the data block will be migrated to other shards in the system. In addition, when new shards are added, the data blocks are also migrated.

2. Mongos: there can be multiple control centers, which are responsible for routing and coordinating operations, making the cluster look like a whole system. Mongos can run on any server, some on the shards server and some on the client server. When mongos starts, it needs to get the basic information from config servers, then accept the request from the client, route it to the shards server, and then organize the returned results and send them back to the client server.

3. Config server: stores cluster information, including shard and block data information. The main storage block data information, each config server has a copy of all block data information, in order to ensure the consistency of the data on each config server.

4. Shard key: in order to split a dataset, you need to develop a sharded key format, which is similar to the key format used for indexing, and is usually composed of one or more fields to distribute data, such as:

{name: 1}

{_ id: 1}

{lastname: 1, firstname: 1}

{tag: 1, timestamp:-1}

The fragmentation of mongoDB is ordered storage (1 is ascending order,-1 is descending order), and the data adjacent to shard key usually exists in the same service.

(data block).

III. Mongodb distributed deployment

There are many ways to deploy a server. First of all, each config server, mongos, and mongod can be a separate server, but this can lead to waste of some servers, such as config server. The following figure shows the cluster deployment shared by physical machines. No additional machines are required.

Of course, there are other solutions, such as deploying mongos on all mongod (server1-6) or deploying mongos on each application server (server7). One advantage of this deployment is that the communication between appserver and mongos is based on localhost interface, reducing communication costs. Of course, this is an official statement, but I have an idea, although reducing the communication costs between appserver and mongos, the communication costs between mongos and mongod have increased, and the deployment of mongos on appserver is not very conducive to sa management, mongoDB should be a relatively independent system, and the coupling of applications should be minimized as far as possible, in case the application wants to change the database, it can also reduce some workload.

IV. Mongodb distributed installation

1. Download

Wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.4.tgz

2. Installation

Tar zxvf mongodb-linux-2.0.4.tgz

Cp-fr mongodb-linux-*2.0.4/* / data/mongodb/

Mkdir-p / data/mongodb/data/ # create a data storage directory

Mkdir-p / data/mongodb/log/ # create log storage directory

Mkdir-p / data/mongodb/config/ # create configuration storage directory

Mkdir-p / data/mongodb/arbiter/ # create the quorum node storage directory

3. Stand-alone mode and a parameter description

Mongod-fork-bind_ip 127.0.0.1-port 11811-dbpath / data0/mongodb/data-directoryperdb-logpath / data0/mongodb/log/db1.log-logappend-nohttpinterface

Netstat-ntlp | grep mongod

Simple parameter description:

-logpath log file path

-master designated master machine

-slave specified as slave machine

-source specifies the IP address of the host machine

-pologSize specifies that the log file size does not exceed 64m. Because resync is very heavy and time-consuming, it is best to avoid resync by setting a large enough oplogSize (the default oplog size is 5% of the free disk size).

-add at the end of the logappend log file

-port enable port number

-fork runs in the background

-only specifies which database to replicate only

-slavedelay refers to the time interval detected from replication

-whether auth requires authentication permissions to log in (username and password)

-noauth does not require authentication permission to log in (username and password)

5. Cluster mode mongos,mongod,configsvr

1.Shard fragmentation--

The first group of pieces

192.168.200.226:

/ data/mongodb/bin/mongod-replSet rep1-fork-port 11813-maxConns 65535-dbpath / data/mongodb/data-directoryperdb-logpath / data/mongodb/log/db.log-logappend-nohttpinterface

# start a data node of rep1 sharding

/ data/mongodb/bin/mongos-- configdb 192.168.201.226 port 11812-- port 11811-- fork-- chunkSize 256-- logpath / data/mongodb/log/ms.log # start the routing node and read the configuration file of the configuration node on 201.226 (start after the configuration node starts)

/ data/mongodb/bin/mongod-replSet rep1-fork-port 11814-dbpath / data/mongodb/arbiter-directoryperdb-logpath / data/mongodb/log/arbiter.log-logappend-nohttpinterface

# start the arbitration node of sharding

192.168.201.226:

/ data/mongodb/bin/mongod-replSet rep1-fork-port 11813-maxConns 65535-dbpath / data/mongodb/data-directoryperdb-logpath / data/mongodb/log/db.log-logappend-nohttpinterface

# start the second data node of rep1 sharding

/ data/mongodb/bin/mongos-- configdb 192.168.201.226 port 11812-- port 11811-- fork-- chunkSize 256-- logpath / data/mongodb/log/ms.log

# start the second routing node (start after the configuration node starts)

2.ConfigServer

# start config server

192.168.201.226:

/ data/mongodb/bin/mongod-configsvr-dbpath / data/mongodb/config-port 11812-fork-logpath / data/mongodb/log/mc.log

# start the configuration node. Note that the configuration node should be started first, otherwise the routing node will fail if it fails to read the configuration node information.

3.Mongos routing

# start mongos and specify config server, chunkSize 256m

192.168.201.226:

/ data/mongodb/bin/mongos-- configdb 192.168.201.226 port 11812-- port 11811-- fork-- chunkSize 256-- logpath / data/mongodb/log/ms.log

# start the routing node

Due to the limited machine, only one shard shard is configured, and there are 2 nodes in the shard. You only need to change the replSet name for the new shard.

4. Configure replSet: connect to any mongod members

Mongo 192.168.201.226:11813

Config = {_ id: 'rep1', members: [

{_ id: 0, host: '192.168.200.226 priority 11813 priority, priority: 2}, # priority is defined as priority, default is 1. High priority is considered to be enabled first for the primary node.

{_ id: 1, host: '192.168.201.226 purl 11813'}

{_ id: 2, host: '192.168.200.226 purl 11814 id, arbiterOnly: true}]

}

Rs.initiate (config)

Rs.status ()

5. Connect mongos to add shard 80g

Mongo 192.168.201.226:11811/admin

Show dbs

Use admin

Db.runCommand ({addshard:'rep1/192.168.201.226:11813192.168.200.226:11813',maxsize:81920})

Db.runCommand ({listshards:1})

6. Connect to mongos to create test libraries and C1 collections, and test

Mongo 192.168.201.226:11811/admin

Db.runCommand ({enablesharding:'test'})

PrintShardingStatus ()

Db.runCommand ({shardcollection:'test.auto_increment_id', key: {_ id:1}, unique: true})

Db.runCommand ({shardcollection:'test.c1', key: {_ id:1}, unique: true})

For (var I = 1; I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.