Shulou(Shulou.com)06/01 Report--

I. Overview

1.1 MongoDB concept

MongoDB is written in C++ language and is an open source database system based on distributed file storage. In the case of high load, adding more nodes can ensure server performance. MongoDB aims to provide scalable, high-performance data storage solutions for WEB applications. MongoDB stores the data as a document, and the data structure consists of key-value (key= > value) pairs. MongoDB documents are similar to JSON objects. Field values can contain other documents, arrays, and document arrays. In nosql database, most queries are in the form of key-value pairs (key, value). MongoDB is a product between relational database and non-relational database, which is most similar to relational database among non-relational databases. It features NoSQL, document storage, Json data model, and supports transactions.

1.2 NoSQL concept

NoSQL refers to a non-relational database. NoSQL, sometimes referred to as the abbreviation of Not Only SQL, is a general term for database management systems that are different from traditional relational databases.

NoSQL is used to store very large-scale data. Google or Facebook, for example, collect terabytes of data for their users every day. These types of data stores do not require fixed schemas and can be scaled out without redundant operations.

1.3Features of NoSQL

High scalability: Nosql removes the relational features of relational databases and is easy to scale out.

High performance: Nosql acquires data in a simple key-value way, which is very fast. And NoSQL's Cache is record-level, a fine-grained Cache, so NoSQL has much higher performance at this level.

Flexible data model: compared with the complexity of adding and deleting fields in relational databases, NoSQL does not need to create fields for the data to be stored in advance, and can store custom data formats at any time.

High availability: NoSQL can easily implement a highly available architecture with little impact on performance. For example, mongodb can quickly configure a highly available configuration through mongos and config server replica set,shard.

1.4 NoSQL classification

The type part represents the characteristics

Column storage

Hbase

Cassandra

Hypertable

As the name implies, data is stored in columns. The biggest feature is that it is convenient to store structured and semi-structured data, and it is convenient to do data compression. It has great IO advantages for queries against a certain column or columns.

Document storage

MongoDB

CouchDB

Document storage is generally stored in a format similar to json, and the stored content is document-based. This gives you the opportunity to index some fields and implement some of the functions of a relational database.

Key-value storage

Tokyo Cabinet / Tyrant

Berkeley DB

MemcacheDB

Redis

Its value can be quickly queried through key. Generally speaking, storage is accepted according to order, regardless of value format. (Redis includes other features)

Graph storage

Neo4J

FlockDB

The best storage of graphic relationships. If the traditional relational database is used to solve the problem, the performance is low, and the design and use is not convenient.

Object storage

Db4o

Versant

Manipulate the database through a syntax similar to that of an object-oriented language and access data in the way of objects.

Xml database

Berkeley DB XML

BaseX

Store XML data efficiently and support XML's internal query syntax, such as XQuery,Xpath.

1.5Conceptual Analysis of MongoDB

The basic concepts in mongodb are documents, collections, and databases, corresponding to relational databases as shown in the following table:

SQL terminology / concept MongoDB terminology / concept explanation / description databasedatabase database tablecollection database table / collection rowdocument data record row / document columnfield data field / domain indexindex index table joins

Table connection. MongoDB does not support primary keyprimary key primary key. MongoDB automatically sets the _ id field as the primary key.

Database: a single MongoDB instance can host multiple databases. They can be regarded as independent of each other, and each database has independent permission control. On disk, different databases are stored in different files. The following system databases exist in MongoDB.

Admin database: a permissions database that automatically inherits permissions from all databases if the user is added to the admin database when the user is created.

Local database: this database is never responsible and can be used to store any collection of local individual servers.

Config database: when MongoDB uses sharding mode, the config database is used internally to hold sharding information.

Collection: a collection is a set of documents, similar to tables in a relational database. The collection is schemaless, and the documents in the collection can be varied. Collections in MongoDB are represented by collections, each

Collection is identified by a name. You need to pay attention to the following points:

The name cannot be an empty string ""

The name cannot contain\ 0 characters because it represents the end of the name

Cannot be created with system. At the beginning

Document: a document is the basic unit of data in MongoDB, similar to rows in a relational database (but more complex than rows). Multiple keys and their associated values are placed together in an orderly manner to form a document.

The key / value pairs in the document are ordered.

The values in the document can be not only strings enclosed in double quotes, but also several other data types (or even the entire embedded document).

MongoDB is type-and case-sensitive.

MongoDB documents cannot have duplicate keys.

The key of the document is a string. With a few exceptions, keys can use any UTF-8 character.

II. Installation and deployment

2.1Configuring yum installation

Configure the yum source of mongodb for installation

Cat > / etc/yum.repos.d/mongodb.repo/usr/local/mongodb/conf/mongodb.conf show dbs; # View database admin 0.000GBdbtest 0.000GBlocal 0.000GB

b. Delete database

The syntax format for MongoDB to delete a database is as follows:

Db.dropDatabase () > use dbtest;switched to db dbtest > db.dropDatabase () {"dropped": "dbtest", "ok": 1} > show dbs;admin 0.000GBlocal 0.000GB

3.2 Collection operation

a. Create a collection

Db.createCollection (name,options) db.createCollection ("user", autoindexID=true)

Capped: type is Boolean. If it is true, create a fixed-size collection that can automatically overwrite previous entries when they reach their maximum. Also specify the parameter size when setting it to true

AutoIndexID: type is Boolean, default is false. If set to true, the index is automatically created on _ id field.s

Size: if capped is true, specify the maximum value of the parameter (in byte)

Max: specifies the maximum number of documents. You don't have to create a collection in Mongodb, because the collection is also created automatically when you create a document.

> db.createCollection ("user", autoindexID=true) # create collection user {"ok": 1} > show collections; # View collection user

b. Delete a collection

Db.collection.drop () > show collections;user > db.user.drop () true > show collections

3.3 document operation

a. Insert document

MongoDB uses the insert () or save () method to insert a document into the collection

Db.COLLECTION_NAME.insert (document) > db.user.insert ({name: "kaliarch", age:30}) # insert insert document WriteResult ({"nInserted": 1}) > db.user.insert ({name: "xuel", age:31}) WriteResult ({"nInserted": 1}) > db.user.save ({name: "anchnet") Age:34}) # save insert document WriteResult ({"nInserted": 1}) > db.user.find () # View document {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "xuel" "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34}

Db.collection.insertOne (): inserts a piece of document data into the specified collection

Db.collection.insertMany (): inserts multiple pieces of document data into the specified collection

b. Update document

MongoDB uses the update () and save () methods to update documents in the collection

-- update--

Update () method db.collection.update (, {upsert:, multi:, writeConcern:})

Parameter description:

Query: the query condition of update, similar to that after where in a sql update query.

Update: the object of update and some updated operators (such as $, $inc...), etc., can also be understood as after set in the sql update query

Upsert: optional, this parameter means that if there is no record of update, whether to insert objNew,true is insert, default is false, do not insert.

Multi: optional. Mongodb defaults to false. Only the first record found is updated. If this parameter is true, multiple records are checked out according to the condition and all of them are updated.

WriteConcern: optional, the level at which the exception is thrown.

> db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "xuel", "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} > db.user.update ({"name": "xuel"}) {$set: {"name": "udxuel"}) # Update nameWriteResult ({"nMatched": 1, "nUpserted": 0, "nModified": 1}) > db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "udxuel" "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34}

-- save--

Save () mode db.collection.save (, {writeConcern:})

Parameter description:

Document: document data.

WriteConcern: optional, the level at which the exception is thrown.

> db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "udxuel", "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} > db.user.save ({"_ id": ObjectId ("5a1a870cceffdd1c65f633ef") "name": "xuel", "age": 31}) # Update nameWriteResult ({"nMatched": 1, "nUpserted": 0, "nModified": 1}) > db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "xuel" "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34}

c. Delete document

The basic syntax format of the remove () method is as follows:

Db.collection.remove (, {justOne:, writeConcern:})

Parameter description:

Query: (optional) conditions for deleted documents.

JustOne: (optional) if set to true or 1, only one document is deleted.

WriteConcern: (optional) the level at which the exception is thrown.

> db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a870cceffdd1c65f633ef"), "name": "xuel", "age": 31} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} > db.user.remove ({"name": "xuel") "age": 31}) # Delete the document WriteResult ({"nRemoved": 1}) > db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34}

d. View document

Db.collection.find (query, projection)

Query: optional, use the query operator to specify query conditions

Projection: optionally, use the projection operator to specify the returned key. When querying, all the key values in the document are returned, simply omitting the parameter (default omitted).

You can use the pretty () method, which has the following syntax format:

> db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} > db.user.find (). Pretty () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch" "age": 30} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34}

3.4 conditional operator

The conditional operator is used to compare two expressions and get data from the mongoDB collection

(>) greater than-$gt

(=) greater than or equal to-$gte

(db.user.find () {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} {"_ id": ObjectId ("5a1a8ddeceffdd1c65f633f1"), "name": "test" "age": 50} > db.user.find ({"age": {$gt:33}}) # query documents with age greater than 33 {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet", "age": 34} {"_ id": ObjectId ("5a1a8ddeceffdd1c65f633f1"), "name": "test" "age": 50} > db.user.find ({"age": {$lt:33}}) # query documents with age less than 33 {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch" "age": 30} > db.user.find ({"age": {$eq:34}}) # query documents with age equal to 33 {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet" "age": 34} > db.user.find ({"age": {$ne:10}}), # query documents whose age is not equal to 10 {"_ id": ObjectId ("5a1a8704ceffdd1c65f633ee"), "name": "kaliarch", "age": 30} {"_ id": ObjectId ("5a1a8729ceffdd1c65f633f0"), "name": "anchnet" "age": 34} {"_ id": ObjectId ("5a1a8ddeceffdd1c65f633f1"), "name": "test", "age": 50

3.5 create an index

Indexes can usually greatly improve the efficiency of queries. If there is no index, MongoDB must scan each file in the collection and select those records that meet the query criteria when reading data.

The query efficiency of this kind of scanning full collection is very low, especially when dealing with a large amount of data, the query can take dozens of seconds or even minutes, which is very fatal to the performance of the website.

An index is a special data structure, which is stored in a data collection that is easy to traverse and read. An index is a structure that sorts the values of one or more columns in a database table.

MongoDB uses the ensureIndex () method to create the index.

> db.COLLECTION_NAME.ensureIndex ({KEY:1})

In the syntax, the Key value is the index field you want to create, 1 is the specified index in ascending order, and if you want to create the index in descending order, specify-1.

In the ensureIndex () method, you can also set to create indexes using multiple fields (called composite indexes in relational databases).

Db.mycol.ensureIndex ({"title": 1, "description":-1})

3.6 backup recovery

A.mongodb data backup

The mongodump command script syntax is as follows:

Mongodump-h dbhost-d dbname-o dbdirectory

-h:

The address of the server where the MongDB is located, for example: 127.0.0.1. Of course, you can also specify the port number: 127.0.0.1 27017.

-d:

Database instance that needs to be backed up, for example: test

-o:

The location of the backup data, for example: C:\ data\ dump. Of course, this directory needs to be established in advance. After the backup is completed, the system automatically creates a test directory under the dump directory, in which the backup data of the database instance is stored.

Syntax description instance mongodump-- host HOST_NAME-- port PORT_NUMBER this command will backup all MongoDB data mongodump-- host localhost-- port 27017mongodump-- dbpath DB_PATH-- out BACKUP_DIRECTORY

Mongodump-- dbpath / data/db/-- out / data/backup/mongodump-- collection COLLECTION-- db DB_NAME this command backs up the specified collection of databases. Mongodump-collection mycol-db test

B.mongodb data recovery

The mongorestore command script syntax is as follows:

> mongorestore-h-d dbname

-host,-h:

Server address where MongoDB resides. Default is: localhost:27017

-db,-d:

The database instance that needs to be restored, such as test. Of course, this name can be different from that of backup, such as test2.

-- drop:

When restoring, delete the current data first, and then restore the backed-up data. That is to say, after recovery, the data added and modified after backup will be deleted, so use it with caution.

:

The last parameter of mongorestore, which sets the location of backup data, for example: / tmp/testdb

You cannot specify both the and-- dir options, and-- dir can also set the backup directory.

-- dir:

Specify the directory of the backup

You cannot specify both the and-- dir options.

At this point, the viewing data has been restored normally.

3.7 password permissions

To set a password for mongodb, you need to start auth and add auth=true in the configuration file.

a. Set up a database administrator

> use adminswitched to db admin > db.createUser ({user: "root", pwd: "mongopwd", roles: ["userAdminAnyDatabase"]}) # create a user password and set permissions Successfully added user: {"user": "root", "roles": ["userAdminAnyDatabase"]} > db.auth ("root", "mongopwd") # Authentication. If 1 is returned, authentication is successful.

Authentication can also log in by directly specifying the user name and password when logging in by entering the database db.auth

Mongo-u root-p mongopwd-- authenticationDatabase admin

b. Create a database user

> use testswitched to dbtest > dbtest > db.createUser ({user: "testuser", pwd: "testpass", roles: ["readWrite"]}) Successfully added user: {"user": "testuser", "roles": ["readWrite"]} > db.auth ("testuser", "testpass") 1

IV. Automatic backup script

Finally, write a mongodb automatic backup script, cooperate with crond to back up data regularly.

Github address

#! / bin/bash#auth:xuel@anchnet.com#backup mongodb. / etc/profileCMD= `which mongodump`DATE = `date +% F`DB = "test" DB_HOST= "localhost" DB_PORT= "27017" DB_USER= "testuser" testpass "BACKUP_DIR=" / data/mongodb/ "TAR_DIR=" / data/tardir/ "TAR_DIR_DATE=" ${TAR_DIR} ${DATE} "TAR_BAK=" mongodb_bak_$DATE.tar.gz "check_dir () {for DIR in ${BACKUP_DIR} ${TAR_DIR_DATE} do If [!-d $DIR] Then mkdir-p $DIR fi done} backup_mongodb () {$CMD-h ${DB_HOST}: ${DB_PORT}-u $DB_USER-p $DB_PWD-d ${DB}-o ${BACKUP_DIR} ${DATE} > / dev/null if [$?-eq 0] Then tar-zcf ${TAR_DIR_DATE} / ${TAR_BAK} ${BACKUP_DIR} ${DATE} & & return 0 fi} clean_tar () {find ${TAR_DIR}-mtime + 7-exec rm-rf {}\ & & return 0} main () {check_dir [$?-eq 0] & & backup_mongodb [$?-eq 0] & & clean_tar} main

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.