Shulou(Shulou.com)05/31 Report--

Python data processing MongoDB operation methods, I believe that many inexperienced people do not know what to do, so this paper summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

1. Preface

MongoDB is an open source NoSql database written by C++ based on distributed storage.

The content store of MongoDB is similar to JSON objects, and there are three kinds of data structures.

They are:

Database-Databases

Database in corresponding relational database (Database)

Collection-Collection

Corresponding Table table (Table) in relational database

Documentation-Document

Corresponding to a piece of data in the database table (Row Data)

two。 Prepare for

Python operates MongoDB in two common ways: Pymongo and Mongoengine

Among them

Mongoengine: face object. For the ORM of a document-based database, it inherits directly from the Document document and adds, deletes, modifies and queries the document.

Pymongo: communicate with MongoDB through JSON to map MongoDB data to Python built-in data types

First, we install the dependency through the pip3 command

# installation dependency # pymongo pip3 install pymongo # mongoengine pip3 install mongoengine

Pymongo and Mongoengine are described below.

3. PyMongo

First, create a database connection object

There are two ways to create database connection objects: multi-parameter and string concatenation.

Import pymongo # two ways to create database connection objects # method 1: multi-parameter self.client = pymongo.MongoClient (host='ip address', port=27017, username= "root", password= "123456", authMechanism= "SCRAM-SHA-1") # method 2: splice # self.client = pymongo.MongoClient ('mongodb://root:123456@ip address: 27017Gap')

Next, specify the database and operation set to operate through the database connection object

For example, to manipulate the students collection in the database temp

# specify the database to operate on: temp self.db = self.client.temp # specify the collection to operate students self.collection_students = self.db.students

Next, let's implement the operation of adding, deleting, changing and searching.

1. Add

Added include: add single data and multiple data

The corresponding method for inserting a single piece of data is:

Insert_one (dict)

The return type of this method is InsertOneResult

Through the inserted_id property, you can get the _ id value of the inserted data

Temp_data = {"id": "1", "name": "xag", "age": 18} # 1. Directly call the collection's insert_one () method to insert data (insert a piece of data) result = self.collection_students.insert_one (temp_data) # the return value is InsertOneResult, and the value print (result.inserted_id) of _ id is obtained through the inserted_id attribute.

The corresponding method for inserting multiple pieces of data is:

Insert_many ([dict1,dict2...])

The return type of this method is InsertManyResult

Through the inserted_ids property, you can get a list of values for the _ id attribute of the inserted data

# 2. Insert multiple pieces of data-insert_many () result = self.collection_students.insert_many ([temp_data, temp_data2]) # the returned value is InsertManyResult, and the _ id list value print (result.inserted_ids) of the inserted data is obtained through the inserted_ids attribute

2. Query

Using PyMongo to query the MongoDB database, the common methods are as follows:

Query a record-find_one () through a property key-value pair

Query a record by ObjectId value-find_one ()

Query multiple records through a property key-value pair-find ()

Compare queries by conditions such as greater than, less than, or equal to

Regular matching query

Because the first three query methods are relatively simple, examples are given directly:

Def manage_query (self): "" query data "" # 1, through an attribute key-value pair To query a record find_one () # the returned value is dictionary type # {'_ id': ObjectId ('5f5c437cfe49fa9a16664179'),' id': '1century,' name': 'xag' 'age': 18} result = self.collection_students.find_one ({"name": "xag"}) print (result) # 2. Query a record by Objectid value result = self.collection_students.find_one ({"_ id": ObjectId (' 5f5c437cfe49fa9a16664179')}) print (result) # 3.1 query multiple records find () # returns a cursor (generator) Pymongo.cursor.Cursor result_lists = self.collection_students.find ({"name": "xag"}) print (result_lists) for item in result_lists: print (item)

Conditional comparison query, including: greater than ($gt), greater than or equal to ($gte), less than ($lt), less than or equal to ($lte), not equal to ($ne), in range ($in), out of range ($nin)

For example: query data older than 18 years old

# 3.2 conditional comparison query, including greater than ($gt), greater than or equal to ($gte), less than ($lt), less than or equal to ($lte), not equal to ($ne), in range ($in), out of range ($nin) # query records older than 18 years old result = self.collection_students.find ({'age': {' $gt': 18}}) for item in result: print (item)

Regular matching query, which contains:

$regex: matches regular expressions

$exists: whether the attribute exists

$type: data type judgment

$mod: data model operation

$text: text contains query

$where: advanced conditional query

For example, query data whose name value begins with "xag"

# regular matching query results = self.collection_students.find ({'name': {' $regex':'^ xag.*'}}) for item in results: print (item)

For more complex features on queries, please refer to:

Https://docs.mongodb.com/manual/reference/operator/query/

3. Update

Update operations include updating one record and updating multiple records

The method to update a record is:

Update_one (query,update_content)

The parameters include: the condition of the query, the content to be modified

# 1. Modify a record update_one (query,update_data) # method has two parameters They are: query condition, content to be modified # query condition query_condition = {"name": "xag"} # content to be modified update_content = {"$set": {"name": "Xinganguo"}} # use the update_one () method to update a record result = self.collection_students.update_one (query_condition, update_content)

The number of records matched by the query and the number of records affected can be obtained from the returned results.

# matched_count: number of matching records # modified_count: number of records affected print (result.matched_count, result.modified_count)

The method to update multiple records is:

Update_many (query,update_content)

The parameters and return values in the method are similar to modifying a single record

# 2. Modify multiple records update_many (query,update_data) # query condition query_condition = {"name": {"$regex": "^ star. *"} # content to be modified update_content = {"$set": {"name": "xag"}} # set all records in the document that begin with a star to xag result = self.collection_students.update_many (query_condition, update_content) print (result) print (result.matched_count) Result.modified_count)

4. Delete

Deleting also includes: deleting the first record queried and deleting all records queried.

The corresponding methods are: delete_one (query), delete_many (query)

In addition, the actual number of deletions can be obtained in the returned result.

Def manage_remove (self): delete operation: return: "# 1. Delete the first record queried delete_one () # result = self.collection_students.delete_one ({'name':" xag2 "}) # print (result) # number of deletions # print (result.deleted_count) # 2, Delete multiple records delete_many () result = self.collection_students.delete_many ({'name': "xag"}) print (result) # number of deletions print (result.deleted_count)

5. Count and rank

Common methods include:

Limit (num): limit the number of results returned

Skip (num): ignore num elements and start with num + 1 elements

Count_documents (): view the number of documents in the collection, and you can also query the number of documents satisfied according to the criteria

Sort (): ascending or descending

Def manage_count_and_sort (self): "" count and sort: return: "" # 1. Limit the number of results returned-limit () # result = self.collection_students.find (). Limit (2) # for item in result: # print (item) # 2, offset skip () # for example: ignore the first two elements Start with the third element # result = self.collection_students.find (). Skip (2) # print ([result ['name'] for result in result]) # 3.1 query the number of documents in the collection count_documents () # result = self.collection_students.count_documents ({}) # print (result) # 3.2 query according to conditions Then judge the number of results # query_regex = {'name': {' $regex':'^ xag.*'}} # result = self.collection_students.count_documents (query_regex) # print (result) # 4, sort sort () # pymongo.ASCENDING: ascending, DESCENDING: descending result = self.collection_students.find (). Sort ('name', pymongo.DESCENDING) print ([result [' name'] for result in result])

4. Mongoengine

Before using Mongoengine to operate MongoDB, you need to define a subclass of Document

This subclass corresponds to the document in MongoDB, and the static variables added internally (including type, length, etc.) correspond to the data in the database document.

A subclass of from mongoengine import * # Document, corresponding to the document object class Student (Document): name = StringField (required=True, max_length=500) age = IntField (required=True, default=18) create_time = DateTimeField (default=datetime.now) # configuration metadata # specifies the collection as student meta = {'collection':' student', 'strict': False}

Use the connect () method built into Mongoengine to connect to the specified database

# connect to database temp def _ _ init__ (self): # connect to database # Database name: temp # auth mode: SCRAM-SHA-1 result = connect ('temp', host='ip address', port=27017, username='root', password='123456', authentication_source='admin', authentication_mechanism= "SCRAM-SHA-1") print (result)

Next, let's implement the operation of adding, deleting, changing and searching.

1. Add

It is very convenient to add a record to the database using Mongoengine

You only need to instantiate a document object and call the save () method to store a record in the database.

Def insert (self): "" insert data: return: "" person = Student (name='xag2', age=20) person.save ()

2. Query

Common query operations include:

Query all records in the collection

Query the first record

Query the data through the primary key _ ID

Conditional query

The corresponding code is as follows:

Def query (self): "ordinary query: return:" # 1. View all the data in the collection # students = Student.objects.all () # print ([item ['name'] for item in students]) # 2. Query the first record # student = Student.objects.first () # print (student.name, student.age) Student.create_time) # 3. Query data result = Student.objects.filter (pk= "5f5c5b34f5b0c049707a1710") through the primary key _ ID. First () print (result.name, result.age, result.create_time) # 4, conditional query # query data aged between 18 and 20 # _ _ gte: greater than or equal to _ _ lte: less than or equal to # is ascending by default, and you can add: -, which represents the reverse order # students = Student.objects (age__gte=18, age__lte=20). Order_by ('name') students = Student.objects (age__gte=18, age__lte=20). Order_by ('-name') # for item in students: # print (item.name, item.age, item.create_time)

It is worth mentioning that Mongoengine provides the keyword Q to implement advanced queries

For example, query data whose name field value is xag and the age is 18 years old.

Def query_advance (self): "Advanced query: return:" # View the first entry student = Student.objects (Q (name= "xag") & Q (age=18)) of the record whose name is xag,age 18. First () print (student.name, student.age, student.create_time)

For more advanced operations, please refer to:

Https://docs.mongoengine.org/guide/querying.html

3. Update

Mongoengine provides filter () and update () methods to filter the data to be updated and the specified update content, respectively.

Def update (self): "Update record: return:"# 1, Modify all records # change the age of name to xag by 1 year # increase by one year: inc__age=1 # decrease by one year: dec__age=1 # Student.objects.filter (name= "xag"). Update (dec__age=1) # Student.objects.filter (name= "xag"). Update (inc__age=1) # name is xag Update all records with age less than 18 to age=23 # _ _ lt: less than # _ _ lte: less than or equal to # _ _ gt: greater than # _ _ gte: greater than or equal to # Student.objects.filter (name= "xag", age__lt=18) .update (age=23) # Student.objects.filter (age__lte=20) .update (age=23)

If you only need to change the first record queried, you can use the update_one () method

# 2. Modify a record # reduce the age by 5 years Student.objects.filter (name='xag'). Update_one (dec__age=5)

4. Delete

Delete operation corresponds to delete () method

Similarly, you can use the filter () and first () methods to limit the scope of deletion

Def delete (self): "" delete data: return: "# 1. Delete the first record # Student.objects.filter (name=" xag "). First () .delete () # 2, Delete multiple records # Student.objects.filter (name= "xag"). Delete () # Delete all records whose name value begins with xag Student.objects.filter (name__startswith= "xag") .delete ()

After reading the above, have you mastered the operation method of Python data processing MongoDB? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.