Shulou(Shulou.com)06/01 Report--

Today, I would like to introduce a software to analyze the structure of MongoDB database tables-- Varity. For the database of Schema Free such as MongoDB, it is difficult to see the specific data structure at a glance by querying the data stored in collection with the software. It is easy to understand the data structure without collection when the author wrote a script of Variety.js. The author hosts the tools on github and welcomes anyone to give advice or add functionality. The following Variety features are translated from the author's blog:

The output format of collection information is ASCII.

You can clearly see what type of data format each key uses

You can see how much key is used in this collection.

You can limit the number of queries in documents

You can limit the depth of the query documents

You can analyze only a subset of documents

You can sort query results

Query results can be saved

Can be exported in JSON format

The tool introduction is easy to use and does not use any other library dependencies

The download address of Variety is https://github.com/variety/variety.

How to use it:

Mongo DATABASE_NAME-- eval "var collection = 'COLL_NAME'" variety.js, for example, my DATABASE_NAME is test and COLL_NAME is users

The data I inserted in advance is

Db.users.insert ({name: "Tom", bio: "A nice guy.", pets: ["monkey", "fish"], someWeirdLegacyKey: "I like Ike!"}); db.users.insert ({name: "Dick", bio: "I swordfight.", birthday: new Date ("1974-03-14")}); db.users.insert ({name: "Harry", pets: "egret", birthday: new Date ("1984-03-14")}) Db.users.insert ({name: "Shanker", bio: "a va?"})

The echo of a normal query Users looks like this.

> db.users.find () {"_ id": ObjectId ("56cfc28fbdae9b9a922a19cb"), "name": "Tom", "bio": "A nice guy", "pets": ["monkey", "fish"] "someWeirdLegacyKey": "I like ike!"} {"_ id": ObjectId ("56cfc2acbdae9b9a922a19cc"), "name": "Dick", "bio": "I swordfight."} {"_ id": ObjectId ("56cfc2c6bdae9b9a922a19cd"), "name": "Harry" "pets": "egret"} {"_ id": ObjectId ("56cfc2e0bdae9b9a922a19ce"), "name": "Shanker", "bio": "caca"}

The result of the query with Variety looks like this

Mongo test-- eval "var collection = 'users'" variety.jsMongoDB shell version: 2.4.9connecting to: testVariety: A MongoDB Schema AnalyzerVersion 1.5.0 Released 14 May 2015Using collection of "users" Using query of {} Using limit of 4Using maxDepth of 99Using sort of {"_ id":-1} Using outputFormat of "ascii" Using persistResults of falseUsing resultsDatabase of "varietyResults" Using resultsCollection of "usersKeys" Using resultsUser of nullUsing resultsPass of nullUsing plugins of [] + -+ | key | types | occurrences | percents | |-| | _ id | ObjectId | 4 | 100.0 | | name | String | 4 | 100.0 | | bio | String | 3 | 75.0 | | pets | String (1) Array (1) | 2 | 50.0 | | someWeirdLegacyKey | String | 1 | 25.0 | +-+

Is the format friendly and easy to read?

If the database does not use the default port, you can use the-- port parameter:

Mongo DATABASE_NAME-port 27111-eval "var collection = 'COLL_NAME'" variety.js

If the db file is not in the default file, you can use the-- dbpath parameter:

Mongo DATABASE_NAME-dbpath / path/to/database/folder-eval "var collection = 'COLL_NAME'" variety.js

If you need to sort the query, you can use this:

Mongo DATABASE_NAME-- eval "var collection = 'COLL_NAME', sort = {date:-1}" variety.js

If you need the output format of JSON, you can use it like this:

Mongo DATABASE_NAME-- eval "var collection = 'users', outputFormat =' json'" variety.js

If a collection has 1 billion data, we can limit the number of queries and use limit to limit:

Mongo DATABASE_NAME-- eval "var collection = 'users', limit = 1000" variety.js

If a colletions has too many levels of nesting, you can use maxDepth to restrict the query:

Db.users.insert ({name: "Walter", someNestedObject: {a: {b: {c: {d: {eWalter 1}) [ibmcloud@bravo:~/variety04:05] $mongo test-- eval "var collection = 'users'" variety.jsMongoDB shell version: 2.4.9connecting to: testVariety: A MongoDB Schema AnalyzerVersion 1.5.0 Released 14 May 2015Using collection of "users" Using query of {} Using limit of 5Using maxDepth of 99Using sort of {"_ id":-1} Using outputFormat of "ascii" Using persistResults of falseUsing resultsDatabase of "varietyResults" Using resultsCollection of "usersKeys" Using resultsUser of nullUsing resultsPass of nullUsing plugins of [] + -- + | key | types | occurrences | percents | |-|- -| | _ id | ObjectId | 5 | 100.0 | | name | String | 5 | 100.0 | | bio | String | 3 | 60.0 | | pets | String (1) Array (1) | 2 | 40.0 | | someNestedObject | Object | 1 | 20.0 | | someNestedObject.a | Object | 1 | 20.0 | | someNestedObject.a.b | Object | 1 | 20.0 | | someNestedObject.a.b.c | Object | | 1 | 20.0 | someNestedObject.a.b.c.d | Object | 1 | 20.0 | | someNestedObject.a.b.c.d.e | Number | 1 | 20.0 | | someWeirdLegacyKey | String | 1 | 20.0 | +-| -+ [ibmcloud@bravo:~/variety05:06] $mongo test-- eval "var collection = 'users' MaxDepth = 3 "variety.jsMongoDB shell version: 2.4.9connecting to: testVariety: A MongoDB Schema AnalyzerVersion 1.5.0 Released 14 May 2015Using collection of "users" Using query of {} Using limit of 5Using maxDepth of 3Using sort of {"_ id":-1} Using outputFormat of "ascii" Using persistResults of falseUsing resultsDatabase of "varietyResults" Using resultsCollection of "usersKeys" Using resultsUser of nullUsing resultsPass of nullUsing plugins of [] + -+ | key | types | occurrences | percents | |-| | _ id | ObjectId | | 5 | 100.0 | | name | String | 5 | 100.0 | | bio | String | 3 | 100.0 | | pets | String (1) | Array (1) | 2 | 40.0 | | someNestedObject | Object | 1 | 20.0 | | someNestedObject.a | Object | 1 | 20.0 | | someNestedObject.a.b | Object | 1 | 20.0 | | someWeirdLegacyKey | String | 1 | 20.0 | +-- +

If you need a conditional query, such as one with carddAbout of true, you can do this:

Mongo DATABASE_NAME-- eval "var collection = 'COLL_NAME', query = {' caredAbout':true}" variety.js

It should be noted that when Variety analyzes the data structure, it is actually done with MapReduce and performs full table scanning, so if you are analyzing the online database, it is recommended to use a backup library that does not provide services or at the low peak of the business. Avoid putting pressure on online business.

Reference address:

Http://www.acetolyne.net/Projects7/node/48

Https://github.com/variety/variety

Http://www.mongoing.com/archives/2282

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.