Shulou(Shulou.com)06/01 Report--

overview

The mongoexport command line is used to export data. The default file format for export is JSON format. Of course, you can also specify a specific file format.

syntax

C:\mongo\bin>mongoexport -help

options:

--help produce help message

-v [ --verbose ] be more verbose (include multiple times for more

verbosity e.g. -vvvvv)

-h [ --host ] arg mongo host to connect to ( /s1,s2 for sets)

--port arg server port. Can also use --host hostname:port

--ipv6 enable IPv6 support (disabled by default)

-u [ --username ] arg username

-p [ --password ] arg password

--dbpath arg directly access mongod database files in the given

path, instead of connecting to a mongod server -

needs to lock the data directory, so cannot be used

if a mongod is currently accessing the same path

--directoryperdb if dbpath specified, each db is in a separate

directory

-d [ --db ] arg database to use

-c [ --collection ] arg collection to use (some commands)

-f [ --fields ] arg comma separated list of field names e.g. -f name,age

--fieldFile arg file with fields names - 1 per line

-q [ --query ] arg query filter, as a JSON string

--csv export to csv instead of json

-o [ --out ] arg output file; if not specified, stdout is used

--jsonArray output to a json array rather than one object per

Line

Description:

-h: IP of database host-u: Database user name-p: Database password-d: Database name-c: Collection name-f: Column name exported-q: Filter condition for exported data-csv: Export format is csv

introduction

Today, when using mongoexport to export data that meets certain conditions, I encountered an error. Now I record it and further learn the number type of MongoDB for this error.

Background and error messages

Today, I received a business request to export data that meets the following conditions from the MongoDB database order collection:

db.qqwj_order.find({"Source":NumberInt("21"),"Batch":"Order reminder in payment: September 5, 2018","MsgContent":/Online payment not yet completed/})

Query and check the MongoDB client tool [NoSQLBooster for MongoDB], and the statement executes normally, showing that the corresponding number of records is 15265.

Export data using the mongoexport command, execute the following command:

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{ "Source":NumberInt("21"),"Batch":"Order reminder in payment: 2018/9/5","MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/sms.csv

But the execution error:

XXX is not valid JSON: json: cannot unmarshal string into Go value of type json.NumberInt

The screenshot of the error is as follows:

Error inference and testing

Because the NumberInt keyword in the error message, at this time to see our query conditions also happen to have this keyword, so speculation is not this problem.

As a result, NumberInt("21") in the export command is directly replaced with 21 and executed again.

The executive order is:

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{"Source":21,"Batch":"Order reminder in payment: 2018/9/5,""MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/ms.csv.

The implementation results are

The results show that the modified data can be exported successfully.

Error Analysis and Principle Exploration

Why is it that when viewed through the interrogator, the data is "Source" : NumberInt("21"), but the execution export command in the shell is written as "Source" : NumberInt("21"), which will give an error? And be sure to convert to "Source":21

The Source field queried by the searcher displays:

Why copy to shell and execute error???

Look back, find the principle. We know that MongoDB currently supports 4 data types.

double32-bit integer64-bit integerDecimal （New in version 3.4.）

Query execution is possible in MongoDB client, but export is not possible in shell. Is this related to these two tools? Does it have to do with NumberInt ('number ') or NumberInt ('number') inserted?

The hypothesis is tested.

Insert test data via NoSQLBooster for MongoDB

Insert test data via shell

View the inserted data type by $type

1> execute db.numbers.find({n:{$type:1}}) // Type is Double; query data of Type Double

The above query results show that the inserted numeric data defaults to Double when the number does not indicate the data type, whether through the client or shell.

2》Execute db.numbers.find({n:{$type:16}}) // Type is 32-bit integer ; query data of Type 32-bit integer

The above query table names, whether through the client or shell, the specified NumberInt(5) or NumberInt ('5 ') are converted to a unified 32-bit integer type in the background.

3》Execute db.numbers.find({n:{$type:18}}) // Type is 64-bit integer Query Type is 64-bit integer data

The above query table name, whether through the client or shell, the specified NumberLong(5) or NumberLong ('5 ') is converted to a unified 64-bit integer type in the background.

The above test shows that when we store numerical data, it automatically dumps (regardless of client tool, shell or [NoSQLBooster for MongoDB], regardless of NumberLong(5) or NumberLong ('5 ');NumberInt(5) or NumberInt ('5')).

A little confused? So, why is the query wrong?

XXX is not valid JSON: json: cannot unmarshal string into Go value of type json.NumberInt.

This means that the shell thinks we passed data of a character type to json.NumberInt.

So if I replace NumberInt("21") in the export command with NumberInt(21)

The executive order is:

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{"Source": NumberInt(21),"Batch":"Order reminder in payment: 2018/9/5","MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/sms.csv

Implementation was also successful.

conclusion

A lot to summarize:

Export commands that failed to execute were:

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{ "Source":NumberInt("21"),"Batch":"Order reminder in payment: 2018/9/5","MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/sms.csv

Successful export commands are:

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{"Source":21,"Batch":"Order reminder in payment: 2018/9/5,""MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/ms.csv.

and

/data/mongodb/mongobin344/bin/mongoexport -h 172.X.X.XXX --port port--db database-u account number-p 'password' --authenticationDatabase authentication database--type=csv -c qqwj_order -f MsgContent,REC_CreateTime -q '{"Source": NumberInt(21),"Batch":"Order reminder in payment: 2018/9/5","MsgContent":/Online payment not yet completed/}' -o /data/mongodb_back/sms.csv

The differences between the three export commands are marked in red font.

P.S1: Later, the author delved into why the same query, the general query results, some show "n" : 5 ; some show "n" : NumberInt("5"). The version is different.

Display of older versions (partial)

Display of new versions (e.g. nosqlbooster4mongo-4.7.1)

P.S 2: When storing digital data, what data type will be stored is actually related to the driver of language. For example, in Ruby and Python, when serializing integers, the driver automatically determines whether to encode a 32-bit integer or a 64-bit integer; the shell needs to display the specification.

summary

The above is the whole content of this article, I hope the content of this article has a certain reference value for everyone's study or work, if you have questions, you can leave a message to exchange, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.