Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to store messages in the EMQ X rules engine to the MongoDB database. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

MongoDB introduction

Non-relational databases (NoSQL) are used to store very large-scale data, such as Google or Facebook, which collects terabytes of data for their users every day. These types of data stores do not require fixed schemas and can be scaled out without redundant operations.

MongoDB is a product between relational database and non-relational database, which is the most functional and most like relational database in non-relational database. MongoDB, written by C++ language, is an open source database system based on distributed file storage. MongoDB aims to provide scalable high-performance data storage solutions for data storage. Under high load, you can easily add more nodes to ensure service performance.

MongoDB stores the data as a document, and the data structure consists of key-value (key= > value) pairs. MongoDB documents are similar to JSON objects. Field values can contain other documents, arrays, and document arrays.

MongoDB download address: https://www.mongodb.com/download-center/community

Scene introduction

This scenario requires that messages that meet certain conditions under the topic specified by EMQ X be stored in the MongoDB database. In order to facilitate subsequent analysis and retrieval, the message content needs to be split and stored.

In this scenario, the information reported by the device end is as follows:

Report topic: cmd/state/:id, in which id represents the vehicle client identification number

Message body:

{"id": "NXP-058659730253-963945118132721-22", / / client identification code "speed": 32.12, / / vehicle speed "direction": 198.33212, / / driving direction "tachometer": 3211, / / engine speed When the value is greater than 8000, you need to store "dynamical": 8.93, / / instantaneous fuel consumption "location": {/ / GPS longitude and latitude data "lng": 116.296011, "lat": 40.005091}, "ts": 1563268202 / / reporting time}

When the reported data engine speed value is greater than 8000, the current information is stored for subsequent analysis of the user's vehicle usage.

Prepare for work to create administrative users

First log in to MongoDB with an account that has permission to create a user, and add a user for emqx_rule_engine_output:

> use emqx_rule_engine_output; > db.createUser ({user: "root", pwd: "public", roles: [{role: "readWrite", db: "emqx_rule_engine_output"}]}); create a data table

Log in with the new user and create the dataset use_statistics:

$mongo 127.0.0.1/emqx_rule_engine_output-uroot-ppublic > db.createCollection ("use_statistics")

Confirm the existence of the data table after successful creation:

Show collectionsuse_statistics configuration instructions for creating resources

Open EMQ X Dashboard, go to the resources page of the left menu, click the New button, and select the MongoDB resource type to create:

The network environment of the nodes in the EMQ X cluster may be different from each other. Click the status button in the list after resource creation to view the connection status of each node resource. If the resources on the node are not available, check whether the configuration and network connectivity are correct, and click the reconnect button to reconnect manually.

Create a rule

Go to the rules page of the left menu and click the New button to create the rules. Here, we choose to trigger the event message release, and trigger the rule for data processing when the message is released.

After the trigger event is selected, we can see the optional fields and sample SQL on the interface:

Filter the required fields

The rule engine uses SQL statements to process rule conditions. In this business, we need to select all the fields in payload separately, using payload. To select the format, you also need the topic, qos and id information of the message context. The current SQL is as follows:

SELECT payload.id as client_id, payload.speed as speed, payload.tachometer as tachometer, payload.ts as ts, idFROM "message.publish" WHERE topic = ~ 'tUniverse'to establish the screening conditions

Use the SQL statement WHERE sentence for conditional filtering. In this business, we need to define two conditions:

Only deal with cmd/state/:id topics and filter topic with the topic wildcard = ~: topic = ~ 'cmd/state/+'

Only messages with tachometer > 8000 are processed and tachometer is filtered using comparators: payload.tachometer > 8000

The SQL obtained from the previous step is as follows:

SELECT payload.id as client_id, payload.speed as speed, payload.tachometer as tachometer, payload.ts as ts, idFROM "message.publish" WHERE topic = ~ 'cmd/state/+' AND payload.tachometer > 8000 use the SQL test function for output testing

With the SQL testing function, we can view the current SQL processed data output in real time, which requires us to specify payload and other simulation raw data.

The payload data is as follows. Remember to change the size of the tachometer value to meet the SQL condition:

{"id": "NXP-058659730253-963945118132721-22", "speed": 32.12, "direction": 198.33212, "tachometer": 9001, "dynamical": 8.93, "location": {"lng": 116.296011, "lat": 40.005091}, "ts": 1563268202}

Click the SQL test switch button, change topic and payload to the information in the scenario, and click the test button to view the data output:

The test output data is:

{"client_id": "NXP-058659730253-963945118132721-22", "id": "589A429E9572FB44B0000057C0001", "speed": 32.12, "tachometer": 9001, "ts": 1563268202}

The test output is as expected, and we can take the next steps.

Add response actions to store messages to MongoDB

After the SQL conditional input and output is correct, we continue to add the response action, configure the write SQL statement, and store the filter results in MongoDB.

Click the add button in the response action, select Save data to the MongoDB action, select the resource just selected, we use ${fieldName} syntax to populate the operation statement, insert the data into the database, and finally click the New button to complete the rule creation.

Collection is configured as: use_statistics

Selector is configured to:

Msgid=$ {id}, client_id=$ {client_id}, speed=$ {speed}, tachometer=$ {tachometer}, ts=$ {ts}

Test expected results

We successfully created a rule that contains a processing action, and the desired effect of the action is as follows:

When the device reports a message to the cmd/state/:id topic, the SQL will be hit when the tachometer value in the message exceeds 8000, and the hit number in the rule list will be increased by 1.

A piece of data is added to the use_statistics table of the MongoDB emqx_rule_engine_output database, and the value is the same as the current message.

Test using the Websocket tool in Dashboard

Switch to the tool = > Websocket page, use any information client to connect to EMQ X, and send the following message to the message card after the connection is successful:

Topic: cmd/state/NXP-058659730253-963945118132721-22

Message body:

{"id": "NXP-058659730253-963945118132721-22", "speed": 32.12, "direction": 198.33212, "tachometer": 198.33212, "dynamical": 8.93, "location": {"lng": 116.296011, "lat": 40.005091}, "ts": 1563268202}

Click the send button, and the tachometer value in the message body meets the condition of tachometer > 8000 set above. The statistical value of the current rule hit is plus 1.

Look at the data table record in the MongoDB command line and get the following data:

So far, we have realized the business development of using the rule engine to store messages to the MongoDB database.

The above is how to store messages in the EMQ X rules engine shared by the editor. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.