Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the design process based on MQTT message bus". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let the editor take you to learn "what is the design process based on MQTT message bus"?

I. the role of the gateway

The scope of the term Internet of things is too broad, it seems that all hardware devices, as long as they can access the network, can be called Internet of things products, it seems that the word Internet of things can include everything.

Such empty words are not conducive to our explanation, so we replace it with a scene that can be perceived and imagined, that is, the smart home system, which is a typical product that best represents the era of the Internet of things.

Instruction forwarding

In a smart home system, suppose there are several devices:

The communication module of these devices, if it is WiFi or Bluetooth, can generally be controlled directly through the mobile phone (of course, the manufacturer needs to provide the corresponding mobile phone APP), the mobile phone is equivalent to a central node, controlling all the devices.

At present, some intelligent devices on the market are in this way of communication, such as air conditioners, vacuum cleaners, air purifiers, refrigerators and so on. Just add a wireless communication module to these devices (for example, ESP8266 module).

If the communication module is other communication modules, such as RF433, ZigBee, ZWave, etc., because the phone does not have these communication modules, it needs a gateway to "forward" instructions.

Both the mobile phone and the gateway are connected to the router at home and are in the same local area network. The mobile phone sends control instructions to the gateway, and the gateway forwards the instructions to the corresponding devices. The communication model is as follows:

External network communication

In the above communication model, the mobile phone and the gateway can communicate directly because they are in the same local area network. What if the phone is not in the local area network? Then it needs to be forwarded through the cloud server. The communication model is as follows:

The phone sends instructions to the server.

The server forwards the instruction to the gateway

The gateway sends instructions to the specified device.

What is described above is the flow of the control instruction. if it is the alarm message sent by the device, the flow of the data is reversed.

It can be seen that the gateway is not only the central node of communication between all devices, but also the transit node between the internal network and the external network, that is, the repeater that connects all kinds of intelligent devices to the Internet.

Protocol conversion

As mentioned above, the communication modules on hardware devices are determined (RF,ZigBee,ZWave, etc.). Generally speaking, these communication modules can be called wireless communication protocols. In a smart home system, most of the wireless communication protocols of all devices are the same.

So, can different types of wireless communication protocol devices co-exist in the same system?

The answer is: yes. As long as the gateway, integrated with the corresponding wireless communication protocol module can achieve this goal! As shown in the following figure:

From the point of view of the mobile phone APP, all devices are the same and do not care what the wireless communication protocol of the device is. Therefore, the control instructions issued are protocol independent.

When the gateway receives the control instruction, it first finds out the target device according to the instruction content, then determines the wireless communication protocol of the target device, and finally sends the instruction to the corresponding hardware communication module. The communication module sends the control instruction to the device through the radio signal.

From the transmission process of this instruction, the gateway acts as the role of protocol conversion.

There is also a communication scenario: when an "input" device in the system is bound / associated with an "output" device, for example:

The infrared sensor is bound to the acousto-optic alarm: when the infrared sensor detects the human body, it sends out a signal, and then controls the acousto-optic alarm to issue an alarm.

Door magnetism and lamp binding: when opening the door, the door magnet sends out a signal and automatically turns on the light.

If the "input" device and the "output" device are different types of wireless communication protocols, gateways are also required for protocol conversion.

Equipment management

In a smart home system, there can be more or less devices, and it is also very important to manage these devices. As the central node of the system, the gateway naturally undertakes the important task of managing the equipment.

Device management features include:

Addition and deletion of equipment

Management of equipment status (power consumption, equipment disconnection, loss of connection, etc.); management of device tree

Edge calculation (automatic control)

Under normal circumstances, the gateway can maintain a long connection with the server through the router. If the processing power of the server is relatively strong, all the things that need to be handled in the smart home system can be left to the server to calculate and deal with, and the server will send the processing results to the gateway after calculation. Looks like the idea is perfect!

However, consider the following two situations:

There is a problem with the router, and the gateway cannot connect to the server, so the local data cannot be reported in time.

There is an abnormal situation in the system, which needs to be dealt with urgently. If the information is reported to the server, calculated by the server and then transmitted back to the gateway, the time spent may exceed the tolerable time, how to deal with it? (you can use the car networking system to supplement this scenario: if a self-driving car encounters an emergency, if you upload all the information to the server and wait for the server's next instructions?

For the above scenarios, it may be more appropriate to put some calculations and processing operations on the gateway side! This is also a popular edge computing in recent years.

1. Edge computing refers to the side close to the object or data source, using the network, computing, storage, application core capabilities as one of the open platform to provide the nearest end services.

two。 Its application is launched on the edge side, which produces faster network service response and meets the basic needs of the industry in real-time business, application intelligence, security and privacy protection.

3. Edge computing is between the physical entity and the industrial connection, or at the top of the physical entity. On the other hand, cloud computing can still access the historical data of edge computing.

Communication between processes within the gateway

When designing the architecture of an application, it can be realized through multi-thread or multi-process. Everyone's habits are different and each has its own advantages. We are not going to discuss the advantages and disadvantages here, because I prefer the design idea of multi-process, so I will discuss it directly according to the multi-process program architecture.

1 what processes are needed in the gateway

All processes that need to be executed in the gateway are determined according to the functions of the gateway. It is assumed that the following functions are included:

(1) Proc_Bridge of the process connecting to the public network

The gateway needs to connect to the server in the cloud, and a process needs to maintain a long connection with the server, so that it can receive the control instructions sent by the server in time and report the internal data of the system to the server in time.

This process needs to forward the instructions received from the server to the gateway system and the information received from the system to the server, which is similar to the bridging function, so it is named Proc_Bridge.

(2) device management process Proc_DevMgr

This process is used to perform device management functions, and the addition (entry) and deletion (withdrawal) of devices are managed by this process.

(3) Protocol conversion process Proc_Protocol

Downlink: convert the unified communication protocols of the application layer into different types of wireless communication protocols and send them to the corresponding wireless modules.

Uplink: convert different types of wireless communication protocols reported by the device into a unified communication protocol at the application layer.

(4) Edge computing process (automatic control) Proc_Auto

Obviously, this requires a separate process to process all kinds of computing, which is equivalent to the brain of the system.

(5) Wireless communication protocol related processes Proc_ZigBee, Proc_RF, Proc_ZWave

In hardware, each wireless communication module communicates with the CPU to the gateway through serial port or other hardware connection. Therefore, each wireless communication module needs a corresponding process to deal with.

(6) other "soft device" process Proc_Xxx

In the previous project, we also encountered some hardware devices, which are logically at the same level as door magnets, sockets and other devices, but are connected to the gateway through TCP. Such devices can also be managed using a separate process.

The running model of these processes in the gateway is as follows:

2 MQTT message bus

These processes need to communicate with each other, not simply point-to-point communication, but a mesh communication model. For example:

Device management process Proc_DevMgr: when any device is added to the system, it needs to be processed, so it needs to communicate with Proc_ZigBee, Proc_RF, Proc_ZWave and other processes.

When a device reports data (for example, Proc_ZigBee), the Proc_Protocol process needs to convert the data to the protocol, and then the Proc_Bridge process reports the converted data to the server, while the Proc_Auto process needs to check whether the data reported by this device has triggered other associated devices.

In other words, the communication between these processes crosses each other, and it is more complex to deal with it through traditional IPC methods (shared memory, named pipes, message queues, Socket, etc.).

With the introduction of the MQTT message bus, each process only needs to be mounted to the bus. Each process only needs to listen to the topic it is interested in to receive the corresponding data.

Since the communication relationship between these processes is complex, a good topic design specification is very important!

3. Design of Topic

The communication model of MQTT is based on the subscription / publish model. After a client (process) connects to the message bus, it needs to register the topic topic that it is interested in, and other clients (processes) send messages to this topic, which can be received by subscribers.

The topic topic is a string separated by a backslash (/) to represent a hierarchical structure with multiple layers. For example, the following two topic are topic related to online upgrade (OTA) in Amazon AWS platform:

$aws/things/MyThing/jobs/get/accepted $aws/things/MyThing/jobs/get/rejected

In our example scenario, the theme topic can be designed as follows:

(1) Proc_DevMgr

Subscribe to topics:

$iot/v1/ZigBee/Register $iot/v1/ZigBee/UnRegister $iot/v1/RF/Register $iot/v1/RF/UnRegister $iot/v1/ZWave/Register $iot/v1/ZWave/UnRegister

(2) Proc_Bridge

Subscribe to topics:

$iot/v1/Device/Report

The subject on which the data is sent:

$iot/v1/Device/Control $iot/v1/Device/Remove $iot/v1/Auto/AddRule $iot/v1/Auto/RemoveRule

(3) Proc_Protocol

Subscribe to topics:

$iot/v1/Device/Control $iot/v1/Device/Remove $iot/v1/ZigBee/Report $iot/v1/RF/Report $iot/v1/ZWave/Report

Send data topic:

$iot/v1/Device/Report $iot/v1/ZigBee/Control $iot/v1/ZigBee/Remove $iot/v1/RF/Control $iot/v1/RF/Remove $iot/v1/ZWave/Control $iot/v1/ZWave/Remove

(4) Proc_Auto

Subscribe to topics:

$iot/v1/Auto/AddRule $iot/v1/Auto/RemoveRule $iot/v1/Device/Report

Send data topic:

$iot/v1/Device/Control

(5) Proc_ZigBee

Subscribe to topics:

$iot/v1/ZigBee/Control $iot/v1/ZigBee/Remove

Send data topic:

$iot/v1/ZigBee/Register $iot/v1/ZigBee/UnRegister $iot/v1/ZigBee/Report

(6) Proc_RF

Subscribe to topics:

$iot/v1/RF/Control $iot/v1/RF/Remove

Send data topic:

$iot/v1/RF/Register $iot/v1/RF/UnRegister $iot/v1/RF/Report

(7) Proc_ZWave

Subscribe to topics:

$iot/v1/ZWave/Control $iot/v1/ZWave/Remove

Send data topic:

$iot/v1/ZWave/Register $iot/v1/ZWave/UnRegister $iot/v1/ZWave/Report

The design of the above themes topic is still somewhat sketchy. With the help of wildcards (#, +, $), you can design a more flexible hierarchy.

Multi-level wildcards: "#" is a wildcard used to match any level in a topic, which represents its parent and any number of child levels.

Single-layer wildcards: the "+" plus sign is a wildcard that can only be used for matching at a single topic level, and single-level wildcards can be used at any level of the topic filter, including the first and last levels.

Wildcard: "$" means to match a character, as long as it is not placed at the beginning of the topic, otherwise it means to match a character.

Let's take a control instruction as an example to sort out how data flows through topic:

The Proc_Bridge process receives the control instruction from the server and sends it to topic: $iot/v1/Device/Control on the message bus.

Because the Proc_Protocol process subscribes to this topic, it receives the instruction immediately.

Proc_Protocol analyzes the instruction contents and finds that it is a ZigBee device, so it converts the protocol and sends a ZigBee control instruction to topic: $iot/v1/ZigBee/Control on the message bus.

Because the Proc_ZigBee process subscribes to the topic, it receives the control instruction.

Proc_ZigBee converts the control instructions into the format required by the ZigBee wireless communication module and sends them to the device light bulb through hardware.

Let's analyze the scenario in which the device data is reported:

Focus on the red arrow in the figure first, ignoring the blue arrow:

After the magnetic door is opened, the information is reported to the process Proc_CF through wireless communication.

The Proc_RF process receives the data reported by the RF433 communication module and sends the "door magnetic open" message to the topic:$iot/v1/RF/Report on the message bus.

Because the Proc_Protocol process subscribed to this topic, it received the reported gate magnetic data.

Proc_Protocol analyzes the data, converts the data of RF433 protocol into the data of unified application layer protocol, and sends it to topic:$iot/v1/Device/Report on the message bus.

Because the Proc_Bridge process subscribed to this topic, it received the data reported this time.

The Proc_Bridge process reports the data to the server.

Let's take a look at the blue arrow process:

In step 4 above: after the Proc_Protocol process converts the RF433 protocol data into the application layer unified protocol, and after sending the data to topic:$iot/v1/Device/Report on the message bus, the Proc_Auto process simultaneously performs the following operations:

5. Because Proc_Auto also subscribed to this topic, it also received the data of this application layer protocol reported by door Magnetic.

6.Proc_Auto looks up its own configuration information (assuming that the user has configured a rule in advance: when the door magnetic opens, the acousto-optic alarm is triggered), and finds that it matches the "door magnetic-> alarm" rule, so it sends an instruction to control the alarm and sends it to topic: $iot/v1/Device/Control on the message bus.

The following four steps are exactly the same as the above control instruction flow.

Comparison between 4 and DBUS bus

From the three data flow scenarios described above, do you feel that the use of topic as a "data pipeline" is particularly similar to the DBUS bus in the Linux system?

DBUS bus is also used for communication between processes. According to my personal understanding, DBUS actually organizes two kinds of communication between processes:

Signal-based data transmission

Method-based RPC remote invocation

The DBUS bus contains more complex concepts, including paths, objects, interfaces, methods, and so on, which are organized together to locate a specific service provider.

In comparison, I think MQTT's approach is a little more concise.

The so-called RPC remote call is to call a function located on the remote machine, which mainly solves two problems:

Network connection

Serialization and deserialization of data

III. Communication between gateway and cloud platform

The design process described above is the way of communication between the functional modules within the gateway, which is also the part that we can give full play to as embedded developers.

The communication methods between gateways and cloud platforms are generally specified by customers (Aliyun, Huawei Cloud, Tencent Cloud, Amazon AWS platform). It is generally required that there is a long connection between the gateway and the cloud platform, so that various instructions on the cloud can be sent to the gateway at any time.

Of course, these cloud platforms will provide the corresponding SDK development package, generally using the MQTT protocol to connect to the cloud platform more. In some documents, the MQTT server located in the cloud is called Broker, which is actually a server.

There are two main functions of process Proc_Bridge:

Data transmission channel with cloud platform

Protocol conversion: convert the protocols related to the cloud platform into the protocols within the gateway, and vice versa.

In other words, the Proc_Bridge process needs to connect to both the MQTT Broker of the cloud platform and the MQTT message bus inside the gateway.

At this point, I believe you have a deeper understanding of "what is the design process based on MQTT message bus". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.