Shulou(Shulou.com)06/03 Report--

Yesterday, I happened to talk about the issue of IOT timing processing in industrial scenarios. Some people discussed which timing database is better, and some people asked whether data collection should be built by cloud service providers' PaaS or self-built, to name a few. The author believes that there is no strongest product, only the most suitable architecture. Putting aside the specific application scenarios and talking about the performance of a single product is tantamount to seeking fish from wood and seeking swords from a boat. According to the specific situation of customers, choose the appropriate architecture, flexibly combine open source tools and cloud vendor services; at the same time, based on the principle of practicality, solve problems rather than choose some products with limited practicality for the sake of the so-called latest technology. Here to share some personal views, the level is limited, throw a brick to attract jade, welcome to discuss.

First of all, we can divide the whole system into several parts, and then discuss them one by one. For the time being, the whole architecture is divided into three parts: terminal, analysis and action.

In the "terminal" part, Microsoft has taken a lot of action recently, and most of the expected investment of US $5 billion in IOT research and development has been spent here, among which the product AzureSphere is highly expected. at present, it has been jointly produced and developed with TSMC and other manufacturers. × × has to fly for a while ~ its biggest advantage is two sets of security mechanisms to ensure the security of customer terminals.

Dark Technology Azuresphere Portal: https://azure.microsoft.com/en-us/services/azure-sphere/

In the "Analysis" section, Microsoft IOT solution contains a wide range of products.

Portal: https://azure.microsoft.com/zh-cn/overview/iot/

Let's start with data collection. Several common methods of data collection are as follows:

The two most commonly compared methods are using cloud vendor PaaS (here Microsoft IOT Hub,Event Hub as an example) vs open source self-built (here Kafka as an example). The comparison is as follows:

Use Microsoft Services

(IOT Hub)

Use Microsoft Services

(Event Hub)

Self-built

(Kafka)

Hosting

Yes

Yes

None

Parallelism

Yes

Yes

Yes

Transmission direction

Two-way

One-way

One-way

Equipment management

Yes

None

None

Transfer

Configurable

At least once.

At least once.

Support protocol

MQTT,HTTPS,AMQP

HTTPS 、 AMQP 1.0 、 Apache Kafka

Kafka

Expandability

Relatively high (can easily scale to TB level)

Relatively high (can easily scale to TB level)

Relatively low

Direct cost

High

Medium

Low

administration cost

Low

Low

High

In the real industrial situation, whether the cloud vendor PaaS can be used to collect data is often determined by the customer terminal equipment, especially the protocol supported by the device. For example, most of Honeywell's industrial machines can use Microsoft IOT to collect data; for example, Siemens has its own system, which is not open, and Microsoft can only collect data by itself, process and sort them out, and then transmit them to Microsoft through customers to do follow-up work such as data analysis and presentation.

Among them, IOT Hub is still in the process of evolution, and devices streams is coming soon, which will bring better ease of use and security to device-side links:

Https://azure.microsoft.com/zh-cn/blog/introducing-iot-hub-device-streams-in-public-preview/

If customers use open source, common nouns such as Kafka, Flume and Storm can be found in the following blog with basic introduction and step-by-step code:

Http://www.cnblogs.com/smartloli/p/4615908.html

Https://www.cnblogs.com/smartloli/p/4632644.html

Data storage and computing cloud vendors all have powerful services, and their functions are more or less the same, so I won't repeat them here. Here are two technical details for discussion:

Database selection: although the time series database is noisy, there are few practical applications. It has certain application scenario value for similar matrix data that need to be displayed for a long time. More often, the traditional database or Hadoop system has a wealth of best practices, the choice depends on the actual situation, simply can solve the problem.

Compressing data: this technique is easy to ignore, but it can be considered especially in the case of a large amount of data. A balance needs to be made here, that is, the trade-off between the increase of computation caused by compression and decompression and the reduction of storage / network load. On the one hand, the obvious advantage is that compressing data can save storage costs, and in the case of distributed computing, it can save the network pressure of data transmission between nodes, reduce the transmission time, and increase the speed of operation.

The comparison of common compression tools is as follows:

The next step is the selection of the data analysis engine. The common selection criteria are query compatibility and latency:

In addition to the innovation of tools, with the evolution of the architecture, serverless is now very popular, and the traditional architecture can also be developed into a fully automatic connection architecture divided into fast path and slow path channels.

The "Action" part, depending on the customer's specific application scenarios, some adopt preventive maintenance, some adopt remote switch, and so on, which will not be carried out here.

Industrial data analysis has been rampant, it seems that the traditional manufacturing industry to adhere to the digital transformation of the bull ear, talk about GE;, and with a series of bubbles extruded, Predix also fell into a fate of sale, quite a bit of "look at the setting sun on the fields, turn from the beginning" flavor. As a practitioner, the general trend can not be violated, start from the small, accumulate bit by bit, join the river and the sea, encourage it together.

The technology is broad and profound, the personal ability is limited, the writing is vulgar, throw a brick to attract jade, welcome to discuss and correct, make progress together.

Reference materials:

Microsoft IOT reference Architecture:

Azure IoT Reference Architecture Guide

Microsoft's mature case in IOT:

Rolls Royce https://customers.microsoft.com/en-us/story/rollsroycestory

POC the blog with specific steps is here:

Https://mp.weixin.qq.com/s/rMJZ2At6AGVqTVZ5ZB0GLA?

The POC code is here:

Https://github.com/jurejoy/Temperature-Forecast

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.