Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use MQTT and function calculation to do heat map", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "how to use MQTT and function calculation to do heat map"!

Features and requirements of the scene:

1. Connection capability of data channel: with the expansion of business, the release of robots will increase synchronously. There is enough flexibility to expand the data channel so that it can be expanded as needed. At the same time, the connection level can support 10W + expansion.

2. The ability of concise data cleaning: the processing of data is essentially the induction and statistics of the data, and the logical implementation is not complex. For the peak and valley changes of the data itself, we can have the most simple and effective matching expansion processing capacity, and we do not want to introduce a complex traditional big data level bulky scheme for this purpose in cleaning.

3. The ability of flexible data access: the heat map information mentioned here will be open to end users in the future. The number of visits varies dynamically, and there will be unpredictable changes with different times, festivals, emergencies, and so on. Therefore, flexible access capability is required in this business. The business side does not want to achieve it through current restriction, because it will have an impact on the business volume itself.

4. Storage capacity with superior performance: in this scenario, the concurrency of data writing and reading is high, and customers want to use NoSQL for storage. The NoSQL type can best support the sorting function. Redis is used in the scheme introduced in this article, so there is no more analysis and introduction.

Analysis of alternative technical solutions

Connectivity of data channels

Self-built Kafka

Advantages:

As a general information channel for data collection, Kafka is widely used and diversified in access methods. The community is perfect and the learning cost is low.

Kafka itself is easy to build, and the coordination plan with downstream big data processing products is mature.

Disadvantages:

It is complex to dynamically deal with the expansion of Kafka.

Additional stability solutions for dealing with clusters need to be set up.

External network traffic management needs to cooperate with additional schemes.

The mainstream scheme is as the collection capability of the connected application, and there is no case verification at the scale level for the connectivity of the terminal.

Message queuing MQTT scheme

Advantages:

Support millions of connections, complete the demand that can cover business development, and leave enough room for business expansion.

The protocol of MQTT is very concise and has an advantage in the transmission between end and service. The QoS quality reached by various messages is supported.

Support a variety of client access implementation languages.

The connection of the client can be observed in real time, and it is convenient to find the abnormal situation.

Disadvantages:

The practice of dealing with big data is not as mature as Kafka, and the selection of downstream products is subject to certain restrictions.

Resilient data cleaning capability

Big data scheme (Storm, Spark, Flink, etc.)

Advantages:

The general open source scheme has a lot of information and the scheme is mature.

Disadvantages:

The construction of operation and maintenance is complex, so it is necessary to provide additional monitoring and recovery means.

You need to learn to accept a variety of component ways (the following figure is an example of Storm).

The use of resources is evaluated in advance, and it is impossible to expand and shrink the use according to the amount of real-time data.

Function calculation scheme

Advantages:

Expand and shrink as needed, 100 millisecond expansion ability, suitable for the pulse peak and valley change of the amount of data.

There is no need to manage the cleaning environment.

The concept is simple and the learning cost is low.

For other benefits, refer to the following figure:

Disadvantages:

Function computing is the product of various cloud vendors. The requirement must run on the cloud.

The ability to flexibly access data

Schemes for traditional applications

Advantages:

As part of the business, embedded in an application implementation, the technology is mature and the learning cost is low.

Disadvantages:

Self-implementation is required to retract according to the amount of business requests, or often use evaluation to deal with resource redundancy.

Calculation scheme of API Gateway+ function

Advantages:

The retractive processing is carried out in real time according to the customer's request quantity. Use on demand, do not worry about peak hours, will not pay idle.

Automatically comes with a professional access monitoring market.

Disadvantages:

It requires a small amount of learning cost.

Summary

In this heat map information collection, sorting and access business, you can refer to the perfect implementation of the solution using the figure below.

Key access steps

Introduction to the calculation of MQTT to function

Please refer to the micromessage queuing MQTT service integration solution calculated by the function.

Introduction of API Gateway extracting data through function calculation

For more information, please see API Gateway function trigger instance.

Take Node.js as an example:

Module.exports.handler = function (event, context, callback) {var event = JSON.parse (event); var content = {path: event.path, method: event.method, headers: event.headers, queryParameters: event.queryParameters, pathParameters: event.pathParameters, body: event.body / / you can write your own logic here. / / the logic to extract data from Redis} var response = {isBase64Encoded: false, statusCode: '200customers, headers: {' header value'}, body: content}; callback (null, response)} At this point, I believe you have a deeper understanding of "how to use MQTT and function calculation to do heat map". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.