Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the streaming service". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what the streaming service is.

When the input and output of a service module are streams, we call it a stream service. The advantage of a flow service is that it can visually describe the business execution process.

The flow service uses DAG to describe the execution process. Each node of the DAG represents a business unit, and each business unit is responsible for certain business logic.

In the business unit, some auxiliary services with specific functions are often used, such as IP analysis, GPS analysis, third-party credit service and so on. It may be a good idea to put the code that implements these accessibility functions directly into the business code of the stream computing application, especially when we care a lot about performance.

After all, integrating the logic of these auxiliary functions into streaming applications will reduce a considerable amount of Imax O operations and ensure the performance of streaming applications. But it is not elegant to do so.

Consider that if the flow computing task requires a lot of auxiliary functions (which is actually quite common), and some of the internal logic in these auxiliary functions is even quite complex, then putting all the implementation code of these functions into the implementation of the business process is bound to cause business logic and technical details to crisscross and the program execution process to be disorganized.

One compromise is to extract auxiliary functions into independent source projects, compile them into libraries, and then link to inflow computing applications.

On the one hand, this can ensure the performance of the stream computing application, on the other hand, it can avoid the code of the stream computing application is too cluttered. This is a better approach, and may be the best choice in the case of performance priority.

However, there is also a problem, that is, each time a change or upgrade is made to an accessible functional service, the stream computing application must be rebuilt, tested, and released.

From the perspective of service governance, we should still strip out auxiliary functions and make them separate services that provide access to REST or RPC.

The following figure describes this architecture that strips accessibility functions into separate microservices that are accessed by the flow service invocation interface.

In this way, the flow computing application is responsible for the overall business logic, while the auxiliary functions are encapsulated in independent micro-services and provide a friendly user interface. The whole flow computing system has a clear architecture and is more flexible when it needs to be adjusted in the future.

There is also a problem with invoking external micro-services in a stream service, that is, a performance problem.

In state storage, we recommend using a local data storage scheme instead of a remote data storage scheme, because the remote data storage scheme may greatly reduce the performance of the streaming service. Similarly, the invocation of external micro-services in the flow service also involves the network Icano, which will also significantly reduce the performance of the flow service. Therefore, we need to optimize the invocation process of micro-services.

On the one hand, microservices should be carefully designed to ensure that microservices can return quickly, whether they succeed or fail, and must return quickly within a given time. On the other hand, when the flow service invokes the micro-service, it can adopt the way of asynchronous Imax O, which can ensure that the flow service will not let the CPU block waiting for the micro-service request to return when handling events, thus improving the throughput of the flow service.

In addition, it must be emphasized that the use of microservices in stream computing is best read-only, or at least idempotent. Because if the external state changes when the flow service accesses the micro service, it may destroy the reliability guarantee mechanism of the flow computing application as a whole.

With regard to the causes of this problem, we have analyzed the message transmission reliability assurance mechanisms of various open source flow computing frameworks earlier, and I will not repeat them here.

Compared with streaming service, micro-service is a kind of service organization structure which is better known to the public.

Formally, the biggest difference between micro-service and stream service is that micro-service is the pattern of request and response, while flow service is event-driven. The micro-service system architecture divides the complex software system into independent service modules according to business functions, and each service module is developed, deployed and provided independently. The independent service modules are naturally loosely coupled.

Microservices do help us break down complex systems, but the problem with it is that it complicates business systems.

Compared with the stream service, a brief DAG represents the complete business process. If the micro-service system does not have additional design documents to explain, then it is difficult for us to figure out the complete execution process of the business system at once.

Compared with micro-services, the service governance solution of flow services is "innate" for the following reasons.

First, most streaming computing frameworks are built on distributed operating systems such as YARN, so the environment in which they run is already cloud-based. This means that applications built on these flow computing frameworks are free to scale horizontally.

Second, most flow computing frameworks provide a more or less management interface, which makes it very easy to monitor and track the status of applications running in the system.

Third, most of the flow computing frameworks have a certain fault-tolerant mechanism, and can automatically complete service recovery when the service fails, without our external intervention. But microservices are different.

There is still a big distance between micro-service system architecture and service governance, and it can even be said that the concept of service governance was originally put forward to better manage micro-service systems.

There are a variety of service governance schemes for micro-service systems, from Apache Dubbo and Spring Cloud, to Docker Swarm and Kubernetes, to Service Mesh, etc., various micro-service governance schemes are in the ascendant, and they are in the process of rapid development and evolution.

Thank you for your reading, the above is the content of "what is the streaming service", after the study of this article, I believe we have a deeper understanding of what the convection service is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.