Shulou(Shulou.com)06/02 Report--

   event-driven programming is a programming paradigm where the execution flow of a program is determined by external events. It is characterized by an event loop that uses a callback mechanism to trigger the corresponding processing when an external event occurs. Multithreading is another common programming paradigm and is easier to understand.

   high-performance universal C++ network framework Nebula is an event-driven multi-process network framework (suitable for instant messaging, data acquisition, real-time computing, message push and other application scenarios). There have been production application cases of instant messaging, buried data acquisition and real-time analysis. People often ask whether each process in Nebula is single-threaded or multithreaded. Why not use multithreading? How do you deal with concurrency without multithreading?

   recently Nebula will be used in a new production project-recommendation engine, before which the team has used a well-known multi-threaded version of the RPC framework recommendation engine (many recommendation engines in the industry have been developed using the well-known open source RPC framework). This article does not compare Nebula with other well-known RPC frameworks, nor does it intend to explain which framework is more suitable for recommendation engine, but only that Nebula can be used in recommendation engine, and is confident that the effect will be very good. What the final result will be, wait and see when the recommendation engine is developed.

Why is    event-driven rather than multithreaded? Event drivers do not require multithreading. Let's first review the server programming paradigm.

1. Server programming paradigm

Nine server design paradigms are introduced in    "UNIX Network programming" Volume 1:

Not all of the nine    server design paradigms are of practical value. Several code examples of TCP server design paradigms are given in the last section of the UNIX Network programming volume:

TCP concurrent server program, each client has a child process TCP pre-derived child process server program TCP pre-derived child process server program, passing descriptor TCP concurrent server program, each client has a thread TCP pre-created thread server program, each thread acceptTCP pre-created thread server program, and the main thread unifies accept

   Nginx uses the fifth of the nine server design paradigms, "pre-spawning child processes, using mutex locks to protect accept", and Nebula uses the sixth of the nine server design paradigms "pre-spawning child processes, where the parent process passes socket file descriptors to the child process."

two。 Comparison of single-thread, multi-thread and event-driven programming models

A typical event-driven program in    is an endless loop, which exists in the form of a thread. The dead loop consists of two parts. The first part is to receive and select an event to be handled according to certain conditions, and the second part is the event handling process. The execution process of the program is to select and handle events, and when no event is triggered, the program will go to sleep due to the failure of querying the event queue, thus releasing cpu.

   in a sense, server-side programs are mostly event-driven, or IO request event-driven. The event-driven in the programming model compared here means that the event handling part is asynchronous, that is, not only IO request event-driven, but also IO response event-driven. Its characteristic is that when the external IO response event occurs, it uses the callback mechanism to trigger the corresponding processing.

   in the single-threaded synchronization model, tasks are executed sequentially. If a task is blocked by ICompO, all other tasks must wait until it is complete before they can be executed in turn. This clear order of execution and serialization behavior is easy to infer. If the tasks are not interdependent, but still need to wait for each other, this makes the program slow down unnecessarily.

   is in the multithreaded model, where each task is executed in a separate thread. These threads are managed by the operating system and can be processed in parallel on multiprocessor systems or interleaved on uniprocessor systems. This allows other threads to continue execution while one thread blocks a resource. This approach is more efficient than synchronous programs that perform similar functions, but programmers must write code to protect shared resources from being accessed by multiple threads at the same time. Multithreaded programs are more difficult to infer because they have to deal with thread safety issues through thread synchronization mechanisms such as locks, reentrant functions, thread local storage, or other mechanisms, which, if not implemented properly, can lead to subtle and excruciating bug. Another problem is that when the operating system kernel switches threads, it also switches the context of threads. When there are too many threads, time will be spent in context switching. Therefore, in the case of large concurrency, the multithreaded structure is still unable to achieve strong scalability.

   in the event-driven version of the program, three tasks are interlaced, but are still under a separate thread control. Register a callback into the event loop when dealing with Icano or other expensive operations, and then continue when the Icano operation is complete. The callback describes how to handle an event. The event loop polls all events and assigns them to callback functions waiting to handle the event when the event arrives. This approach allows the program to be executed as much as possible without the need for additional threads. When there is no IO operation, each task takes up less cpu time, and the process will be idle. In the case of the same amount of concurrency, the system resources consumed by the event driver are better, and when the load is large enough, the event driver can utilize cpu to 100%. Event-driven programs are easier to infer behavior than multithreaded programs because programmers do not need to care about thread safety.

3. Event driven! = only one thread

A very representative implementation of    event-driven Node.js and redis is a single-process (single-threaded) service (redis data landing or master-slave synchronous thread exclusion, the service is single-threaded), and event handling is performed through asynchronous callbacks. In the comparison of single-thread, multi-thread and event-driven programming model in the second section, it seems that event-driven is single-threaded, and Node.js, a typical event-driven service, is also single-threaded, which leads many people to think that event-driven can only be single-threaded and can not make full use of multi-CPU and multi-core resources. In fact, Nginx is also a typical event-driven service, while Nginx is multi-process. The back-end services are logically divided, Nginx is classified as the access communication layer (the implementation of business logic by openresty is not discussed in nginx+lua), and Node.js is classified as the business logic layer. The characteristic of access communication layer is that IO behavior consumes little and CPU is naturally suitable for event-driven and easy to implement, while the characteristics of business logic layer determine that the implementation of event-driven mode is very complex, but this does not mean that multi-thread event-driven in business logic layer is difficult to implement.

   Nebula is a typical multi-process event-driven service. Each event-driven process is efficient enough, and multiple processes (multi-threads) make full use of multi-CPU and multi-core resources. The process model of Nebula is similar to that of Nginx, except that Nginx is locked by each worker, while Nebula is passed to the worker process by the master process after accept (similar to Memcached). Nebula is developed from the Starship framework developed for instant messaging applications, and it is accidental that it is similar to nginx's process (thread) model. Why did Nebula choose to transfer file descriptors instead of worker processes robbing accept? It has something to do with Nebula positioning. Nebula needs to be not only the access communication layer and data proxy layer, but also the business logic layer. All layers of distributed services can and should be implemented by Nebula, which means that every worker process is close to the function of a node of distributed services. If it is a worker preemptive accept, it cannot do directional routing. Why choose multi-process instead of multi-threading? Let's first look at the advantages and disadvantages of multi-process and multi-threading:

   multiprocess:

Programming is relatively easy; usually there is no need to consider locking and synchronizing resources: one advantage over multithreading is that one process crashes without affecting the kernel-guaranteed isolation of other processes: data and error isolation is expensive to switch

   multithreading:

The creation speed is fast to share data, and multi-threads can share the same virtual address space. Data sharing between multiple processes requires shared memory, semaphores and other IPC technology. Light context switching overhead once one thread dies, the whole process may fail and need to synchronize access to shared resources.

The first three points of    multiprocess are advantages, and the fourth is disadvantages. When Nebula chooses multiple processes, it does not need to consider locking and synchronizing resources, data and error isolation, worker process crash will not affect the entire node service, will be quickly pulled up by the master process. The fourth disadvantage does not need to be considered in Nebula, because there is no need to switch between Nebula event-driven processes, it can be approximately considered that each worker process is a node, and there is only network communication between nodes, and there is no need to share resources and do not need to switch.

4. Event-driven applicable scenarios

   for IO-intensive business, event-driven than multi-thread synchronization concurrency ability is much higher, it can be said that it is not an order of magnitude. Most of the Internet services are IO-intensive, so event-driven scenarios are very extensive. There are many highly independent tasks in the program, some of which block while waiting for an event to arrive, and a single task requires less CPU resources.

   Nebula is suitable for instant messaging, data acquisition, real-time computing, message push and other application scenarios, as well as web backend services. Nebula has a production application case of instant messaging, buried point data acquisition and real-time analysis, and there will soon be a recommendation engine production application case for hundreds of millions of users.

5. Recommendation engine framework selection

When    talks about recommendation system, the first thing that comes to mind may be recommendation technologies such as content-based, collaborative filtering, demographics-based, knowledge-based, community-based, hybrid recommendation and so on. The implementation of recommendation technology is usually based on hadoop, which is implemented by hive, spark, storm, flink and so on. These are often referred to as recommended data mining parts.

   recommendation engine is one of the cores of recommendation system, which is responsible for pushing the results of data mining to users according to a certain sort, which is the main function of recommendation engine.

   knows that industry recommendation engines use C++ development and Java development, and C++ development accounts for the majority. Most of the recommendation engines developed by C++ that Bwar learned about use the rpc framework, using 4 of thrift, 2 of brpc, 1 of grpc, and 1 of tars. Because these open source rpc frameworks are not specifically developed for recommendation engines, developers usually build another layer of framework on top of these frameworks, followed by business logic development. One of the recommendation engines contacted by Bwar is to develop its own framework based on brpc and then to develop business logic, which is difficult to develop and not easy to expand. Perhaps developers do not have a deep understanding of these open source rpc frameworks, which leads to the complexity of business logic development and the difficulty of subsequent requirements expansion.

   Nebula is a C++ network framework developed by Bwar. It is born as a distributed service and has been applied in two production environments. Nebula is not a rpc framework but a basic proactor (framework layer implementation of proactor rather than operating system support) event-driven (callback) framework. Unlike most asynchronous event callback frameworks, developers need to register callback functions themselves, Nebula is also an IoC framework, through the ingenious design and implementation of actor classes to reduce the complexity of asynchronous programming, developers really only need to focus on business logic development.

The Cmd class provided by the    Nebula framework is very suitable for the logical entrance of the recommendation service, supports dynamic loading, and upgrades the recommendation algorithm recommendation model without downtime. The Step class fetches data from storage such as redis asynchronously, and non-blocking waiting allows cpu resources to be used only for recommendation logic. The session class is used to cache user, item, model, and so on. All data acquisition and transmission can be obtained conveniently and efficiently through session intelligent pointer.

   in those recommendation engines based on the rpc framework, many developers mention reflection and implement so-called reflection through a large number of macros in a laborious and difficult way to understand. These are not IoC frameworks. Bwar does not understand why it is necessary to implement reflection. It will be very easy to do it with Nebula. Nebula is an IoC framework. All actor instances are created through reflection, and developers do not need to do anything other than business logic. Nebula's reflection implementation is elegant, if you are interested, you can refer to this article, "C++ reflection mechanism: variable parameter templates to achieve C++ reflection".

The purpose of    to develop Nebula framework is to provide a high-performance distributed service based on C++. If you think this article is useful to you, don't forget to go to Nebula's Github or Code Cloud to give a star, thank you.

Reference:

Introduction to http://www.aosabook.org/en/twisted.htmlPython Twisted

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.