Shulou(Shulou.com)06/02 Report--

This article is about how nginx achieves high performance and scalability. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

The overall architecture of NGINX is characterized by a set of processes working together:

Main process: responsible for performing privileged operations, such as reading configuration files, binding sockets, and creating / notifying coordination (Signalling) child processes.

Working process: responsible for receiving and processing connection requests, reading and writing to disk, and communicating with upstream servers. When NGINX is active, only the worker process is busy.

Cache loader process: responsible for loading the disk cache into memory. This process exits immediately after it is run at startup.

Cache manager process: responsible for collating disk cached data to ensure that it does not cross the bounds. This process runs intermittently.

The key to achieving high performance and scalability for NGINX depends on two basic design choices:

Limit the number of worker processes as much as possible to reduce the overhead of context switching. The default and recommended configuration is to have each CPU kernel correspond to a worker process to make efficient use of hardware resources.

The worker process is single-threaded and handles multiple concurrent connections in a non-blocking manner.

Each worker process of NGINX processes multiple connection requests through a state machine, which is implemented in a non-blocking manner:

Each worker process needs to process several sockets, including listening sockets or connection sockets.

When the listening socket receives a new request, a new connection socket is opened to handle communication with the client.

When an event arrives at a connection socket, the worker process responds quickly and turns to processing any other newly received events from the socket.

Garrett said that NGINX chose this design to make it fundamentally different from other Web servers. Typically, Web servers choose the mode of assigning each connection to a separate thread, which makes it easy to handle multiple connections because each connection can be thought of as a linear sequence of multiple steps, but this results in context switching overhead. In fact, worker threads spend most of their time blocking, waiting for clients or other upstream servers. The cost of context switching becomes apparent when the number of concurrent connections / threads trying to perform operations such as Icano exceeds a certain threshold, or when memory is exhausted.

On the other hand, NGINX is designed not to allow worker processes to block network traffic unless there is no work to be done. In addition, each new connection consumes very few resources, including only a file descriptor and a small amount of worker process memory.

In general, this working mode of NGINX can handle hundreds of HTTP concurrent connections per worker process after system tuning.

Going deep into NGINX: how do we design its performance and scalability

The reason why NGINX is so superior in performance is due to the design behind it. While many web servers and application servers use a simple threaded (threaded) or process-based (process-based) architecture, NGINX stands out with a complex event-driven (event-driven) architecture that supports thousands of concurrent connections on modern hardware.

Inside NGINX infographic covers everything from the mining of high-level process architectures to the diagrams of single-process processing and multiple connections in NGINX. This article explains the details of these work.

Setting up the scene-- NGINX process model

Setting the Scene? The NGINX Process Model

To better understand design, you need to understand how NGINX works. NGINX has a main process (master process) that performs privileged operations such as reading configuration and binding ports, and a series of worker processes (worker process) and worker processes (helper process).

In this quad-core server, the NGINX main process creates four worker processes and two cache worker processes (cache helper processes) to manage the disk content cache (on-disk content cache).

Why is architecture important?

Why Is Architecture Important?

The fundamental foundation of any Unix application is a thread or process. (from the perspective of the Linux operating system, threads and processes are basically the same, with the main difference being the extent to which they share memory.) A process or thread is a set of independent instructions that can be scheduled by the operating system and run on the CPU kernel. Most complex applications run multiple threads or processes in parallel for two reasons:

● can use more computer cores at the same time.

● threads and processes make parallel operations easy to implement (for example, handling multiple connections at the same time).

Both processes and threads consume resources. They all use memory and other OS resources, causing frequent kernel switching (an operation called context switching (context switch)). Most modern servers can handle hundreds of small, active (active) threads or processes at the same time, but once memory is exhausted, or high Imax O load leads to a large number of context switches, server performance degrades seriously.

For network applications, a thread or process is usually assigned to each connection. This architecture is easy to implement, but its scalability can be problematic when applications need to handle thousands of concurrent connections.

How does NGINX work?

How Does NGINX Work?

NGINX uses a predictable process model to schedule available hardware resources:

1. The main process performs privileged operations, such as reading configuration and binding ports, and is also responsible for creating child processes (the following three types).

two。 The cache loading process (cache loader process) runs at startup, loads the disk-based cache (disk-based cache) into memory, and then exits. Its scheduling is very cautious, so its resource requirements are very low.

3. The cache management process (cache manager process) runs periodically and the disk cache (prunes entries from the disk caches) is reduced to keep it within the configuration range.

4. The worker processes is the process that performs all the actual tasks: handling network connections, reading and writing to disk, communicating with upstream servers, and so on.

In most cases, NGINX recommends that each CPU core run a worker process to make the most efficient use of hardware resources. You can set the following instructions in the configuration:

Worker_processes auto

When the NGINX server is running, only the worker process is busy. Each worker process handles multiple connections in a non-blocking manner to reduce the overhead of context switching.

Each worker process is single-threaded and runs independently, grabbing and processing new connections. Cached data, session persistence data (session persistence data), and other shared resources are shared among processes by sharing memory.

The working process within NGINX

Inside the NGINX Worker Process

Each NGINX worker process is initialized by NGINX configuration (NGINX configuration) and a set of listening sockets (listen sockets) is set by the main process.

The NGINX worker process listens for events (accept_mutex and kernel socket sharding) on the socket to decide when to start working. The event is initialized by a new connection. These connections are assigned to state machines (state machine)-HTTP state machines are the most commonly used, but NGINX also implements state machines for streams (native TCP) and a large number of mail protocols (SMTP,IMAP and POP3).

A state machine is essentially a set of instructions that tell NGINX how to handle a request. Most web servers that have the same functionality as NGINX use similar state machines-- only with different implementations.

Scheduling state machine

Scheduling the State Machine

Think of the state machine as the rules of chess. Every HTTP HTTP transaction is a chess game. On one side of the board is the web server-a master player who can make decisions quickly. On the other side is the remote client-the web browser that accesses the site or application in a relatively slow network.

However, the rules of the game can be complicated. For example, the web server may need to communicate with the parties (proxying an upstream application) or with the authentication server. The third-party module of the web server can also extend the rules of the game.

Blocking state machine

A Blocking State Machine

Recall our previous description of processes and threads: a set of independent instructions that are schedulable by the operating system and run on the CPU kernel. Most web servers and web applications use a connection / process or a connection / thread model to play this chess game. Each process or thread contains an instruction to play the game to the end. In this process, the process is run by the server, and most of its time is spent "blocked", waiting for the client to complete its next action.

The 1.web server process (web server process) listens for new connections (new competitions initiated by clients) on listening sockets.

two。 After a new game is launched, the process begins to work, and after each move is played, it goes into a blocking state, waiting for the client to take the next move.

3. Once the game is over, the web server process will see if the customer wants to start a new game (which is equivalent to a viable connection). If the connection is closed (the client leaves or times out), the web server process returns to the listening state, waiting for a new game.

It's important to remember that every active HTTP connection (every chess game) requires a dedicated process or thread (a master player). This architecture is very easy to extend third-party modules ("new rules"). However, there is a huge imbalance: a lightweight HTTP connection, represented by a file descriptor (file descriptor) and a small amount of memory, maps to a single process or thread-- very heavyweight operating system objects. This is convenient in programming, but it causes a lot of waste.

NGINX is a true master.

NGINX is a True Grandmaster

You may have heard of the wheel exhibition race, in which a chess master has to deal with dozens of opponents at the same time.

Kiril Georgiev played 360 players at the same time in Sofia, Bulgaria, with a record of 284 wins, 70 draws and 6 losses.

This is how the NGINX worker process plays chess. Every worker process is a master (remember: usually, each worker process takes up a CPU kernel) and can play hundreds of chess players (actually thousands) at the same time.

1. The worker process waits for events on listening sockets and connection sockets.

two。 Events occur on sockets, and the worker process handles them.

The event on the ● listening socket means that the client has started a new game. The worker process creates a new connection socket.

The event on the ● connection socket means that the client has moved the pawn. The working process will respond quickly.

The work process never stops on the network, and it waits for its "opponent" (client) to respond all the time. When it has moved the pieces of this game, it will immediately deal with the next game, or meet a new opponent.

Why is it faster than a blocking multiprocess architecture?

Why Is This Faster than a Blocking, Multi-Process Architecture?

The scale of NGINX can well support tens of thousands of connections in each work process. Each new connection creates another file descriptor and consumes a small amount of extra memory in the worker process. The extra consumption for each connection is very small. The NGINX process can maintain a fixed CPU utilization. When there is no work, there is less context switching.

In a blocking, one-connection / one-process mode, each connection requires a lot of additional resources and overhead, and context switching (from one process to another) is very frequent.

To learn more, check out the article on NGINX architecture written by Andrew Alexeev, vice president of development and co-founder of NGINX.

With proper system tuning, NGINX can handle hundreds of thousands of concurrent HTTP connections per worker process on a large scale without losing any information during peak traffic periods (new race starts).

Configuration updates and NGINX upgrades

Updating Configuration and Upgrading NGINX

The NGINX process architecture, which contains only a small number of worker processes, makes the configuration, and even the update of the binary itself, very efficient.

Updating the configuration of NGINX is a very simple, lightweight, and reliable operation. Just run the nginx? s reload command, which checks the configuration on disk and sends a SIGHUP signal to the main process.

When the main process receives the SIGHUP signal, it does two things:

1. Reload the configuration and fork a new set of working processes. These new worker processes immediately begin to accept connections and process traffic (traffic) (using the new configuration).

two。 Signal the old work process to exit quietly. These old processes will not accept new connections. As long as the HTTP request they are handling ends, they will cleanly close the connection. Once all connections are closed, the worker process exits.

This process results in a small peak in CPU usage and memory usage, but this small peak is negligible compared to loading resources from active connections. You can reload the configuration multiple times in a second. In rare cases, generation after generation of worker processes have problems waiting for the connection to close, but even if they do, they will be resolved immediately.

The binary upgrade process of NGINX is even more magical-you can upgrade NGINX itself very quickly without any loss of connection, downtime, or service interruption on the server.

The binary upgrade process is similar to configuration updates. The new NGINX main process is parallel to the original main process, and they share listening sockets. Both processes are active (active) and their respective worker processes handle their respective traffic (traffic). You can then notify the old main process and its working process to exit perfectly.

In Controlling NGINX, the whole process is described in more detail.

Thank you for reading! This is the end of this article on how to achieve high performance and scalability in nginx. I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.