Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to achieve GRPC connection pool". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's ideas to study and learn how to achieve GRPC connection pool.

Preface

In distributed highly concurrent servers, connections between client to server and multiple nodes in server are often managed by connection pooling. To put it simply, the connection created in advance is saved in the pool, and when a request arrives, the connection in the pool is directly used to access the server, saving the overhead of creating and destroying the connection (three-way handshake when TCP establishes the connection and four waves when releasing the connection), thus improving performance.

Design principle

Expansion and contraction of connection pool

Timeout and keeping alive of idle connection

The handling mechanism of pool fullness

Expansion and contraction of connection pool

Typically, the connection pool property contains the maximum number of idle connections and the maximum number of active connections.

Maximum number of idle connections: the number of connections maintained by the connection pool, regardless of whether these connections are used or not. If the client does not make much use of the connection pool, it will cause a waste of connection resources on the server side.

Maximum number of active connections: the maximum number of connections held by the connection pool. If the number of client requests exceeds the number of times, the requests that have not been connected will be processed according to the processing mechanism of the pool full.

Capacity expansion: when the request arrives, if there are no idle connections in the connection pool and the number of connections does not reach the maximum number of active connections, a new connection will be created to serve the request according to a specific growth strategy, and will be returned to the pool after use instead of closing the connection.

Capacity reduction: when the connection pool is not used for a period of time and the number of connections in the pool exceeds the maximum number of idle connections, some connections will be closed so that the number of connections in the pool will always be maintained at the maximum number of idle connections.

Timeout and keeping alive of idle connection

Timeout if the connection is not used by the client, it will become an idle connection. After a period of time, the server may close the idle connection according to its own timeout policy, and the idle connection has expired. If the client uses the invalid connection again, the communication will fail. In order to avoid this situation, the connection in the connection pool usually has a maximum idle timeout (preferably slightly less than the idle connection timeout of the server). When obtaining a connection from the pool, determine whether the idle timeout or not. If it times out, turn it off, and if there is no timeout, you can continue to use it.

Keep alive if the server restarts, all the connections in the connection pool will fail. If you get the connection from the pool again, no matter which one you get, the communication will fail. Therefore, the connection pool must consider the problem of keeping the connection alive, and there are two solutions:

1. Set a Ping function for connection pool, which is specially used to keep the connection alive. When getting a connection from the pool, Ping the server. If you get a response, the connection is still valid and you can continue to use it. If the timeout does not respond, close the connection and generate a new connection. Since you have to Ping every time, the delay will inevitably increase. You can also use a thread or co-program to execute the Ping function periodically in the background to keep the connection alive. The disadvantage is that there will be a certain delay in perceiving the failure of the connection, and it is still possible to obtain the failed connection from the pool.

2. The client adds the corresponding retry mechanism. For example, if you retry 3 times and get the connection from the pool for execution in the first two times, if the error reported is an error related to the connection problem, such as an invalid connection, then when you get it from the pool for the third time, take the parameters and specify that the newly created connection is acquired. At the same time, the connection pool removes the invalid connection obtained from the previous two times.

The handling mechanism of pool fullness

The connection pool cannot hold connections indefinitely, and when the pool is full, there are two processing mechanisms:

1. Create a new connection in the pool and return it to the client. When the client runs out, close the connection if the pool is full, otherwise put it in the pool.

2. Set a certain timeout to wait for an idle connection. The client needs to add a retry mechanism to avoid errors caused by not getting a free connection after a timeout.

Basic principles

Establish a connection pool when the service starts.

Initialize the connection pool and establish a maximum of several idle connections.

When the request comes, get a connection from the pool. If there are no idle connections and the number of connections does not reach the maximum number of active connections, create a new connection; if the maximum number of active connections is reached, set a certain timeout time and wait for the idle connection to be obtained.

Get the communication service after getting the connection.

Release the connection, which is to put the connection back into the connection pool and close the connection if the pool is full.

Release the connection pool and close all connections.

GRPC characteristics

The introduction to GRPC will not be described here. You can read and understand the GRPC protocol in depth, or you can Google on your own. This paper briefly describes the two characteristics of GRPC: multiplexing and timeout reconnection.

Multiplexing GRPC uses HTTP/2 as the transport protocol at the application layer, and HTTP/2 multiplexes the underlying TCP connections. Each RPC call produces a new Stream, and each Stream contains multiple Frame,Frame, which is the smallest data transfer unit in the HTTP/2. At the same time, each Stream has a unique ID identity. If it is created by the client, the ID is odd, and the ID created by the server is even. If the ID on a connection runs out, Client creates a new connection, and Server sends a GOAWAY Frame to Client to force Client to create a new connection. A GRPC connection allows multiple Stream to be sent and received concurrently, and the control parameter is that the server side of MaxConcurrentStreams,Golang defaults to 100.

Timeout reconnection when we create a connection by calling the Dial or DialContext function, the default is to return the ClientConn structure pointer and start a Goroutine asynchronously to establish the connection. If you want to wait for the connection to be established before returning, you can specify grpc.WithBlock () to pass in Options to implement. The timeout mechanism is simple, just pass in a context of timeout when you call it. The reconnection mechanism is realized by starting an Goroutine asynchronously to establish a connection, which can avoid the client connection failure caused by the server closing the connection and restarting the server due to the long idle time of the connection. In other words, the reconnection mechanism of GRPC can perfectly solve the problem of timeout and survival of idle connections in the design principle of connection pool.

Take the GRPC client of Golang as an example:

GRPC tuning

The default parameters of GRPC are not friendly enough for transmitting big data blocks, and we need to tune specific parameters.

The maximum number of bytes allowed to be sent by MaxSendMsgSizeGRPC. The default is 4MiB. If it exceeds the GRPC, an error will be reported. We have both transferred Client and Server to 4GiB.

The maximum number of bytes allowed to be received by MaxRecvMsgSizeGRPC. The default is 4MiB. If it exceeds the GRPC, an error will be reported. We have both transferred Client and Server to 4GiB.

InitialWindowSize Stream-based sliding window, similar to TCP sliding window, used for flow control, default 64KiB, throughput can not go up, Client and Server we set to 1GiB.

InitialConnWindowSize is based on the sliding window of Connection. The default is 16 * 64KiB, and the throughput does not go up. We also set both Client and Server to 1GiB.

KeepAliveTime sends PING frames to measure the minimum round-trip time every KeepAliveTime time to determine whether the idle connection is still valid. We set it to 10s.

KeepAliveTimeout exceeds KeepAliveTimeout, close the connection, we set it to 3s.

If PermitWithoutStream is true, PING frame monitoring is still sent when the connection is idle, and if it is false, ignore is not sent. We set it to true.

Detailed rules for realization

Code: https://github.com/shimingyah/pool

Based on the multiplexing and timeout reconnection characteristics of GRPC, we can easily implement GRPC connection pool.

Interface design

Provides a concise interface design between Pool and Conn.

Connection multiplexing

GRPC supports multiplexing, so one of the differences between designing GRPC pools and other connection pools is that they support connection multiplexing, controlled by MaxConcurrentStreams, which defaults to 64. We call a single GRPC a physical connection and a reused connection a logical connection. The actual valid connection logical connection of the pool = physical connection * MaxConcurrentStreams.

Expansion and reduction of capacity

The valid number of connections (logical connections) of the expansion initialization pool is: the maximum number of idle connections * MaxConcurrentStreams. Each request counts the references of the pool + +, and hash calculates the selected connections. When the reference count exceeds the number of logical connections, it needs to be expanded. If the maximum free connections do not reach the maximum number of active connections, then expand the capacity according to double. If the maximum number of active connections is reached, the capacity will be expanded. We will go further according to the value of the Reuse parameter: if true, continue to use the connection in the pool, even if you are using the logical connection of the physical connection, count the atoms on the reference when you close the connection-- if it is false, create a new connection, and you need to Close the connection when you close it.

If the reference count of the pool is 0, the capacity reduction operation will be triggered, and the connection will be maintained to the maximum number of idle connections.

Overtime to keep alive

Based on the Keepalived feature of GRPC, we do not need to implement the survival mechanism ourselves, nor do we need to pay attention to whether the connection in the connection pool is valid, because even if it fails, GRPC will reconnect automatically, but the time-consuming will increase slightly. In other words, it is considered that the connection in the connection pool is always valid except for reasons such as the server is always in down state.

Tips

Due to the use of hash remainder, the concurrent Stream on each GRPC may exceed the MaxConcurrentStreams.

The configuration of connection pool is also different in different scenarios. You need to find out the best parameter configuration of connection pool according to your own scenario.

Thank you for your reading, the above is the content of "how to achieve GRPC connection pool", after the study of this article, I believe you have a deeper understanding of how to achieve GRPC connection pool, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.