Shulou(Shulou.com)06/01 Report--

This article mainly explains the exception analysis of redis distributed Jedis type conversion. The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn the exception analysis of redis distributed Jedis type conversion.

1 Type conversion exception scenario

When we use Jedis, we often encounter type conversion exceptions, such as:

Multithreaded environment

Jedis is thread unsafe. If there are multiple threads using the same Jedis, there will be type conversion exceptions and there are a lot of wrong explanations on the Internet. Let's take a case to reproduce this problem, which is easy to understand.

Single threaded environment

Type conversion exceptions can occur even in the case of a single thread, so here is a case study

2 Jedis type conversion exception case 2.1 case introduction

The case is the notes for using Jedis returnResource from here.

The code is as follows:

Public static void main (String [] args) throws Exception {Jedis jedis = new Jedis ("192.168.126.131", 6379); System.out.println ("get name=" + jedis.get ("name")); System.out.println ("Make SocketTimeoutException"); System.in.read (); / waiting for manufacturing SocketTimeoutException try {System.out.println (jedis.get ("name")) } catch (Exception e) {e.printStackTrace ();} System.out.println ("Recover from SocketTimeoutException"); Thread.sleep (50000); / / continue to sleep for a period of time until the network is fully restored boolean isMember = jedis.sismember ("urls", "baidu"); System.out.println ("isMember" + isMember); jedis.close ();}

And contains two commands to block and remove network communication

Block network communication

Sudo iptables-An INPUT-p tcp-- dport 6379-j DROP

Unblock the network

Sudo iptables-F

Description of the case run process:

1 create Jedis, send get command, start the connection with redis, and get the response data after the connection is successful

2 program blocks in System.in.read (), waiting for input, at this time we need to block the network connection, execute the above blocking network command

3 enter arbitrary data, so that the program is no longer blocked, go on, execute the get command, at this time, due to the network is not connected, resulting in SocketTimeoutException exception

4 print out the exception, continue to go down, sleep 50s, at this time we need to unblock the network and execute the above corresponding command

After 550s, the sismember method of jedis will be executed, and a type conversion exception will occur.

2.2 introduction to Jedis principle

There is a Socket inside the Jedis that connects to the redis server. When the Jedis object is created, the connection is not established, but the connection is checked first when the command is executed, and if not, the connection is established.

Once the Socket connection is established, it gets the OutputStream of Socket, wraps it with RedisOutputStream, gets the InputStream of Socket, and wraps it with RedisInputStream. RedisOutputStream contains an array of byte buf [] inside.

That is to say, when jedis writes a command to OutputStream, it will first write to the above buf array, and then when it is read, it will flush the above data, write it to the OutputStream of Socket, and call flush. Take the get method of Jedis as an example

Public String get (final String key) {checkIsInMulti (); client.sendCommand (Protocol.Command.GET, key); return client.getBulkReply ();}

The client.sendCommand method writes the data to the buf inside the RedisOutputStream. The client.getBulkReply method first executes flush, that is, the data from the buf is written to the OutputStream of the Socket, and the flush of the OutputStream of the Socket is called.

2.3 causes of type conversion exceptions

Many people on the Internet say that the abnormal type conversion of the above scenarios is due to:

After a SocketTimeoutException exception occurs, the last command remains in the buf of RedisOutputStream without cleaning up, resulting in the execution of other commands along with the previous commands.

After looking at the source code of RedisOutputStream, it is true that buf does not take the initiative to remove the original data, but every time it is directly overwritten, marked with a count pointer, but this will not cause the impact mentioned above, RedisOutputStream is OK.

First of all, we need to understand what the SocketTimeoutException exception is: after the data of the above Jedis is sent, the Socket of the above Jedis will read the data, that is, read the data in the InputStream of Socket, and for a certain blocking time. If the redis server delays returning the data, once the SO_TIMEOUT (that is, the read timeout of Socket) is exceeded, the client will throw a SocketTimeoutException exception.

There are many reasons for this exception:

Network flash break (TCP retransmission): the above case scenario is that the network is disconnected and the packet fails to be sent, and the TCP retransmission occurs.

The network is not down, but the transmission is slow, or the redis server is slow to process.

All of the above reasons will cause the client read timeout. Once the timeout occurs, our Jedis program throws an exception and continues to go down. if it executes other commands at this time, it will still read the server-side response, and the response read at this time is the response of the last request, so it will cause a type conversion exception. If it is consistent with the type of the last request, it will be even more frightening, and the error will be deeply covered up.

3 solution to the exception of Jedis type conversion

The above problem is: we do not treat this SocketTimeoutException exception correctly, that is, once a SocketTimeoutException exception occurs, we have to discard the Jedis. So for Jedis in a single-threaded environment, once this exception occurs, we need to re-new a new Jedis to use.

When an exception occurs in the internal execution of the Jedis, such as an SocketTimeoutException exception, it marks a boolean broken=true, which means that the connection has been abandoned.

The important pit is here. We usually use JedisPool to deal with the use of Jedis in a multithreaded environment. The general usage is as follows:

Jedis jedis = null;// get resources from pool try {jedis = pool.getResource (); jedis.set ("K1", "v1");} catch (Exception e) {e.printStackTrace ();} finally {if (jedis! = null) {pool.returnResource (jedis); / / do not forget to "return" resources to the connection pool. }}

For JedisPool, we use the returnResource method to release the Jedis back to the pool, but this returnResource ignores the above boolean broken attribute and directly puts the abandoned connection of a tag back into the pool, which is bound to go wrong the next time someone fetches it.

Therefore, in the case of JedisPool, the solution is as follows:

1 catch SocketTimeoutException exception in the above catch, call the returnBrokenResource method of pool to release Jedis (this method marks the Jedis instance as offline and cannot be obtained by others), but this is not recommended, other exceptions should be considered, and so on.

2 the other is to directly call the close method of Jedis. In the latest version 2.9.0 (other versions are not verified), the close method processes the above boolean broken tag and marks the returnResource as obsolete, as follows

Public void close () {if (dataSource! = null) {if (client.isBroken ()) {this.dataSource.returnBrokenResource (this);} else {this.dataSource.returnResource (this);}} else {client.close ();}}

The above this.dataSource can be understood as JedisPool. That is, once it is broken, the returnBrokenResource method of pool is called, otherwise the returnResource method of pool is called.

So in the end, it should be written as follows:

Jedis jedis = null;// to get resources from pool try {jedis = pool.getResource (); jedis.set ("K1", "v1");} finally {if (jedis! = null) {jedis.close ();}} 4 problems

You can think of two problems:

Why does 1:jedis expose such a dangerous API to users (that is, users are required to consciously close, unconsciously bear the consequences)

If we are in the development framework to be used, then we should try to avoid this kind of API design, close automatically hidden in the framework to avoid misuse by users, while reducing the complexity of the code, even if the final writing is very ugly, to complete a set function, to pay attention to too many places, this part can be packaged at the bottom of the framework, only to give users a set method.

Question 2: mismatch between request and response

How to deal with this mismatch when synchronous and asynchronous respectively?

Synchronous communication: at design time, a request must be sent and a response must be read to match in this way. However, in some cases, the read response has a certain timeout, and once the timeout occurs, a SocketTimeoutException exception is thrown to end the read, and the response may arrive later, which will cause a mismatch. To avoid this, the Socket must be discarded, so if the client is designed to communicate synchronously, once it encounters such an exception, it needs to be abandoned and the connection reestablished.

Asynchronous communication: a request id is generally assigned to each request at design time. After processing the request, the server returns the request id to the client. The client matches the response corresponding to that request according to the returned request id, so the matching mismatch problem mentioned above will not occur. Asynchronous communication usually performs the read operation only when there is data to read, which can reduce the SocketTimeoutException problems caused by network congestion or other reasons in synchronous communication. The cost of the benefits of asynchronous communication is more complex than synchronous communication.

So if we are in the design, we need to consider such a problem to avoid creating a big hole.

Thank you for reading, the above is the content of "exception analysis of redis distributed Jedis type conversion". After the study of this article, I believe you have a deeper understanding of the problem of exception analysis of redis distributed Jedis type conversion, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.