Shulou(Shulou.com)06/01 Report--

Today, the editor will bring you an article about pipeline in redis. The editor thinks it is very practical, so I will share it for you as a reference. Let's follow the editor and have a look.

First, the background of pipeline:

There are four procedures for redis to execute a command: send a command, queue a command, execute a command, and return a result.

This process is called Round trip time (RTT for short, round trip time). Mget mset effectively saves RTT, but most commands (such as hgetall, without mhgetall) do not support batch operation and need to consume N times of RTT. In this case, pipeline is needed to solve this problem.

Second, the performance of pepeline

1. Pipeline is not used to execute N commands

2. Use pipeline to execute N commands

3. Performance comparison between the two.

Summary: this is a set of statistical data. The execution speed using Pipeline is faster than that of one by one. In particular, the greater the network latency between the client and the server, the more obvious the performance and physical fitness.

The test code is posted below to analyze the performance difference between the two:

@ Test public void pipeCompare () {Jedis redis = new Jedis ("192.168.1.111", 6379); redis.auth ("12345678"); / / requirepass password of redis.conf corresponding to authorization password Map data = new HashMap (); redis.select (8); / / use the 8th library redis.flushDB () / / clear all data in the 8th library / / hmset long start = System.currentTimeMillis (); / / Direct hmset for (int I = 0; I < 10000; iTunes +) {data.clear (); / / clear map data.put ("k _" + I, "v _" + I) Redis.hmset ("key_" + I, data); / / cycle 10000 pieces of data to insert redis} long end = System.currentTimeMillis (); System.out.println ("Co-insert: [" + redis.dbSize () + "].. "); System.out.println (" 1, it takes time to set values in batches without using PIPE "+ (end-start) / 1000 +" seconds. "); redis.select (8); redis.flushDB (); / use pipeline hmset Pipeline pipe = redis.pipelined (); start = System.currentTimeMillis () / / for (int I = 0; I < 10000; iTunes +) {data.clear (); data.put ("k" + I," v "+ I); pipe.hmset (" key_ "+ I, data) / / encapsulate the value into the PIPE object, which is not executed at this time, and still stays on the client side} pipe.sync (); / / send the encapsulated PIPE to redis end = System.currentTimeMillis () once; System.out.println ("PIPE insert: [" + redis.dbSize () + "].. "); System.out.println (" 2, it takes time to set values in bulk using PIPE "+ (end-start) / 1000 +" seconds. ") /-/ / hmget Set keys = redis.keys ("key_*") / / query all the result keys of the above values / / directly use Jedis hgetall start = System.currentTimeMillis (); Map result = new HashMap () For (String key: keys) {/ / according to the above setting result, keys has a total of 10000 and loops result.put (key, redis.hgetAll (key) 10000 times); / / use the redis object to take the value according to the key value, and put the result into the result object} end = System.currentTimeMillis () System.out.println ("total value: [" + redis.dbSize () + "]. "); System.out.println (" 3, it takes time to batch value without using PIPE "+ (end-start) / 1000 +" seconds. "); / / use pipeline hgetall result.clear (); start = System.currentTimeMillis (); for (String key: keys) {pipe.hgetAll (key) / / use PIPE to encapsulate the key that requires a value, but still stay on the client side and do not really execute the query request} pipe.sync (); / / submit it to redis for query end = System.currentTimeMillis (); System.out.println ("PIPE shared value: [" + redis.dbSize () + "].. ); System.out.println ("4, it takes time to batch value using PIPE" + (end-start) / 1000 + "seconds."); redis.disconnect ();}

3. Comparison between native batch commands (mset, mget) and Pipeline

1. The original batch command is atomic and pipeline is non-atomic.

(atomicity concept: a transaction is an indivisible minimum unit of work that either succeeds or fails. Atomic operation means that one of your business logic must be inseparable. Either succeed or fail in dealing with one thing, the atom is inseparable)

2. Native batch commands: one command has more than one key, but pipeline supports multiple commands (transactions exist) and is non-atomic.

3. The native batch command is implemented by the server, while pipeline needs to be completed by both the server and the client.

IV. Correct use of Pipeline

The number of commands assembled with pipeline cannot be too large, otherwise the amount of data is too large, increasing the waiting time of the client, and may also cause network congestion. You can split a large number of commands into multiple small pipeline commands to complete.

1. How to use pipeline in Jedis

As we all know, redis provides mset and mget methods, but does not provide mdel methods. If you want to implement it, you can use pipeline to implement it.

2. The steps for using pipeline in Jedis:

Get the jedis object (usually from the connection pool)

Get the pipeline object of the jedis object

Add instruction

Execute instruction

Test class methods:

@ Test public void testCommond () {/ / utility class initialization JedisUtils jedis = new JedisUtils ("192.168.1.111", 6379, "12345678"); for (int I = 0; I < 100; iTunes +) {/ / set jedis.set ("n" + I, String.valueOf (I)) } System.out.println ("keys from redis return =" + jedis.keys ("*"));} / / batch delete @ Test public void testPipelineMdel () {/ / utility class initialization JedisUtils jedis = new JedisUtils ("192.168.1.111", 6379, "12345678") using pipeline List keys = new ArrayList (); for (int I = 0; I < 100; iTunes +) {keys.add ("n" + I);} jedis.mdel (keys); System.out.println ("after mdel the redis return -" + jedis.keys ("*"));}

Mdel method under JedisUtils:

/ * Delete multiple strings key and release connection * * @ param keys* * @ return successfully returned value failed to return null * / public boolean mdel (List keys) {Jedis jedis = null; boolean flag = false Try {jedis = pool.getResource (); / / borrow the Jedis object Pipeline pipe = jedis.pipelined () from the connection; / / get the pipeline object for (String key:keys) {pipe.del (key) of the jedis object / / put multiple key in the pipe delete instruction} pipe.sync (); / / execute the command, and complete the remote call of the pipeline object flag = true;} catch (Exception e) {pool.returnBrokenResource (jedis) E.printStackTrace ();} finally {returnResource (pool, jedis);} return flag;}

Use pipeline to submit all actions and return the execution result:

@ Test public void testPipelineSyncAll () {/ / tool class initialization Jedis jedis = new Jedis ("192.168.1.111", 6379); jedis.auth ("12345678"); / / get pipeline object Pipeline pipe = jedis.pipelined (); pipe.multi () Pipe.set ("name", "james"); / / adjust pipe.incr ("age"); / / add pipe.get ("name"); pipe.discard () / / merge and submit different types of operation commands, and return the operation as list List list = pipe.syncAndReturnAll (); for (Object obj: list) {/ / print out the operation result System.out.println (obj) } / / disconnect and release the resource jedis.disconnect ();}

5. Redis transaction

Pipeline is a combination of multiple commands, and to ensure its atomicity, redis provides simple transactions.

1. Simple transactions of redis

A set of commands that need to be executed together are placed between the multi and exec commands, where multi represents the start of the transaction and exec represents the end of the transaction.

2. Stop the transaction discard

3. Incorrect command and incorrect syntax, so that the transaction cannot end normally.

4. running error, correct syntax, but wrong type, the transaction can end normally

5. Watch command:

After using watch, multi fails and transaction fails.

The mechanism of WATCH is that when the transaction EXEC command is executed, the Redis will check the key of the WATCH, and the EXEC will be executed only if the key of the WATCH has not changed since the beginning of the WATCH. If the key of WATCH has changed from the WATCH command to the EXEC command, the EXEC command returns a failure.

After reading the above, do you have a general understanding of the pipeline in redis? If you want to know more about the content of the article, welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.