Shulou(Shulou.com)05/31 Report--

This article will explain in detail the example analysis of the pipeline mechanism in Redis. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Introduction to Pipeline

The Redis client executes a command:

Send a command

Command queue

Execute a command

Return the result

Where sending commands and returning results can be called Round Trip Time (RTT, round trip time). Batch operation commands, such as mget, mset, etc., are provided in Redis, which effectively saves RTT. However, bulk operations are not supported for most commands.

For this reason, Redis provides a mechanism called Pipeline to assemble a set of Redis commands, transmit them to Redis through a RTT, and then transmit the execution results of these Redis commands sequentially to the client. Even if you execute the command n times with pipeline, the whole process requires only one RTT.

Test the performance of Pipeline

We use redis-benchmark to test the performance of Pipeline, which provides the option of-P, which means that n Redis requests are processed using a pipeline mechanism, with a default value of 1. The tests are as follows:

# execute 100000 get set requests without pipes [root@iz2zeaf3cg1099kiidi06mz ~] # redis-benchmark-t get,set-Q-n 100000SET: 55710.31 requests per secondGET: 54914.88 requests per second# the number of commands organized by each pipeline is 100 [root@iz2zeaf3cg1099kiidi06mz ~] # redis-benchmark-P 100-t get Set-Q-n 100000SET: 1020408.19 requests per secondGET: 1176470.62 requests per second# the number of commands organized by each pipeline is 10000 [root@iz2zeaf3cg1099kiidi06mz ~] # redis-benchmark-P 10000-t get,set-Q-n 100000SET: 321543.41 requests per secondGET: 241545.89 requests per second

As you can see from the above tests, the number of requests per second processed by Redis with pipeline is much greater than that without pipeline.

Of course, the number of commands organized by pipeline cannot be unrestrained each time, otherwise the amount of Pipeline data assembled at one time is too large, on the one hand, it will increase the waiting time of the client, on the other hand, it will cause some network congestion.

As can be seen from the above tests, if the number of commands organized by pipeline is 10000, but its corresponding QPS is less than the number of pipeline commands at a time. So it is not as many commands as possible to organize the Pipeline at a time. You can split the Pipeline containing a large number of commands into several smaller Pipeline.

Pipeline's note about RTT

There is a description on the official website:

It roughly means:

The Pipeline pipeline mechanism is not just a way to reduce RTT, it actually greatly improves the QPS of Redis. The reason is that, without using a plumbing mechanism, it is very cheap to service each command from the point of view of accessing data structures and generating responses. But from the perspective of the underlying socket, this is very expensive, which involves read () and write () system calls, switching from user mode to kernel state, which is expensive to switch context. In the case of using Pipeline, a single read () system call is usually used to read many commands, and then a single write () system call is used to pass multiple replies, thus improving the QPS

Comparison between batch commands and Pipeline

Batch commands are atomic and Pipeline is non-atomic

Batch commands are one command. Multiple key,Pipeline supports multiple commands.

Batch commands are implemented by the Redis server, while Pipeline needs to be implemented by both the server and the client

Use jedis to execute pipeline

Public class JedisUtils {private static final JedisUtils jedisutils = new JedisUtils (); public static JedisUtils getInstance () {return jedisutils;} public JedisPool getPool (String ip, Integer port) {JedisPoolConfig jedisPoolConfig = new JedisPoolConfig (); jedisPoolConfig.setMaxIdle (RedisConfig.MAX_IDLE); jedisPoolConfig.setMaxTotal (RedisConfig.MAX_ACTIVE); jedisPoolConfig.setMaxWaitMillis (RedisConfig.MAX_WAIT); jedisPoolConfig.setTestOnBorrow (true); jedisPoolConfig.setTestOnReturn (true); JedisPool pool = new JedisPool (jedisPoolConfig, ip, port,RedisConfig.TIMEOUT,RedisConfig.PASSWORD); return pool } public Jedis getJedis (String ip, Integer port) {Jedis jedis = null; int count = 0; while (jedis = = null & & count

< RedisConfig.RETRY_NUM) { try { jedis = getInstance().getPool(ip, port).getResource(); } catch (Exception e) { System.out.println("get redis failed"); } count++; } return jedis; } public void closeJedis(Jedis jedis) { if (jedis != null) { jedis.close(); } } public static void main(String[] args) throws InterruptedException { Jedis jedis = JedisUtils.getInstance().getJedis("127.0.0.1", 6379); Pipeline pipeline = jedis.pipelined(); pipeline.set("hello", "world"); pipeline.incr("counter"); System.out.println("还没执行命令"); Thread.sleep(100000); System.out.println("这里才开始执行"); pipeline.sync(); }} 在睡眠100s的时候查看 Redis，可以看到此时在pipeline中的命令并没有执行，命令都被放在一个队列中等待执行: 127.0.0.1:6379>

Get hello (nil) 127.0.0.1 get counter 6379 > nil)

After sleep, use pipeline.sync () to complete the call to the pipeline object.

127.0.0.1 get counter 6379 > get hello "world" 127.0.0.1

You must execute pipeline.sync () to finally execute the command, and of course you can use the pipeline.syncANdReturnAll callback mechanism to return the pipeline response command.

This is the end of this article on "example Analysis of Pipeline Mechanism in Redis". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.