Shulou(Shulou.com)05/31 Report--

This article mainly introduces how to achieve Rocketmq pull message paging number test, has a certain reference value, interested friends can refer to, I hope you have a lot of gains after reading this article, let Xiaobian take you to understand.

One machine deployment

1. Machine composition

7 machines, all with 16 gigabytes of memory

Each server has 4 CPUs, 2 cores

2. Operating environment configuration

3, brush plate mode

Each machine master machine adopts asynchronous brushing mode

II. Performance evaluation

1. Purpose of evaluation

Test pull consumption mode, maximum number of messages pulled in a single batch.

2. Evaluation indicators

Maximum number of messages to pull in batch msgExtSize

Custom Configuration Max. Number of Pulled Items maxNums

3. Evaluation logic

(1) Send 10000 messages first, wait for all messages to be sent, and then start the consumer end to consume messages.

(2) Configure maxNums values as 1, 3, 5, 8, 16, 20, 24, 27, 31, 32, 33, 34, 35, 36, 54, 80, 100 respectively, and then compare the number of msgExtSize items in each pull message.

(3) Record the maximum number of messages pulled in batches each time, so as to test the maximum number of messages pulled in batches.

(4) After finding out the maximum number of batch pull messages in step 3, set consecutive numbers before and after this value to further verify this value.

4. Evaluation process

(1) The first group: open 20 threads, each thread sends 3000 pieces of data, and sends a total of 60W messages to the topic name "pullTest"(the more messages, the better, the consumer consumption rate is fast, so the more data sent, the better, so as to see the effect).

The message was sent as follows:

Set maxNums=1 consumption record, output msgExtSize=1

Set maxNums=3 consumption record, output msgExtSize=2

Set maxNums=5 consumption record, output msgExtSize=4

Set maxNums=8 consumption record, output msgExtSize=7

Set maxNums=16 consumption record, output msgExtSize=15

Set maxNums=20 consumption record, output msgExtSize=19

Set maxNums=24 consumption record, output msgExtSize=23

Set maxNums=27 consumption record, output msgExtSize=26

Set maxNums=32 consumption record, output msgExtSize=31

Set maxNums=36 consumption record, output msgExtSize=32

Set maxNums=54 consumption record, output msgExtSize=32

Set maxNums=80 consumption record, output msgExtSize=32

Set maxNums=100 consumption record, output msgExtSize=32

By analyzing the test data of the above conditions, it can be seen that the maximum number of batch pull messages is 32.

(2) The second group: numbers before and after the maximum number of pull messages, fine-grained retesting.

Set maxNums=33 consumption record, output msgExtSize=32

Set maxNums=34 consumption record, output msgExtSize=32

Set maxNums=35 consumption record, output msgExtSize=32

Set maxNums=31 consumption record, output msgExtSize=30

Further analysis of the test data shows that the maximum number of messages pulled in batch is 32.

The results of the analysis are as follows:

Custom maximum pull number

(Expected) Maximum number of messages pulled in batch

(Actual) 113254871615201924232726313032313323432353236325432803210032

II Evaluation Results

1. The following test conclusions are based on scenarios where messages are stored in memory (rather than on disk).

In the Pull consumption scenario, let maxA = the maximum number of user-defined pull messages and maxB= the maximum number of actual batch pull messages, then the following conclusions are drawn:

If maxA = 1, maxB = 1

If maxA = 32, maxB = 32

2. Refer to RocketMQ configuration file. If the message is stored on disk, the maximum number of actual batch pull messages is 8 (not the expected value of 32). However, this test scenario is not easy to reproduce. It has not been tested yet and will be further reproduced in the future.

Thank you for reading this article carefully. I hope that Xiaobian's "How to Realize Rocketmq Pull Message Pagination Number Test" will be helpful to everyone. At the same time, I hope that everyone will support you a lot and pay attention to the industry information channel. More relevant knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.