Shulou(Shulou.com)06/02 Report--

Recently, I need to test the rgw of ceph in my work, so I learn while testing.

First of all, the tool uses cosbench, an open source tool of intel, which is also the mainstream object storage testing tool in the industry.

1. Installation and startup of cosbench

Download the latest cosbench package

Wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c4/0.4.2.c4.zip

Decompression

Unzip 0.4.2.c4.zip

Install related toolkits

Yum install java-1.7.0-openjdk nmap-ncat

Add execution permissions to all sh scripts

Chmod + x * .sh

Cancel http Agent

Unset http_proxy

If it is a controller node, execute start-all.sh

If it is a driver node, execute start-driver.sh

In my environment, one controller+3 driver,controller and one driver share a node.

Modify conf/controller.conf

Modify the conf/driver.conf of each node

Controller executes start-all.sh

Driver node executes start-driver.sh

Http://controllerip:19088/controller/

Log in to the controller web page

All three driver nodes are ok

2. There are various configuration files in the conf directory

3. Ceph's object storage supports S3 and swift interfaces. This article first discusses S3.

Test S3, which can be modified according to s3-config-sample.xml

-accesskey secretkey endpoiont to obtain authentication information from its own ceph environment

-init phase, mainly to create bucket. The above configuration is to create 32 bucket prefixed with s3testqwer.

-prepare phase, mainly to create object in bucket. The above is to create 64k object in each bucket, with 64k configurable range. A pre-created database can be used for the following read test.

-main, execution phase, ratio is the percentage of execution, workers is the number of concurrency, and runtime is the execution time

-cleanup, clean up object

-dispose, clean up bucket

The work in the same workstage is executed concurrently. For example, if you have multiple driver, you will be assigned to multiple driver to execute. The worker in each work is concurrent, worker is also loaded to multiple driver, and multiple workstage is executed sequentially.

Import the written workstage script

And then submit

Execute init-prepare-main-cleanup-dispose in turn

Execution result

The description of each category in the figure above:

Op-type-Action type

Op-Count-Total number of operations

Total Byte-Count-byte

Avg-ResTime-response time, which is the sum of data transfer time and processing time

Avg-ProceTime-time of read and write operation

Throughput-Throughput, operations per second

Bandwidth-bandwidth

Succ-Ratio-success rate of the operation

The icon can see that the fluctuation of the whole data is stable as a whole.

There are 33 mistakes.

You can find the wrong log in the log directory of cosbench

Check the log of the corresponding mission

This is an error with java startup:

1. Stop-all.sh stops the cosbench process (controller, drivers)

2. Edit the cosbench-start.sh file, find the java startup command line parameter, and add "- Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true" to turn off the MD5 check function of S3.

3. Start start-all.sh

Execute workload again after modification, and no longer report an error. You can adjust the script according to your own configuration. For example, if you only want to test write and retain data, you can not write cleanup and dispose. Of course, you can also customize work.

Attach a xml for the swift interface test

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.