Shulou(Shulou.com)06/03 Report--

What is object storage

1. Object storage, that is, key storage, uploads and downloads data to the storage service through its interface instructions, that is, simple GET, PUT, DEL and other extended instructions.

2. All data in object storage is considered to be an object. Therefore, any data can be stored in the object storage server, such as pictures, video, audio, etc.

2. The composition of Ceph object storage

Ceph object storage is mainly implemented through RGW, so what is RGW?

1. RGW is the full name of Rados Gateway.

2. RGW is a Ceph object storage gateway, which is used to provide storage interface and RESTful API access interface to client applications.

3. RGW can deploy multiple servers for high availability and load balancing. The topology is shown below:

III. Deploy RGW services

1. Install the ceph-radosgw package into ceph-e through ceph-deploy on ceph-a

[root@ceph-a] # ceph-deploy install-- rgw ceph-e

2. Synchronize the configuration file and key file to ceph-e

[root@ceph-a ceph] # ceph-deploy admin ceph-e

3. Start a RGW service

[root@ceph-a ceph] # ceph-deploy rgw create ceph-e

4. Log in to ceph-e to verify whether the service is started.

[root@ceph-e yum.repos.d] # ps-ef | grep radosgw

5. View rgw status

[root@ceph-e yum.repos.d] # systemctl status ceph-radosgw\ *

6. Check whether the port is listening

[root@ceph-e yum.repos.d] # netstat-tpnl | grep 7480

Through the above 6 steps, our RGW service can be run successfully.

7. RGW has built a web server called civetweb, which is similar to apache/nginx. Since it is a web service, it is more convenient to use port 80. For convenience of application, you can change the default port 7480 to port 80.

Modify the ceph.conf configuration file of the ceph-e node by appending the following:

[root@ceph-e ceph] # vim ceph.conf [client.rgw.ceph-e] host = ceph-ergw_frontends = "civetweb port=80"

Save exit and restart the service

[root@ceph-e ceph] # systemctl restart ceph-radosgw.target

8. Check whether the port has been modified successfully

[root@ceph-e ceph] # netstat-tpnl

9. Verification

Open the browser and type http://192.168.20.148, as shown below

Or enter curl http://192.168.20.148\ r on the command line to send the enter key instruction, as shown below

Access using third-party software

1. Use Amazon S3 client to access

①, log in to ceph-e (RGW) to create users

[root@ceph-e radosgw] # radosgw-admin user create-uid= "radosgw"-display-name= "First User"

User: radosgwaccess_key: N6ALEK0KS0ISYCIM5JBGsecret_key: qK9hrpX2uwna4elPP1VsuErmAHBw3So40fE2K4yM

②, authorized users, allow radosgw to read and write users information:

[root@ceph-e radosgw] # radosgw-admin caps add-uid=radosgw-caps= "users=*"

Allow radosgw to read and write all usage information:

[root@ceph-e radosgw] # radosgw-admin caps add-uid=radosgw-caps= "usage=read,write"

③, create sub-users, which are used for later swift client access

[root@ceph-e radosgw] # radosgw-admin subuser create-uid=radosgw-subuser=radosgw:swift-access=full

④, create key

[root@ceph-e radosgw] # radosgw-admin key create-subuser=radosgw:swift-key-type=swift-gen-secret

User: radosgw:swiftsecret_key: NhTk42HlMcAnv2GBXqatqMPeI20eSLod30AOJLwl

⑤, install S3 client software in ceph-f

First, let's find out if the yum source has the rpm package of S3.

[root@ceph-f ~] # yum list | grep S3

Found that there is a rpm package for S3, so you can install it directly.

[root@ceph-f ~] # yum-y install s3cmd.noarch

⑥, configure S3

[root@ceph-f] # s3cmd-- configure

Note:

Access Key: N6ALEK0KS0ISYCIM5JBGSecret Key: qK9hrpX2uwna4elPP1VsuErmAHBw3So40fE2K4yMDefault Region [US]: # do not modify it here, or you will get an error S3 Endpoint [s3.amazonaws.com]: 192.168.20.148DNS-style bucket+hostname:port template for accessing a bucket [% (bucket) s.s3.amazonaws.com]:% (bucket) s.192.168.20.148 bucket # is equivalent to the creation folder of Baidu network disk. Here is the fixed format Path to GPG program [/ usr/bin/gpg]: # keep the default Use HTTPS protocol [Yes]: no # write no here, because there is no https port HTTP Proxy server name: # there is no need to write here, because there is no proxy Test access with supplied credentials? [Y/n] ySave settings? [y/N] y

⑦, client-side testing, viewing content

[root@ceph-f ~] # s3cmd ls

Empty, because just like Baidu cloud disk, it has just applied and has not been created, so it is empty.

⑧, creating a bucket for storing data (similar to a directory for storing data) requires xxx_yyy format

[root@ceph-f ~] # s3cmd mb s3://cephdir

Found that the wrong report, look at the picture above, found that the error is because our writing format is not correct, rewrite according to the correct format, do it again

[root@ceph-f ~] # s3cmd mb s3://ceph_dir

As pictured above, it was a success.

⑨, view client content again

[root@ceph-f ~] # s3cmd ls

The test script for S3 is provided here

[root@ceph-f ~] # yum-y install python-boto [root@ceph-f ~] # vim s3test.pyimport boto.s3.connectionaccess_key = 'N6ALEK0KS0ISYCIM5JBG'secret_key =' qK9hrpX2uwna4elPP1VsuErmAHBw3So40fE2K4yM'conn = boto.connect_s3 (aws_access_key_id=access_key, aws_secret_access_key=secret_key, host='ceph-e', port=80, is_secure=False, calling_format=boto.s3.connection.OrdinaryCallingFormat () ) bucket = conn.create_bucket ('xxx_yyy') for bucket in conn.get_all_buckets (): print "{name} {created}" .format (name=bucket.name, created=bucket.creation_date,)

Replace your access_key, secret_key, hostname and port when using it

2. Swift interface test

①, the following packages need to be installed

[root@ceph-f ~] # yum install python-setuptools [root@ceph-f ~] # easy_install pip [root@ceph-f ~] # pip install-- upgrade setuptools [root@ceph-f ~] # pip install-upgrade python-swiftclient

②, command line access

Swift-A http://192.168.20.148/auth/v1.0-U radosgw:swift-K NhTk42HlMcAnv2GBXqatqMPeI20eSLod30AOJLwl list

Swift-A http://192.168.20.148/auth/v1.0-U radosgw:swift-K NhTk42HlMcAnv2GBXqatqMPeI20eSLod30AOJLwl stat-v

The test script for swift is provided here:

[root@ceph-f ~] # vim swift.pyimport swiftclientuser = 'radosgw:swift'key =' NhTk42HlMcAnv2GBXqatqMPeI20eSLod30AOJLwl'conn = swiftclient.Connection (user=user, key=key, authurl=' http://192.168.20.148/auth/v1.0',) for container in conn.get_account () [1]: print container ['name']

Please replace your access_key, secret_key and authurl when using them.

At this point, the Ceph object storage is over. If you have anything you don't understand, please refer to the official website.

For the implementation of Ceph cluster, please refer to another blog post: https://blog.51cto.com/4746316/2329558

For the application of Ceph block devices, please refer to another blog post: https://blog.51cto.com/4746316/2330070

For the application of CephFS file system, please refer to another blog post: https://blog.51cto.com/4746316/2330186

V. Summary

Ceph object storage is not stable at present, and the official documents are relatively old. Here we will only give a general explanation, but not do any in-depth research.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.