Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to solve the errors caused by boto support aws4". The explanation content in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian slowly and deeply to study and learn "how to solve the errors caused by boto support aws4" together!

Environment Introduction

software version

root@demo:/home/demouser# ceph -vceph version 10.2.6 (656b5b63ed7c43bd014bcafd81b001959d5f089f)boto Version:2.46.1

rgw configuration

[client.radosgw.cn-zone1] rgw dns name = ceph.work rgw frontends = fastcgi socket_port=9000 socket_host=127.0.0.1 host = demo keyring = /etc/ceph/ceph.client.radosgw.keyring rgw socket path = /home/ceph/var/run/ceph-client.radosgw.cn-zone1.sock log file = /home/ceph/log/radosgw.cn-zone1.log rgw print continue = false rgw content length compat = trueboto some pits for region support

boto use case

from boto.s3.connection import S3Connectionimport botoimport osos.environ <$'S3_USE_SIGV4 ']='True' #Start support for aws4 endpoint ='ceph.work 'bucket_name ='test1'access_key =''secret_key ='' conn = boto.connect_s3( aws_access_key_id=access_key, aws_secret_access_key=secret_key, host=endpoint, is_secure=False, calling_format=boto.s3.connection.SubdomainCallingFormat(), validate_certs=True,)bucket = conn.get_all_buckets()print bucket

abnormal information

Traceback (most recent call last):..... File "/Users/demouser/lwc/lib/python2.7/site-packages/boto/auth.py", line 690, in determine_region_name return region_nameUnboundLocalError: local variable 'region_name' referenced before assignment

boto logical processing of region_name

def split_host_parts(self, host): return host.split('. ') def determine_region_name(self, host): # S3's different format(s) of representing region/service from the # rest of AWS makes this hurt too. # # Possible domain formats: # - s3.amazonaws.com (Classic) # - s3-us-west-2.amazonaws.com (Specific region) # - bukkit.s3.amazonaws.com (Vhosted Classic) # - bukkit.s3-ap-northeast-1.amazonaws.com (Vhosted specific region) # - s3.cn-north-1.amazonaws.com.cn - (Beijing region) # - bukkit.s3.cn-north-1.amazonaws.com.cn - (Vhosted Beijing region) parts = self.split_host_parts(host) if self.region_name is not None: region_name = self.region_name else: # Classic URLs - s3-us-west-2.amazonaws.com if len(parts) == 3: region_name = self.clean_region_name(parts[0]) # Special-case for Classic. if region_name == 's3': region_name = 'us-east-1' #Here's a pit, as we'll see else: # Iterate over the parts in reverse order. for offset, part in enumerate(reversed(parts)): part = part.lower() # Look for the first thing starting with 's3'. # Until there's a ``.s3`` TLD, we should be OK. :P if part == 's3': # If it's by itself, the region is the previous part. region_name = parts[-offset] # Unless it's Vhosted classic if region_name == 'amazonaws': region_name = 'us-east-1' break elif part.startswith('s3-'): region_name = self.clean_region_name(part) break return region_name

Reading the boto code, I found that in fact, the region_name field is to press host ". "Split, take the previous part to process, if the host is ceph.work, then the split"ceph"of course does not correspond, so to make boto support AWS4, your host must have 3 fields, such as" s3.ceph.work ". If you take the selfie approach, just go adjust the boto code as follows

from boto.s3.connection import S3Connectionimport botoimport osos.environ ['S3_USE_SIGV4 '] = 'True'endpoint s3.ceph.work' #Add a field bucket_name ='test1 'access_key =''secret_key ='' conn = boto.connect_s3( aws_access_key_id=access_key, aws_secret_access_key=secret_key, host=endpoint, is_secure=False, calling_format=boto.s3.connection.SubdomainCallingFormat(), validate_certs=True,)bucket = conn.get_all_buckets()print bucket

Then you will see the following exception

Traceback (most recent call last):... File "/Users/Diluga/lwc/lib/python2.7/site-packages/boto/s3/connection.py", line 444, in get_all_buckets response.status, response.reason, body)boto.exception.S3ResponseError: S3ResponseError: 400 Bad RequestInvalidBucketNames3tx000000000000000000001-0058d4d9ad-85cc-default85cc-default-default

This 400 error means that you submitted the data in the wrong posture.

From the packet capture point of view, this us-east-1 is the region_name you submitted. Here we also buried a pit. If the first field of our host is "s3", then region_name is hardcoded as "us-east-1". If it is "s3-abc" or something else, region_name becomes our custom field. Fortunately, ceph allows you to fill in region_name in the request header by default, otherwise it will be really tragic.

The correct posture here should be to adjust the rgw configuration in ceph.conf as follows.

Scheme 1

[client.radosgw.cn-zone1] rgw dns name = s3.ceph.work #Add a new field based on the original host, the region_name submitted by boto will be "us-east-1" rgw frontends = fastcgi socket_port=9000 socket_host=127.0.0.1 host = demo keyring = /etc/ceph/ceph.client.radosgw.keyring rgw socket path = /home/ceph/var/run/ceph-client.radosgw.cn-zone1.sock log file = /home/ceph/log/radosgw.cn-zone1.log rgw print continue = false rgw content length compat = true

Scheme 2

[client.radosgw.cn-zone1] rgw dns name = s3-abc.ceph.work #Add a new field based on the original host, the region_name submitted by boto will be "abc" rgw frontends = fastcgi socket_port=9000 socket_host=127.0.0.1 host = demo keyring = /etc/ceph/ceph.client.radosgw.keyring rgw socket path = /home/ceph/var/run/ceph-client.radosgw.cn-zone1.sock log file = /home/ceph/log/radosgw.cn-zone1.log rgw print continue = false rgw content length compat = true Thank you for reading, the above is "boto support aws4 caused by errors how to solve" content, after the study of this article, I believe that we support aws4 caused by errors how to solve this problem has a deeper understanding, the specific use of the situation also needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.