Shulou(Shulou.com)05/31 Report--

This article is to share with you about OSS Select using SQL statements to select the contents of OSS files, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article. Let's take a look at it.

Object storage OSS (Object Storage Service) has the characteristics of magnanimity, reliability, security, high performance and low cost. OSS provides standard, low-frequency, archive types, covering a variety of data storage requirements from hot to cold. The size of a single file ranges from 1 byte to 48.8TB, and there is no limit to the number of files that can be stored. OSS has become the infrastructure of Internet and enterprise data applications.

In general, the way to get object storage data is to get the entire object, or to get the data according to the specified byte range. Today, we launch OSS Select, which directly uses simple SQL statements to select what you need from OSS files.

OSS Select introduces the use of SQL to select the contents of OSS files

OSS Select (in public trial) allows developers to directly use SQL statements to select the desired content from the OSS file.

Using OSS Select to get only the query results required by the application and supporting concurrent sharding queries will greatly improve the performance of the program, usually by 400%.

Public test instructions:

File format: unencrypted CSV format or delimited UTF8 text files are supported during public testing, refer to RFC4180

Standard and low frequency Object are supported during public testing.

RangeQuery is supported (Use Header Name is not supported in RangeQuery mode during public trial)

Free of charge during the OSS Select public test

In the future, Ali Cloud EMR, DataLakeAnalytics, MaxCompute, HybridDB and so on will support OSS Select one after another.

Use the example (python) #-*-coding: utf-8-*-import osimport oss2def select_call_back (consumed_bytes, total_bytes = None): print ('Consumed Bytes:' + str (consumed_bytes) +'\ n') # first initialize AccessKeyId, AccessKeySecret, Endpoint and other information. # get it through environment variables, or replace things such as "" with a real AccessKeyId. # # taking Hangzhou region as an example, Endpoint can be: # http://oss-cn-hangzhou.aliyuncs.com# https://oss-cn-hangzhou.aliyuncs.com# is accessed via HTTP and HTTPS protocols, respectively. Access_key_id = os.getenv ('OSS_TEST_ACCESS_KEY_ID','') access_key_secret = os.getenv ('OSS_TEST_ACCESS_KEY_SECRET','') bucket_name = os.getenv ('OSS_TEST_BUCKET','') endpoint = os.getenv ('OSS_TEST_ENDPOINT',') # confirm that the above parameters are entered correctly for param in (access_key_id, access_key_secret, bucket_name, endpoint): assert'

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.