Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about how to understand object storage and CDN implementation analysis. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

Share a topic about the use of object storage and CDN. Generally speaking, in addition to the services provided by public cloud vendors, there are also some value-added services that adapt to different application scenarios. What does it mean?

Object storage essentially solves the problem of mass storage, but data not only needs to be stored reliably, but in most cases it has to be processed before it can be valuable, so public cloud vendors generally have data processing services, such as image processing, similar thumbnails, audio and video transcoding, video screenshots.

Secondly, there are special requirements for data access, such as uploading once and downloading multiple times, use CDN's service together, because the download traffic of CDN will be lower than the cost of downloading object storage service.

What is CDN? The full name Content Delivery Network is essentially an edge server deployed everywhere, providing the nearest data access experience and effectively reducing the pressure on the back end of the data server.

In terms of technical implementation, CDN is actually a caching proxy for the back-end server.

Use posture

Schematic diagram:

While public cloud vendors provide object storage services, they will also provide cdn services. Because these two services are the same, they are naturally convenient for collaborative operation.

Data upload:

The data is uploaded to the domain name of the object storage service (endpoint)

Data download:

Data can be downloaded from the domain name of the object storage service, or from the cdn domain name

After cdn is enabled for object storage, the client requests data using the domain name of the cdn service. If the data is not in the cdn service, the cdn service will automatically use the object storage Bucket domain name to pull the data to the cdn cache. All the pressure after this is unloaded on the cdn service. So, from the above description, we realize once again that cdn is actually a simple caching proxy for the back-end storage service. If the business wants to use cdn, you need to specify the domain name of the cdn service, the domain name of the object storage, and configure the mapping relationship in the cdn service (this step is actually done by the public cloud vendor for us). The object storage domain name is used to upload data, and the cdn domain name is used to read data.

Cdn back-to-origin usage scheme

When a user's request fails, cdn needs to pull data from the back-end of the origin. So have you ever thought about what kind of scheme cdn uses to get the data correctly? Because S3 protocol has permission to verify, that is, digital signature, S3 v4 protocol will sign host in digital identity verification, that is, in general, cdn cannot be forwarded directly. In general, there are three solutions for the implementation of object Storage Service and cdn:

Presign authorizes public read

Presign

Presign is also called pre-signature. In this scheme, the client prepares the signature completely, and all the signature elements and verifications are in the query url. Cdn is really a pure forwarding agent in this scenario. However, there is a restriction on presign that you can only use S3 V2 signature, because the host will change when the cdn is pulled back to the origin, and the v4 signature will countersign the host, so the signature verification will not pass in this case.

Policy authorization scheme

This is the general approach. S3's Bucket supports fine-grained permission allocation, that is, Policy policy. It allows various operation permissions to be assigned to various specified objects.

On the public cloud, users apply for cdn service resources and mainly do several configurations (which can be done for you by the public cloud vendor):

Provide mapping relationship between CDN domain name and Bucket domain name business and authentication method between CDN

The public cloud vendor will also grant the Bucket of the business side read permission to the cdn service. In this way, when cdn misses and cdn pulls back to the origin, you can use your own account to request data from the object storage service (because when cdn is enabled, it has been authorized to cdn through policy, so you can read the data).

Public reading scheme

The Bucket of S3 can be configured for public reading, also known as anonymous access. Any client can get the data directly by curl, so cdn can also get the data. This is a special scheme, and we don't usually do this. It is conceivable that the application of this approach is limited, because data security cannot be guaranteed.

The above is the analysis of how to understand object storage and CDN implementation shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.