Shulou(Shulou.com)05/31 Report--

This article is about how to see an excellent back-end service design and implementation from docker-hub and docker-registry. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

# # requirements

Recently, due to the needs of my work, I began to study the implementation and service construction of docker-registry. What is docker-registry? It is the image storage server of the popular container docker.

Or to put it this way, what docker does is package the entire application, operating system, and configuration into a static image that can be quickly turned on and off into a dynamic running container. But this ability doesn't make much sense for an individual. We need a place to save the image and share it with others with a url or link.

If it were you, how would you design it? Open a public FTP so that everyone can save the mirror and share it? It's a good idea, but. Docker image has such a setting, that is, an image is composed of multiple layers. If all files are transferred each time, it will cause a waste of time and traffic to the client, server and user. So.

Requirement 1: remote storage service, upload and download need to intelligently identify whether there is this layer on the opposite side. If the uuid of the two layers is the same, it will not be transmitted.

Simply uploading and downloading according to the name is not convenient for people's daily use, but also needs a web interface that allows people to log in, search, and distinguish between public and private images, and so on. This is a human requirement, not a client program requirement.

Requirement 2: web interface, support for search

The size of each mirror layer is about tens of megabytes to hundreds of megabytes. It is conceivable that when many people upload to one place, the storage capacity of a single server is absolutely unsustainable and requires a cluster that can scale horizontally, but the web interface cannot be separated, and client programs should not bother to find where to download.

Requirement 3: support for horizontally scalable cluster storage

The division of labor between docker-hub and docker-registry is as follows:

# # docker-hub

Responsible for saving information access in the set, about

User account

Validation code of the mirror image

Distinction between public and private image repositories

Docker-hub has several components:

Web UI

Meta-data metadata storage (notes, stars, common library list)

Access authentication

Token management

Dokcer-registry

There are several characteristics.

Store the family pedigree of the mirror image and the mirror layer

No user account data

Do not know the user's account and security

Entrust security and authentication to docker-hub, and use token to ensure delivery security

There is no need to reinvent the wheel, supporting multiple storage backends

No local database

What happened behind a docker pull or push

The index in these two figures is hub. You can see that each time the client accesses the index, decides which registry to upload or download the image file from, and then goes to the corresponding registry operation. From reading the source code, you can see that on registry, each mirrored layer is stored in tar.gz format.

Consideration of building docker image service on your own

Since it is a private server, we also need to consider user, security authentication, search and other issues. It can be said that docker developers considered these issues when designing image services, leaving Web to each private server developer to implement, and abstracting the back-end storage into interfaces to call. The source code for docker-registry is placed here. In order to ensure the normal use of subsequent development, I decided to read this source code first, but encountered a lot of problems:

Docker-registry is implemented in python. My knowledge of python is limited to simple scripts. I don't know much about the packages, modules and classes of python, so I learned python.

Docker-registry uses egg to package and release, gunicorn as the application server (similar to tomcat), flask as the mvc framework (similar to spring), followed by sqlalchemy as the search backend. These techniques require simple understanding and in-depth learning when needed.

Back-end storage, because the image is statically stored on the server in the way of tar.gz and does not require real-time read or write, so it is suitable for object storage rather than block storage, so the problem becomes to find or write a private storage driver. The officially supported drivers are Amazon AWS S3, ceph-s3, Google gcs, OpenStack swift,glance and so on. Qiniu in China has also written its own driver. Later, I need to write a driver in my environment.

Search, I haven't covered this yet, we'll see later.

The implementation of Web UI, now there are several projects on github, such as docker-registry-web, docker-registry-frontend, we'll see later.

Recently, I have been studying the implementation of PaaS with Docker. You are welcome to communicate with me if you have any ideas: -)

Finally, analyze the advantages of this architecture.

Decoupling docker-hub is a collection of web-ui, user authentication, and mirror metadata. In this respect, different organizations have different practices, so they need to be independent. Docker-registry is a part that can be reused by all organizations and is used solely for mirror storage services.

Do not repeat the wheel docker-registry to implement a set of object storage? No, because there are already many excellent implementations in the field of object storage. So docker-registry is a http interface service, only a layer of mirrored family pedigree is packaged on the object store, and the underlying layer supports multiple object storage.

Horizontal scalability in simple usage scenarios, docker-registry also supports local file system storage, which can be said to be the design of all-in-one and can be used right out of the box. When extending this scenario for large-scale enterprise applications, the relationship between docker-hub and docker-registry is 1registry, and registry itself is a stateless service that can be easily extended horizontally. This is also the cunning of the designer, who removes the stateful part and makes the storage the biggest state mechanism that can be placed on other object stores, so that there will be no performance problems in large-scale use scenarios. There will be no single point of problem. Any registry hanging up is tolerable and can be easily restored without side effects.

The above is how to look at the excellent back-end service design and implementation from docker-hub and docker-registry. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.