Shulou(Shulou.com)05/31 Report--

How to understand the design and implementation of remote mirror replication function in Harbor new version, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more small partners who want to solve this problem find a simpler and easier way.

The lack of good tools for container mirror replication and publishing has been a major pain point in actual development and operation. The open source Harbor Registry provides powerful mirror replication/synchronization capabilities and has become a killer feature loved by many users.

In the recently released version, Harbor added policy-based Docker image replication, which can synchronize images between different data centers and different operating environments, and provides a friendly management interface, which greatly simplifies image management in actual operation and maintenance. There have been cases where users have deployed remote image bidirectional replication. The editor will give a detailed introduction to the implementation principle of this function.

Harbor mirror replication management interface

About Us

In terms of functional design, Harbor is still centered on "projects," by configuring "replication policies" for projects, indicating which projects need to be replicated and mirrored. The administrator specifies the target instance in the replication policy, the "destination" for replication, and sets its address and username password to use when connecting. When the replication policy is activated, all mirrors under the source project will be replicated to the target instance. In addition, when mirrors under the source project are added or deleted (push or delete), as long as the policy is still activated, the changes in the mirrors will be synchronized to the target instance, as shown in the following figure:

In a larger container cluster, multiple Registry servers are often required to do Load Balancer, and the master-slave publishing mode can be adopted. Mirrors only need to be published once and can be pushed to multiple Registry instances. At the same time, it also supports dual-master replication and hierarchical multi-level mirror publishing, as shown in the following figure:

design and implementation

Replicating mirroring between different Registry instances is a common requirement, and it used to be common to replicate objects by copying mirrored data, such as periodically synchronizing mirrored data in a file system via rsync, or, in the case of deployments on IaaS services, by configuring the IaaS storage service tier, often using different tools depending on the storage used by the registry. For Harbor, however, we want to reduce this dependency and increase flexibility, for example, a user may have a development registry that uses the file system as storage and wants to synchronize mirrors to a remote distribution registry based on S3 storage. With this in mind, we chose to download and transfer the image by calling registry's own API, thus making it independent of the underlying storage.

On the control side, we introduced a new component, Job Service, to manage mirror replication tasks. When replication is carried out in units of projects, a series of jobs will be generated in units of mirrors, which are scheduled and managed by Job Service. Job Service updates the status of each task to the database during task execution, so that users can view it through UI. The general structure is shown in the figure below:

The following describes the implementation of Job Service. From the outside, it also receives request scheduling and executes tasks through REST API. There are two main problems. First, when a large number of replication requests are received, current limiting is required to avoid consuming too many IO resources. Second, the replication policy may change during task execution, such as failure, which requires a mechanism to intervene on running tasks from the outside.

We implement the producer-consumer model through task queues, dispatchers and worker pools. Using Go's built-in channel, each task will be placed into the channel through the scheduler, and the dispatcher will get the task through the channel. At the same time, the worker will be placed into another channel after the work is finished. The dispatcher will pair with the worker through this channel, so the idle worker will get the task ID through the dispatcher and execute the task. This makes it easy to control the number of concurrency through the number of workers in the worker pool:

For another problem, each worker is an abstract state machine that registers handlers for different states to perform specific tasks, and the state machine can be intervened to cancel tasks halfway, or to discard tasks in error state or hand them over to the scheduler for retry when task execution is abnormal. In addition, since the state of the state machine is customizable, it is easy to expand and adjust. For an abstract task, its state transitions are shown below:

For specific tasks of remote synchronous mirroring, the Running state will be further subdivided into multiple sub-states, as shown in the following figure:

First, download the manifest of the corresponding tag from the source Harbor instance, analyze the blobs it contains, check whether it already exists in the target instance for each blob, and synchronize the blob if it does not exist. Finally, check if the manifest already exists in the target instance, and if it doesn't, upload the manifest. Checking the existence of blobs can effectively reduce unnecessary network traffic; since the upload of manifest may trigger the synchronization of mirrors, checking the existence of manifest can avoid entering a state of endless synchronization when multiple synchronized harbors form a loop. Repeat the above process for each tag in the same mirror to complete the synchronization of the entire mirror.

About how to understand the new version of Harbor remote mirror copy function design and implementation of the answer to the problem shared here, I hope the above content can be of some help to everyone, if you still have a lot of doubts not solved, you can pay attention to the industry information channel to learn more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.