Shulou(Shulou.com)06/03 Report--

This article mainly explains "to create a mature GitOps assembly line needs to prepare what work", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in-depth, together to study and learn "to create a mature GitOps assembly line need to prepare what work" bar!

GitOps is a recent hot trend in the area of software delivery, following and extending trends such as DevOps, infrastructure as code, and CI/CD.

The advantages of GitOps can be simply summarized as follows:

Audit changes freely

Continuous integration and delivery

Better control of change management

However, the reality is that building a GitOps pipeline is not easy, it involves many decisions, large and small, and these decisions will bring a lot of trouble to the implementation. We call these decisions the "GitOps architecture", which can lead to many challenges in the implementation process.

The good news is that with a certain amount of planning and experience, the pain of transitioning to the GitOps delivery model can be greatly reduced.

In this article, I will explain some of these challenges through a company story. The company has grown from a fragmented start-up to a standard multinational company using GitOps. Although this accelerated growth is rare, it does reflect the experience of many teams in large organizations, from proof-of-concept to minimum feasible products (minimum viable product), to mature systems.

A simple start

If you're just getting started, the easiest thing to do is to create a single Git repo with all the required code in it. These may include:

Application code

Dockerfile, used to build application images

Some CI/CD pipeline code (such as GitLab CI/CD or GitHub)

Actions)

Terraform to configure the resources required to run the application

In addition, all changes are made directly to the master, so the changes can take effect directly.

The main advantage of this approach is that you have a single reference point and that all the code is tightly integrated. If all your developers are fully trusted and speed is everything, this approach will continue to work.

Unfortunately, as your business continues to grow, the disadvantages of this approach will soon become apparent.

First, as more and more code is added to the code base, the expansion of the code base will confuse engineers as they encounter more conflicts between changes that must be resolved. If there is a significant increase in team membership, the subsequent reassignment or merger can lead to further confusion.

Second, you will encounter difficulties if you need to control the operation of the assembly line separately. Sometimes you just want to quickly test changes to the code rather than deploy for full end-to-end delivery. This approach creates more and more problems that need to be addressed in terms of integrity, and as these changes proceed, they may affect the work of others.

Third, as you grow, you may want the boundaries of responsibility between the engineer and the team to be more refined. This can be achieved through a single repo, and an repo is usually a clearer and cleaner boundary.

Separate Repository

As the business grows, the assembly line becomes more and more crowded, and merge begins to become painful. In addition, your team needs to be professional, dividing different areas of responsibility into different members.

So you need to isolate the Repo. At this point, you first have to face a large number of decisions, such as how far repo should be separated? Do you need to create a separate repo for the application code? Does it look reasonable? And then put the stuff built by Docker into it? Then such a separation actually doesn't make any sense.

What about the Terraform code for all the teams? Should it be in a new repo? This sounds reasonable, but: the newly created central "platform" team wants to control access to the core IAM (identity and access management) rule definitions in AWS, and the team RDS configuration code is included, and the development team needs to adjust it regularly.

So you decided to split the Terraform into two repo: a "platform" repo and an "application-specific" repo. This presents another challenge, as you now need to separate the state files of Terraform. This is not an insurmountable problem, but it is not the fast feature delivery that you are used to, so the product manager will have to explain why feature requests take longer than before.

Unfortunately, there are no established best practices or patterns for these GitOps decisions.

The problem of separation does not stop there. Previously, coordination between components built within a pipeline was negligible (because all required components coexisted), but now you have to coordinate the flow of information between repo. For example, when building a new Docker image, this may require triggering the deployment in the centralized platform repo and passing the new image name as part of the trigger.

Again, these are not insurmountable problems, but in the early stages of building an GitOps pipeline, these challenges are easier to implement. Later, as steps, policies, and processes become more mature, it takes more time to make changes.

Distributed vs centralized

Your business is growing, and you are building more and more applications and services. It is increasingly clear that you need some kind of structural consistency in how to build and deploy applications. The central platform team needs to try to implement these standards. But you may encounter opposition from the development team, who believe that they were given more autonomy and control in centralized IT than they did before DevOps and GitOps.

If the above sounds familiar to you, it may be because there are some similarities between monomer and microservice debates in GitOps and application architecture. As you can see in these arguments, as the system matures and expands in size and scope, tensions between distributed and centralized IT will occur more and more frequently.

In a way, your GitOps process is like any other distributed system, and if you don't design it well, one part of it can go wrong and cause unpredictable problems.

Surroundings

As you decide to separate repo, you realize that you need a consistent approach to managing different deployment environments. It is no longer possible to go online directly, because the QA team is required to test the changes before going live.

Now you need to specify different Docker tags for your application images in test and QA environments, and you may also want to enable instance sizes or replicas of different sizes in different environments. How do you manage the configuration of these different environments in the source code? A more straightforward approach is to establish a separate Git repository (e.g. super-app-dev,super-app-qa,super-app-live) for each environment.

There are "distinct" benefits of separating repo, as we saw when we divided the Terraform code above. However, few people will eventually like this solution because most teams do not have Git knowledge and relevant expertise to migrate changes between different repo. To make matters more complicated, there is bound to be a lot of duplicate code between repo, and there may be a lot of drift over time.

If you want to keep things in a single repo, you have at least three options:

Each environment has a directory

Every environment has a branch.

Each environment has a label.

Synchronization step selection

If you rely heavily on YAML generation tools or templates, you can consider another approach. For example, Kustomize strongly encourages directory-based environment separation. If you are using the original YAML, the branching or tagging method will be more suitable for you.

Runtime environment granularity

However, in your runtime environment, you can choose what level of separation you want. At the cluster level, if you are using Kubernetes, you can choose from the following situations:

A cluster manages all

One cluster per environment

One cluster per team

In extreme cases, you can put all your environments into one cluster. In general, however, there is at least one separate cluster for production in most organizations.

Once you have figured out your clustering strategy, at the namespace level, you still have a choice:

Each environment has a namespace

Each application / service has a namespace

Each engineer has a namespace

Each build has a namespace

Platform teams usually start with "dev", "test", and "prod" namespaces, and then realize that they want to divide the team's work more finely.

You can also mix and match these options-for example, provide each engineer with a "desk testing" namespace, as well as each team's namespace.

Total knot

Here we just give a brief introduction to the decision-making areas required for a mature GitOps process. If your business really grows into that multinational company, you can also consider requirements such as RBAC/IAM.

Usually, the launch of GitOps gives the impression that it is just an investment and may not end up with much satisfactory output. But before GitOps, teams often experience confusion and delays, because no one is sure what state anything should be. All of these will lead to secondary costs, because auditors will conduct spot checks, and interruptions caused by unexpected and unrecorded changes occupy a lot of employees' attention, which is a high cost.

However, as your GitOps process matures and the benefits multiply, it will solve many of the problems that existed before. But more often, you are under pressure to demonstrate the advantages of the GitOps process more quickly.

The biggest challenge for GitOps right now is that there are no established patterns or best practices to guide your choices. GitOps consultants usually just guide the team to find the solution that works best for them, and guide the team in certain directions based on experience.

But what I have observed is that early choices that were abandoned because they seemed "too complex" tend to regret it later. This does not mean that you should jump directly to the point where you create a namespace for each build and each team has an Kubernetes cluster for two reasons:

Whenever you add complexity to the GitOps architecture, you end up increasing the cost and time to deliver a viable GitOps solution

You may never need this setting.

Until we accept the feasible standards in this field, the right GitOps architecture is always an art, not a science.

Thank you for your reading, the above is "to create a mature GitOps assembly line need to prepare for what work" content, after the study of this article, I believe you need to create a mature GitOps assembly line to prepare for this problem has a deeper understanding, the specific use of the situation also needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.