Shulou(Shulou.com)05/31 Report--

This article introduces you how to choose the best way to manage Helm Chart, the content is very detailed, interested friends can refer to, hope to be helpful to you.

Like it or not, you have to admit that Helm is a unique tool for managing Kubernetes applications, and you can even use it in different ways.

In the course of using Helm, we have noticed several problems that keep popping up:

Where do you put your Helm chart?

Do you use app files to save them or chart repositories?

How do you divide Helm chart?

Do you use a shared chart or maintain a chart for each service?

I am trying to solve these problems through my past experience in various startups, but I have also learned from the practices of large companies.

Here are a few of the methods I would like to outline:

Use a chart repository to store a large shared chart

Use a chart repository to store many service-specific chart

Use service-specific chart, which is stored in the same warehouse as the service itself

Then, I'll introduce the factors that should be taken into account when deciding on these options, such as dependency differences and team structure.

Option1: maintain a large shared chart in a chart repository

In one of our projects, we started with a large chart for deploying multiple services. It is stored in ChartMuseum and maintained by the person responsible for deploying the infrastructure.

If your services are similar in nature, sharing chart can save you a lot of trouble. Here we use the situation described by Helm maintainer Josh Dolitsky on KubeCon 2019:

I have recently been in charge of a project that includes nine microservices. I realized that they were almost all the same HTTP snooping service. So I decided to just build a helm chart to deploy nine different services, with different configurations for each service-- just set a new docker tag for a specific service.

In this case, it makes sense to store Helm chart in chart repositories such as ChartMuseum, because only values need to be stored in the repositories of these specific services.

Option2: maintain several service-specific chart in a chart repository

The advantage of service-specific chart is that you can change one service without worrying about breaking another. But they can lead to repetitive work-if you want to update the generic configuration, you have to make the same changes in each chart.

Whether you need to keep them in a chart repository is another question. If these chart are service-specific, there is no strong architectural argument for storing them together. Of course, if you have a dedicated person or team to maintain all the chart, it is usually easier to store multiple service-specific chart together.

For example, a DevOps engineer I work with maintains 15 different microservices chart in a central chart repository. It is much easier for him to update all chart in the same location than to submit pull requests to 15 different repositories. Developers certainly know how to update chart, but dealing with resource-related settings is obviously more attractive to them.

Option3: maintain service-specific chart in the same repository as the service itself

For micro-service-based applications, service-specific chart is a good choice. When you keep each chart in the same repository as the service code, it is better to use a service-specific chart.

If you store Helm chart in a service repository, you can more easily deploy services on a continuous basis independent of other projects. And you can commit chart updates (such as adding new variables) with changes to the application logic to make it easier to identify and restore major changes.

However, the advantage of this option depends on the number of microservices you maintain. If your number of microservices is moving into double digits, the advantages of this option are less obvious and more hindering. This is especially true if you are dealing with very homogeneous services such as Josh Dolisky.

Factors to consider when deciding on options

In general, there are two aspects to consider:

Dependencies and reproducibility: how different are the dependencies of each service? How risky is a change to one service to disrupt another service? How do you reproduce specific development conditions?

Team structure: are you in charge of a small autonomous team for each service? Do you have any developers who know DevOps? What is the popularity of DevOps culture in your team?

Dependencies and reproducibility

If you maintain your chart and application separately, their versions will be different from each other. If you encounter problems with deployment and need to reproduce the conditions that caused the problem, you need to determine: a) the version of the service; and b) the chart version used to deploy it. You may want to take a shortcut and use the "latest" chart to test the service x.x.x, but this is not a good idea because you will never be able to reproduce the exact conditions that caused the problem.

So, what if you often need to change the version of chart? Should we test these changes together?

Consider the scenario where many developers need to create branch versions of the same shared chart:

Developers (Edeltraud and Eberhardt in the figure) work in different branches and want to test their changes and diagram changes in the development environment-- so they also need a branch chart. Meanwhile, the DevOps engineer updates some common components in his branch of shared chart.

If no one updates their chart changes to each branch, it is possible to break another service deployment.

Not long ago, we happened to have this problem. The Chart maintainer updates the shared chart with a new condition block. This statement checks whether a new variable "foo" is set to "enabled". However, the variable "foo" has not yet been defined in the value files for all services. For services that lack this variable, the deployment is interrupted. Unfortunately, the default rollback behavior was not defined in chart at that time.

If the chart and the code are in the same repository and can be tested in the same branch, testing against these issues will be easier.

Even if it seems to be overkill at first, we will do so. We work with services that have few dependencies. For each service, Helm chart deploys only one main container with a specific Docker tag. The name of the chart and the docker tag are passed in as variables. However, we avoided using a shared chart and chose to place a separate chart in each service repository.

This is mainly because we have only dealt with four services. But our developers also prefer to control all configurations that affect CI/CD. However, this is not always the case, so now is a good time to study another dimension.

Team structure

The problem with Chart maintenance also depends on who manages the deployment process.

Here is another article recommended by Helm maintainer Matt Farina, in which he addresses the topic that Helm is trying to address complexity.

He clarified three main roles that must deal with the complexity of Kubernetes. For clarity, I will explain its content and describe the role as follows:

App developer-this role focuses on building services, adding features, and fixing bug

The role of Deployer-- is responsible for bringing applications to the world. Ideally, there is a good automated program that can deploy the application for them, but they know how it works and can modify it as needed.

Systems engineer-this role is responsible for maintaining the Kubernetes environment for deployer deployment. They are experts in managing computer resources and can minimize downtime for any service.

You can find positions in the company that match the content of the first and third roles, while the role of Deployer is a bit vague, and the content of this role is often taken over by the other two roles-which affects how you manage your Helm chart.

DevOps of startups in the early stages

As mentioned earlier, our business is to provide operation and maintenance support for start-ups, which often need to expand rapidly. We have seen a lot of "unconventional" settings and division of labor. In the early stages, App developers may be responsible for a variety of things, and some may even help with system administrator tasks, such as setting up printers or configuring office networks. They will try their best to understand what the other two characters are responsible for, because no one can help them (until we are involved).

Once they want to know about Helm, most application developers will put their chart in the easiest place to deal with-that is, the same repo they maintain.

DevOps in large Enterprises

You may work in a larger, more structured team

In this case, you may have your own DevOps engineer or even the entire DevOps department. And this person or team often feels responsible for the role of "Deployer". Most likely, they will prefer a more centralized approach, such as storing all chart in chart repositories such as ChartMuseum. Even less willing to let application developers participate too much in Helm chart (often for good reasons).

For example, I recently watched a classic technical lecture called "Building Helm chart from scratch," which was delivered by Amy Chen, a VMWare systems engineer. In her opening remarks, she said: in terms of infrastructure, your main goal is to be ready for failures without trust-in that sense, just like I don't want to trust my APP developers, and I don't need to trust my APP developers.

This is understandable. You don't want app developers to mess up settings, such as CPU and memory limits, or pod to interrupt the budget. But the whole concept of "DevOps culture" has evolved specifically to improve the sometimes estranged relationship between infrastructure maintainers and developers.

Practice DveOps culture

Atlassian (owner of JIRA and Trello) has published a team Handbook that defines the DevOps culture:

DevOps culture is about a common understanding between developers and operators and sharing responsibility for the software they build. This means increased transparency, communication and collaboration, as well as cooperation between development, IT/ operations and "operations".

If it is actually applied to Helm chart maintenance and general infrastructure configuration, most of the responsibility will be in the hands of application developers. They will also assume the role of "Deployer" and change the configuration in the warehouse they own.

Systems engineers can still centralize the settings they specifically maintain. For example, some teams also maintain a central infrastructure repo that holds common resources such as Terraform configuration or Helm files that are needed to start a new project (for example, to set up ingress controller and cert manager). Helm 3 also supports so-called "library chart", which can only be deployed as part of another chart. This makes it easier to distinguish between common and service-specific change responsibilities.

Even when the chart is stored in the service warehouse, the systems engineer can still act as the gatekeeper for important changes. For example, you can use the GitHub CODEOWNERS file to ensure that any changes made by the systems engineer in the chart directory in your repo will be added as reviewers.

If the system engineer needs to proactively make changes that have nothing to do with application development, you can instruct the developer to make changes for them and explain why these changes are necessary. The following pictures may reflect this situation:

Developers can learn more about infrastructure and how these changes affect their services.

Rule of thumb

If there is a simple rule of thumb, it is: learn about option 3 first. Try to maintain a Helm chart for each service in the service repository. Or at least consider the hybrid approach I described earlier.

If you have dozens of services that are very similar, sharing chart is a better choice. Just remember, you have to maintain it in a central repo. But this increases the risk of accidental coupling, which can break a service deployment. Increased risk means that you need to be more careful when deploying, which in turn means that you will deploy less frequently.

Even if you have service-specific chart, you may need to store it centrally because you don't have enough people or expertise to manage these chart in a distributed way. Or maybe your team needs to draw a clear line of responsibility between "Deployer" and "application developer".

Whatever you decide to do, I hope I have made it clear what you need to consider when making the final decision. It's not easy to be a Deployer, especially if it's not your daily job.

On how to choose the best way to manage Helm Chart for you to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.