Shulou(Shulou.com)06/01 Report--

What is the Linux operating system for Kubernetes? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

You already know about Kubernetes (or are considering exploring some Kubernetes deployments). There are many good reasons to know it, as you may already know, Kubernetes is responsible for managing containers, scheduling workloads to clusters, handling scalability and redundancy, and automating scrolling (updates) and rollback. It is an infrastructure-independent system that uses declarative statements to describe the state that systems and applications should be in and drives managed elements to reach that state. This makes it easier to manage powerful and scalable systems. Of course, the "easy to manage" here has a learning curve, but it is well worth it to reap the benefits of modern container-based software development, that is, an infrastructure that provides scalability and infrastructure portability.

While Kubernetes does enable operational scalability and management of containers, it does not directly help you manage the infrastructure on which Kubernetes itself depends. Kubernetes itself is an application (or a set of apps) that needs to be run somewhere. Although you may have heard that Kubernetes is not an operating system, it still depends on the Linux or Windows system to be installed on the node. Kubernetes can run on cloud service providers such as AWS or GCE, or on virtualized platforms such as VMware, but all of this still requires the operating system to be installed first. (some, such as AWS EKS, do not need to manage the control plane node, but you still need to set up a Linux server for the work node.)

Operationally, the focus is on Kubernetes and the workload it runs, which is supposed to be the case, but this can lead to a common problem in Kubernetes deployment. Although Kubernetes patches and upgrades regularly, maintenance, updates, security, and operations of the underlying operating system are often forgotten or ignored, at least until the security audit. I often hear from SRE and system administrators about managing both Linux and Kubernetes, which leads to extra work. Just like the general Linux operating system, Kubernetes also needs to patch, update, protect and control user access, and so on. However, just because these tasks are done at the Kubernetes level does not mean that they can be ignored at the operating system level. However, choosing the appropriate underlying operating system distribution can greatly reduce the workload of maintaining the operating system and reduce the impact of not updating in time.

So, considering that you need to install Linux before running Kubernetes on it, which will involve the underlying operating system, which Linux distribution should you choose to run? There are many options, but they are usually divided into two types, namely container-optimized operating systems, or general-purpose operating systems.

General Linux operating system

These are normal types of Linux.

Most people are familiar with running a common type of Linux operating system, such as Ubuntu, Debian, CentOS, RHEL, or Fedora. This is one of the main advantages of running a general-purpose operating system in a Kubernetes cluster, and your system administrator will be familiar with how to install, update, and harden your Linux distribution. You can use the existing toolset to start the server, install the operating system, and configure it to a basic level of security. The existing patch management and security detection tools should work properly on these systems, even if you run Kubernetes on them.

However.

Using a common type of Linux system comes with the common Linux management overhead. This means that user account management, patch management, kernel updates, service firewalls, SSH security, disabling root login, disabling unused daemons, kernel tuning, etc., all need to be done and up-to-date. As mentioned earlier, most of these tasks can be accomplished using existing tools such as Ansible,Chef,Puppet, but updating the manifest or control file to fit the server configuration file to the Kubernetes master and worker nodes is not easy.

Another problem is the coordination of operating system changes with Kubernetes maintenance. Inconsistencies often occur so that the operating system remains intact after installation. Over time, Kubernetes will (hopefully) upgrade, but the underlying operating system may remain intact, slowly accumulating the burden of known CVE (common vulnerabilities and exposures) in various packages and installed kernels.

Ideally, you want the automation platform (such as Ansible or Puppet) to coordinate with Kubernetes so that the node's operating system can be upgraded without affecting Kubernetes operations. This means that the operating system needs:

Set the node to be unschedulable so that the new workload will not be scheduled to the node

Expel the node to move all running Pod to other nodes

Update and patch the node

Set the node to schedulable

Of course, the system needs to ensure that there are not too many nodes updating at the same time to ensure that the workload capacity of the cluster will not be adversely affected (and not too few nodes, lest the update speed of large clusters is slower than the release speed of patches and updates). You may want to coordinate operating system updates with Kubernetes updates to reduce restarts and interruptions, but you still need to support more critical operating system updates in a short period of time.

The biggest advantage of a common type of Linux operating system is how familiar the staff are with it. This means that they will be familiar with deployment and have troubleshooting technology. They can install and use common operating system tools such as tcpdump, strace, lsof, and so on. The configuration can be easily changed to correct errors and test alternatives (this is both a good and a bad thing!) The disadvantage is the need to maintain the overhead of system administration and the need to coordinate updates with Kubernetes infrastructure and operations.

Container-specific operating system

The National Institute of Standards and Technology (NIST) has a good summary of defining container-specific operating systems and lists some advantages.

The container dedicated host operating system is a minimalist operating system explicitly designed to run only containers, which disables all other services and features and uses read-only file systems and other hardening practices. When using the container-specific operating system, the attack surface is usually much smaller than the general-purpose type operating system, so there are fewer opportunities to attack and destroy the container-specific host operating system. To sum up, if possible, organizations should use container-specific host operating systems as much as possible. " Quoted from "NIST Special Publication 800190 Application Container Security Guide"

To sum up, it is obvious that the fewer software and packages the operating system runs, the smaller the attack surface and the fewer vulnerabilities. This makes the container-specific operating system significantly more secure from the start, even in the absence of frequent patches.

Container-specific operating systems can also use other security methods, such as making the root file system (preferably all file systems) read-only to mitigate the impact of any vulnerabilities.

Container-specific operating systems typically do not run (or do not support) package management. This reduces the chance that a conflict caused by installing or updating a package will cause the node or service to stop running. Due to the lack of management tools such as Chef and Puppet, the chances that incomplete operation will adversely affect the stability of the system are reduced. Instead, a complete operating system image with all updates and configurations applied is installed in an alternate boot mechanism and is started on the next reboot, or rolled back to a previously known good image. This means that the configuration of the node is fully known at any time, and any version can be restored from the version control system used.

Some container-specific operating systems are more like general Linux distributions. For example, VMware's PhotonOS installs fewer packages than ordinary Linux distributions, but still includes package manager, SSH access, and does not mount the file system as read-only. What is sometimes confusing is that the "cloud-optimized" version of the general Linux system is still the general Linux system, such as the "cloud mirror" released by Ubuntu, which is "customized by the Ubuntu engineering department and can run on the public cloud". However, these are still complete Linux distributions, with all the packages installed, but with an extra cloud-init package, which makes it easier to configure startup without human intervention.

CoreOS is the first widely adopted container-specific operating system and popularizes the idea of running all processes in containers to improve security and isolation. CoreOS cancels the package manager and uses reboot to one of the two read-only / usr partitions to ensure that the update is atomic and can be rolled back. But the project has been terminated since CoreOS was acquired by RedHat.

Current container-specific operating systems take the smallest posture (few packages are installed in the operating system), lock up (to some extent), run processes in the container (for better security, stability, and service isolation), and provide atomic updates (by booting to one bootable partition and updating another). Such examples are:

Google's Container-Optimized OS, which supports a read-only root file system, but allows SSH and runs only in GCP.

RancherOS, which runs SSH and does not use a read-only file system to protect the root partition

K3OS, also developed by Rancher, does not run a full Kubernetes distribution. Management is through kubectl, but SSH is supported.

AWS Bottlerocket is another operating system with an immutable root file system and support for SSH, that is, at least initially, it focuses on AWS workloads.

The exception is Talos, which is the most controversial container-specific operating system. Like other systems, the Talos operating system is minimized, has no package manager, uses only read-only file systems (except / var and / etc/kubernetes, and one or two special files that are temporarily writable (reset on restart), such as / etc/resolv.conf), and integrates with K8s through the upgrade controller.

However, the Talos operating system goes a step further than other systems to put forward the concept of immutable infrastructure, which cancels all SSH and console access and enables all OS access and management through the API driver. On the node running Kubernetes, everything you want to do has API calls, check all the containers, check network settings, and so on. But you can't do things you shouldn't do on the node, such as unmounting the file system. Talos also chose to completely rewrite the Linux Init system, which does only one thing, and that is to start Kubernetes.

No user-defined services can be managed (these should be managed through Kubernetes), which further improves security (no SSH, no console), reduces maintenance (no users, no patches), and reduces the impact of any CVE (because the file system is immutable and short-lived). You may not agree with the idea that giving up SSH access, restricting the movement of SRE, and forcing nodes to be completely immutable is desirable, but this is also a recent argument against immutable containers, which is worth exploring. Having an API-managed operating system is also very suitable for large-scale operation and management. If you need to check the log of a container on a node, a class of nodes, or all nodes, it is just the same API call with different parameters.

Summary

If you have adopted the view that container management is "cattle rather than pets" (that is, the production software infrastructure can be replaced at any time), that is, destroy the container and enable a new version when deploying an update or repair, then it makes sense to ensure that the same approach is applied to the infrastructure that supports containers. Using a container-like management model, destroying and reconfiguring nodes for updates rather than patches may require some training, but using a container-specific operating system helps to adopt this model, reducing administrative overhead and improving security. The container-specific operating system also helps improve operational stability, and system administrators or developers do not need to change the configuration to make it work, thus eliminating the possibility of human error or misconfiguration leading to the failure of the next upgrade.

Given that many enterprises are still in the early stages of the Kubernetes adoption life cycle, now is a good time to familiarize themselves with this next-generation operating system. By tightly combining the operating system with Kubernetes, the entire Kubernetes cluster can be treated as a single computer, reducing overhead and promoting enhanced security. This keeps people's attention focused on the workload and value provided by the computing infrastructure and is another step towards an API-driven data center.

After reading the above, have you mastered the method of the Linux operating system for Kubernetes? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.