In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-04-01 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)05/31 Report--
What is the initial plan for the mixed deployment of yarn and K8s? in order to solve this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.
Version
Yarn 2.6.0
K8s v1.15.2
Background
With the increasing popularity of K8s and the expandable and self-repairing advantages of K8s, K8s is bound to be the advantage of the Internet, but a large part of the company's resource management tool is still yarn, and yarn and K8s are bound to be parallel for a long time.
However, because the isolation type of yarn is still very weak compared to K8s, fortunately, yarn 2.6.0 also integrates LinuxContainerExecutor (the default is DefaultContainerExecutor), which means that yarn does not support cpu restrictions for a long time. LinuxContainerExecutor uses cgroup to limit cpu, while memory isolation uses container for isolation.
We know that for the mixed deployment of multiple resource management tools, the competition for resources must exist. Analyze that k8sforce k8s supports docker runtime by default, that is to say, when k8s supports docker, it uses docke for resource isolation, while docker uses liunx's namespace for isolation and cgroup for resource restriction. As opposed to yarn, you only need to modify the configuration file.
Specific implementation and configuration of k8s
For K8s, according to the official website of K8s, we changed the driver of docker cgroups to systemd. For the specific modification method, please see the official website:
# Set up the Docker daemoncat
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.