Shulou(Shulou.com)05/31 Report--

Editor to share with you what to think before the container on Ceph, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Hardware resource requirements for services

First of all, you must understand the software and hardware requirements of MON, OSD, MDS, MGR and RGW services in Ceph, know the scale of Ceph you plan, and whether the resources currently allocated to the corresponding containers are appropriate. Otherwise, when you need to make various hardware resource adjustments and have to restart the container later, your service availability may be greatly reduced. In short, in a word, hardware resources are in place in one step, don't mess around. Don't let OOM become normal!

Software smooth upgrade

Do not think that you can easily deal with software version upgrades on the container. Ceph can theoretically achieve mixed deployment of small versions of software, but once you find that there are holes in a certain version, you will find a lot of problems when you have to adjust to the same version, upgrade Mon or OSD first? What if a large number of services can't come up? What if there is a slow request? If you naively think that after the container, through a few simple container commands to achieve a smooth upgrade of the ceph version, or even cross-major version upgrade, then you are lucky, cross-version upgrade is rarely without a problem, the most important thing is that the upgrade operation is basically non-return, dare to take production data to upgrade are "real warriors". And the upgrade process of a variety of strange problems, senior operators may not be able to hold live, and finally have to turn to developers to help deal with, but do you know how difficult it is to recruit an engineer who understands Ceph source code development?

Stateless service?

Mon and OSD need to store data to the local file system and LevelDB (RocksDB), and all have certain performance requirements for storage devices (especially OSD). In order to maintain data consistency, Ceph introduces epoch mechanism internally, which means that every time you restart the service, the version will change. These basic data and the corresponding services are strongly bound. In order to achieve the container service drift to any physical node, First of all, you have to solve the container volume data sharing, so you have to introduce a shared file system, and dig a hole for yourself. Since stateless services are not available, roles such as MON and OSD should consider whether to complicate simple problems before becoming containerized. Finally, the only role in Ceph that can implement stateless service is RGW, and the load balancing implemented by RGW combined with containerization is very suitable for the scenario. If you want to achieve stateless containerization, RGW is the only choice.

Network architecture

Each Ceph service process needs to be bound to a static IP (frequent changes to the IP will greatly increase the management cost of maintaining a unified configuration), and it is best not to deploy service nodes of a single ceph cluster across network segments (some holes will also be buried across network segments), so whether your container network supports these static configuration network requirements of Ceph needs to be carefully considered in advance. If your container network is three layers and two layers, layer by layer, layer by layer, the cost should not be underestimated. The most fatal thing is that any distributed system is strongly dependent on the clock. High network latency between nodes will inevitably lead to clock deviation, which will eventually affect cluster stability and data consistency.

Performance loss

OSD can use bare storage devices do not use the file system, in view of the poor performance of Ceph, it is absolutely wise to shorten the IO path as much as possible.

Operation and maintenance complexity log management

All kinds of strange faults in Ceph need to be located with the help of logs. It is best to see the fault site at the first time, but it is not so easy to view logs after containerization. If you really want to be containerized, you should do centralized log management with a set of similar ELK. (coredump is also a pit.)

Service monitoring

You may have written monitoring plug-ins for Ceph such as zabbix, but now switch to containers, is the original solution still applicable? Including the existing MGR dashboard, there is still a long way to go.

Hardware failure

This is what makes me complain the most. Originally, the failure of OSD disk can be solved with a few commands. Now, with the introduction of the container, the operation complexity of changing disk has increased a lot. Although script automation can be used to realize these things, higher skills are required for operation and maintenance personnel. The original technology stack for disk replacement is Linux+ceph, and now it is a Linux+ceph+ container, which virtually increases the threshold of recruitment. Of course, the salary will be higher, but in the end, the bosses will be unhappy, especially the layoffs in the economic crisis, you give the boss a good reason to kill you.

The above is all the content of the article "what are the thoughts before the container on Ceph"? thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.