Shulou(Shulou.com)06/03 Report--

In recent years, large enterprises and open source community continue to promote the development of open source technology in China. Today, China has become a paradise for open source technologies such as OpenStack & Ceph to shine.

The picture shows the salon around Ceph China.

Ceph domestic user ecology

As the most popular open source distributed storage project in the world, Ceph also has a very hot development in China, constantly beginning to integrate in different fields, different industries and customer systems. Typical applications in domestic first-tier Internet companies as well as operators, government, finance, radio and television, energy, games, live broadcast and other industries.

The current situation of Ceph in China has undergone decisive changes compared with previous years. With the use of more and more domestic users in various industries, it is sufficient to witness its stability and reliability. Ceph China user ecology has been formed, we can see China Mobile, Tencent, Ali, NetEase, Letv, Ctrip, Jinri Toutiao, China Telecom, ZTE, Hengfeng Bank, Ping an Technology, YY, bilibili, 360and so on. It is precisely because the use of many users not only verifies its stability and reliability, but also promotes the progress of Ceph, resulting in the emergence of many new things, such as SPDK, BlueStore, RDMA and other high-performance underlying technologies.

Domestic contribution of Ceph

Homag also talked about the contributors to the Ceph community in his previous article. What is very interesting is that the users of Ceph occupy a considerable contribution ranking, which to some extent reflects the current situation of Ceph. To be able to really control Ceph, you must go deep into the community and grow with it. Therefore, for an open source project that is not as mature as Linux, especially for a storage system, the degree of code contribution basically determines the understanding, risk control and use of Ceph. The closed-loop effects of the development, use of problems, iterations, fixes, upgrades, and testing processes within the community can greatly improve the participating companies' understanding of Ceph. Most companies that really use large-scale or Ceph-based products are involved or indirectly involved in the community, which is very similar to the early cases of Linux and OpenStack.

So which domestic companies are involved in the contribution of the community? Let me talk about it, such as: China Mobile, XSKY, ZTE, Chaochao, H3C, Ali, NetEase, Letv, United Stack, 99cloud and so on. I won't talk about it here (no ranking of the above companies). For details, please see Community mail list.

You may say that the participation of domestic enterprises to contribute to the Ceph community is far lower than that of the OpenStack community. I think there is nothing wrong with this, because storage is a very serious matter, and things at the control level are far from being compared with things at the data level. In addition, the threshold for storage is very high, so you can't play casually. The community demands high-quality code and refuses to pollute the cultural environment of open source technology. For example, the Ceph RGW:Lifecycle submitted by Letv and the DPDK, SPDK, RDMA, AsyncMessenger and so on submitted by XSKY partner.

Ceph Community: a continuous Innovation Environment

From the perspective of the ecological chain of traditional IT infrastructure, industry leaders at all levels have invested manpower and material resources for Ceph to continuously promote innovative operation, development and production environment.

As shown in the figure, RedHat, SUSE, Canonical and FreeBSD constitute the manufacturers of Ceph software distribution packages. Intel,Mellanox,AMD and Cisco promote their integration into the Ceph system at different hardware component levels. SanDisk,HDS and Fujitsu adopt Ceph integration on their own storage systems. CERN and Deutsche Telekom are the enterprise users with the most participation and feedback from the Ceph community, respectively. At the same time, in recent years, China Mobile, a domestic operator-level user, is also participating in the contribution of the Ceph community.

Ceph through its open community and plug-in code architecture to accommodate more and more underlying vendors to participate in it, whether Mellanox promotes Infiniband/RDMA, Seagate's Kinetic API, or Intel x86 architecture, ARM is actively involved, making use of its own advantages to continue to innovate and develop the Ceph software system.

For example, at the network level, Mellanox and XSKY provide a network solution based on RDMA, and Chelsio cooperates with XSKY to achieve a RDMA storage network based on iWARP.

Ceph storage engine

Ceph supports a variety of different storage engines on the storage backend, a bit like MySQL supports InnoDB,MyISAM and so on. There were FileStore,KeyValueStore, NewStore and MemStore before, but FileStore is the only one that is really used as an online system among these storage engines. However, due to historical problems, FileStore has the inherent problem of relying too much on the local file system, and there is a serious performance bottleneck under high utilization.

Therefore, starting from the Infernails version, Sage began the development of NewStore to achieve efficient back-end storage by combining the efficient management of key-value database and the spatial management of data in the local file system. However, due to the difficulty of the perfect combination of RocksDB and XFS, after several blows, Sage Weil decided to stab to the end and directly replace XFS with a simple user-mode file system for management. The project is named BlueStore.

BlueStore architecture diagram

In this new BlueStore design and implementation, RocksDB is expected to manage the entire metadata system, while the entire data space will use some space allocators optimized for Ceph to work. Currently, Ceph supports the conversion from offline FileStore to BlueStore.

Ceph backup disaster recovery

As a distributed storage system, Ceph seldom mentions whole cluster full backup. After all, as a huge multi-copy storage pool, it is difficult to invest in a larger backup system as a support. More often, Ceph itself achieves the reliability of traditional storage plus backup machines through replica and background verification plus parallel recovery.

However, Ceph still provides a variety of ways in different interface systems. In block storage, users often need to back up several important volumes, even if the Ceph cluster is completely unable to start in the worst case, it can ensure that important data will not be lost.

Ceph RBD remote disaster recovery is called Ceph RBD Mirroring and is announced to be available in the Ceph Jewel version. Prior to this, Ceph block storage solution (commonly known as RBD) could not be well replicated across regions (disaster recovery). It needs to be reminded here that because Ceph is highly consistent, a write operation is considered complete only when all copies have been written. This is why it is usually not a good idea to build a cluster that spans long distances, because the latency is generally high. The cluster must wait until all writes are complete, so the client may need a lot of time to confirm.

Therefore, a mechanism is needed to allow block devices to be replicated between clusters in different regions. In the current version of Jewel, the main purpose is to implement an one-to-one relationship between two daemons, which will be extended to 1-to-N in the future. In this way, in future versions of Jewel, you will be able to configure a cluster backup to multiple target backup clusters.

Enabling and disabling the RBD Mirror function can be applied to the entire Pool or a single p_w_picpath. If RBD Mirror is enabled at the resource pool level, every log-enabled mirror in the resource pool will be replicated by Mirroragent.

At present, Ceph focuses on the solution of the interface layer in the multi-cluster solution, rather than the implementation at the RADOS level. For example, RADOS Object Storage synchronizes data between clusters through Agent. Of course, in the Jewel version, RADOS Object Storage V2 already supports multi-read and multi-write mechanisms. Due to the weak semantics of object storage, the cross-site consistency of RADOS Object Storage is still the final. It defines the concepts of Zone,ZoneGroup and federated cluster. Each Zone can be understood as a part of a traditional Ceph cluster. ZoneGroup is a collection of multiple Zone, which is usually composed of Zone in different Ceph clusters, while only one Master ZoneGroup is allowed to write in the whole federated cluster. So logically, Master ZoneGroup can be made up of multiple Ceph clusters, and Slave ZoneGroup can also use other pools of these Ceph clusters as Zone. In this way, the cluster scheme of multi-location and multi-activity is completed.

The new version of Multi-Site follows the architecture of log resynchronization, the code is basically rewritten, and the cooperative framework of boost is introduced to make the configuration clearer. The data between multiple Zone in the same domain is in multi-master mode and can be written at the same time; metadata is written in master-slave mode by master Zone and synchronized to slave Zone to ensure metadata consistency. And bucket-level synchronization will be supported soon. Recently, the mainline has merged the plug-in framework of the synchronization model, and users can customize plug-ins to interface with elasticsearch to implement metadata indexing, or customize operations such as backup to the cloud.

The Future Prospect of Ceph

1.Ceph and Elasticsearch

Some time ago, I saw that Ceph supports Elasticsearch,RGW+Elasticsearch, which is a hot feature of Ceph object storage this year. I believe that Ceph will be more perfect in data search and analysis in the era of big data.

2.CephFS

CephFS claims that the production environment is ready in the community Jewel version. At present, the Active/Standby mode is relatively stable, and the Multi Active mode is not stable. In addition, there are still some problems when it is used on a large scale. I hope that the community will improve the CephFS-related features as soon as possible. From the user's point of view, there are still many people looking forward to using CephFS.

Ebay has tested J version of CephFS before. If you are interested, you can take a look at their test report on Slideshare (http://www.slideshare.net/XiaoxiChen3/cephfs-jewel-mds-performance-benchmark)

3.Ceph and New hardware

At the same time, with the rapid development of hardware today, it is also hoped that Ceph can run better and higher performance in the latest hardware 3D Xpoint of Intel, which can make Ceph more adapt to high-performance scenes.

4.Ceph personnel training

Finally, with regard to the training of Ceph talents, the three-year plan of the Ministry of Industry and Information Technology also announced, "it is necessary to establish an innovative talent training model, encourage colleges and universities to strengthen the construction of cloud computing-related disciplines, support enterprises and universities to jointly carry out on-the-job personnel training, and resume a number of talent training bases." With the landing of Ceph in China's operators, government, finance, radio and television, energy, games, live broadcast and other industries, resulting in a large number of job vacancies.

So now we need to establish a set of standard Ceph training system to alleviate the current shortage of Ceph talents, and at the same time carry out Ceph campus trips to colleges and universities in the Beijing-Tianjin-Hebei region as a pilot to radiate the whole country, the so-called open source, Ceph promotion starts from the campus, in response to the national call to promote college students' employment and actively participate in the contribution of the open source community.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.