Shulou(Shulou.com)06/01 Report--

This article is to share with you about the use of KeyValueStore in Ceph. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

KeyValueStore is another storage engine supported by Ceph (the first is FileStore). It is proposed and implemented by myself on Add LevelDB support to ceph cluster backend store Design Summit in the Emporer version, and the docking with ObjectStore is realized in the Firely version. It has been merged into Ceph's Master.

KeyValueStore is a lightweight implementation compared to FileStore, and the goal is to use the capabilities provided by different Backend to serve different application scenarios of Ceph. For example, the current default engine is LevelDB, which is expected to provide high-performance write performance.

Main data structure

KeyValueStore mainly consists of three parts, one is the KeyValueStore class that inherits ObjectStore, the other is GenericObjectMap (similar to FileStore's DBObjectMap), and the last one is the StripObjectMap that inherits GenericObjectMap. GenericObjectMap is mainly used to access the back-end Engine implementation, its function is a bit similar to VFS, and Engine is a variety of different FileSystem, it abstracts some basic methods (read/write) and some high-level interfaces (rename/clone) and so on. At first, when we first started to design GenericObjectMap, we intended to directly use the existing FileStore DBObjectMap, but after some investigation, we found that DBObjectMap lacked some expansibility. It is difficult to implement without breaking the existing interface, so it is finally agreed with Sage to directly implement the new ObjectMap. So what is ObjectMap? as mentioned in the last FileStore article, ObjectMap is a multi-level Map implemented using the OSD V interface to give OSD's most important Object an independent and efficient KV space that can also use no-copy 's clone.

After GenericObjectMap provides a general KV space for Object, StripObjectMap inherits GenericObjectMap to implement the encapsulation of Object Data. ObjectStore has three types of data: Data, attr and Omap, the latter two are a single KV implementation, which can be directly implemented using GenericObjectMap's native interface, but the interface of Data is similar to Posix requires the ability of Parity Write, so simply using an Object Data as a key-value pair is not appropriate, need to do a Strip work The Data of an Object is divided into multiple key-value pairs according to a certain width, which is done by StripObjectMap.

Finally, the KeyValueStore class uses StripObjectMap to complete the method implementation of ObjectStore.

Struct OpWQ defines the operation queue

Op_tp defines the thread pool

Primary IO path

Similar to the implementation of FileStore, KeyValueStore will also generate a message queue, and all IO requests generated by the upper PG will be put into this queue first, and then multiple KeyValueStore threads will be processed as consumer access requests of the queue. Because of the inherent isolation of PG, KeyValueStore currently uses PG as an isolation unit, and only one thread processes the same PG request at a time. The KeyValueStore thread generates a buffer space for each request, because a request as a transaction will contain multiple atomic operations. In order to ensure the atomicity and isolation of the transaction, each request can not be written to the persistence layer in the intermediate stage, only some operation sequences can be generated, and the possible side effects need to be saved by the buffer space as the context of subsequent operations. Finally, the KeyValueStore thread submits the request to complete the transaction.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.