Shulou(Shulou.com)11/24 Report--

Overview

Tide Cloud Sea InCloud Rail super converged all-in-one machine realizes the resource pooling of servers through software-defined computing, storage and network technology, providing higher availability, security and scalability for the entire IT environment, and can meet the needs of enterprises for reducing costs, simplifying management, improving security and scalability, helping enterprises migrate core business to the cloud and build enterprise cloud data centers.

In the face of massive data processing requirements, Chaochao Yunhai Super Fusion all-in-one computer proposes a data storage space optimization scheme that integrates compression and deletion correction functions, which is based on the QAT accelerator provided by Intel ®strong ®scalable processor to achieve compression function, while using Intel ®intelligent acceleration library to achieve deletion correction function, which can support data compression and deletion correction at the same time. In the massive application archiving data scene of super-integrated machine, it needs to improve the data storage efficiency as much as possible under the premise that the running quality of the application is less affected. Under the premise of cost control, enabling both compression and deletion correction features can save up to 70.5% of storage space for database applications, and has a low impact on performance, which can help users improve the investment return of the database system and release data value.

Wave Cloud Sea Super Fusion all-in-one distributed Storage InCloud dSAN

InCloud dSAN is a new generation of software-defined storage products in Chaoyunhai Super Fusion computer, which is oriented to a wide range of application scenarios, such as private cloud, big data, high-performance applications, cloud natives, cloud-edge collaboration, and so on. With the popularity of high-performance physical hardware such as Intel new generation CPU, NVMe hard disk and RDMA intelligent network card, InCloud dSAN designs a variety of features according to the characteristics of the new generation physical hardware in system design:

● full asynchronous and lock-free programming technology, give full play to CPU multi-core performance, based on SPDK (Storage Performance Development Kit) high-performance storage development kit, the introduction of Reactor, Thread, Poller three layers of logic abstraction, design polling, asynchronous, lock-free software framework, from the network, disk, management and other modules to play CPU multi-core capabilities.

● multi-link transmission technology, support a variety of networking forms: design full-stack RDMA function, from the virtual machine, storage protocol layer, replica data forwarding layer and other internal network transmission, support full-stack user-mode RDMA link transmission, using RDMA zero-copy characteristics to reduce delay.

● new storage engine design, play the extreme performance of NVMe: design a new engine based on bare disk read and write, design metadata, log, data and other logic space, achieve metadata management, data distribution and I / O scheduling and other logic. Especially for NVMe hard disk, a unique NVMe storage engine is implemented based on the user-mode NVMe library in the SPDK storage development kit.

Panoramic picture of InCloud Rail all-in-one product

Compression and erasure correction function design of wave cloud sea super fusion in the new generation G7 hardware platform

In the super-converged application scenarios, database applications aimed at, and other industries not only require high storage performance for real-time read and write, random I / O access and large-scale data sets, but also put forward new requirements for the storage space of archived data, such as PACS (Picture Archiving and Communication System) image archiving and communication systems commonly used in the industry. The daily images (including nuclear magnetic resonance, CT, ultrasound and other images) are stored in large quantities by digital means, and can be quickly recalled and viewed when needed, which has high requirements for storage system archiving and performance. In order to cope with this scenario, the InCloud dSAN of Chaoyunhai's new generation of super-integrated machine has designed a new type of data compression and deletion correction function, which integrates Intel ®QAT accelerator and ISA-L acceleration library, and meets the requirements of extreme performance and extreme data compression on the basis of the original high-performance design scheme.

The main idea of the compression and deletion correction scheme: the compression and deletion correction calculation is placed in the cache brushing stage, which can avoid the performance loss of compression and deletion correction calculation in the process of I / O access. Through intelligent cache management, it ensures that the I / O access of hot data is concentrated in the cache layer and avoids overwriting in the process of data access.

InCloud dSAN compression and erasure function design architecture diagram

● data hierarchical management: data storage is divided into cache layer and data layer, in which the cache layer uses high-performance NVMe and Sata SSD storage, while the design of multiple copies in the cache layer provides high-performance data storage capacity. The data layer provides cold data storage, which is mainly stored by HDD hard drives, providing high-capacity storage space.

● intelligent cache management: design hot and cold layered intelligent cache management, efficient identification of hot data and cold data. Priority can be set for specific data to ensure that data areas that need to support compression and deletion can be preferentially saved in the cache space.

Efficient storage of ● data space: due to replica storage in the cache layer, small blocks of IO applied in the upper layer are aggregated. When the data becomes cold, the data is brushed down to the storage layer, and because the lower brush adopts data alignment, it can be aggregated into large sequential stripes to read and write, and can be efficiently stored in the data layer.

● compression and EC accelerated computing: when the cache layer data is brushed, it is first processed by QAT compression. After the compressed data is called EC calculation, the erasure correction and compressed data are stored in the data storage layer.

In order to verify the performance of the scheme, the test data are based on the basic Linux virtual machine image, the SQLServer database of Windows system, and the Oracle database application of Linux system, which are common in the industry and other industries. Adjust the level of QAT compression and ISA-L 's EC erasure correction K / M model to test the space savings after different model compression, deletion correction and compression and deletion correction. From the analysis of the test results, the maximum space savings can be up to 70.5% when QAT and EC are enabled at the same time.

Comparison of functional space savings between compression and deletion correction

Income

Taking full advantage of the Intel ®QAT accelerator built into the fourth-generation Intel ®strong ®scalable processor, the QAT accelerator saves 58.4% of data space for database applications. EC can calculate the proportion of space saved according to different K / M models, and save 70.5% in specified K / M model compression and erasure correction functions.

Summary

Relying on Intel ®QAT and ISA-L software acceleration library technology, the new generation of Chaoyunhai super convergent all-in-one machine can support different space saving schemes such as compression, deletion correction, "compression and deletion correction" at the same time, and adopts unique data layering, intelligent cache management, software and hardware cooperation design, which perfectly solves the technical problems such as high performance, high data redundancy and high storage space utilization.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.