Shulou(Shulou.com)06/02 Report--

openstack storage type

Storage in openstack can be divided into two categories, as shown in the following figure:

Openstack currently supports three types of persistent storage: block storage, object storage, and file system storage

Because Manila is less used at present, the focus is on CInder and Swift.

Block Storage CinderCinder Introduction

Cinder's position and role in OpenStack

Cinder's core function is volume management, allowing processing of volumes, volume types, volume snapshots, and volume backups. It provides a unified interface for different storage devices on the backend. Different block device service vendors implement their drivers in Cinder, which can be integrated and managed by Openstack.

Cinder's structure

Cinder Client encapsulates Cinder to provide rest interface, providing users with rest API in CLI form, parsing operation requirements, routing API to find corresponding processing method. This includes volume addition, deletion and query (including source volume, mirror, snapshot creation), snapshot addition, deletion and query, backup, volume type management, mount/unmount (Nova call), etc. Cinder Scheduler is responsible for collecting the capacity and capability information reported by backend. The algorithm of root setting completes the scheduling of volume to specified cinder-volume. Cinder Volume multi-node deployment uses different configuration files to access different backend devices. Each storage vendor inserts driver code to interact with devices to complete the collection of device capacity and capability information. Volume operation Cinder Backup realizes the backup of volume data to other storage media.(SWIFT/Ceph/TSM currently provides drivers) SQL DB provides storage volume, snapshot, backup, service and other data, and supports SQL databases such as Mysql, PG,MSSQL, etc.

Cinder Component-API

Cinder API provides REST API to analyze operation requirements and invoke processing methods:

volume create/delete/list/show snapshot create/delete/list/show volume attach/detach(nova call) Other:

Volume types

Qutotas

BackupsCinder Component-Scheduler

The Cinder scheduler is responsible for collecting the capacity and capability information reported by the backend, and completing the scheduling of the volume to the specified cinder-volume according to the set algorithm. It filters out the appropriate backend through filtering and weighting:

Filter based on backend capabilities

Drivers periodically report on the capabilities and status of the backend Volume types created by the administrator When creating a volume, the user specifies the volume type Cinder component-Volume

Cinder volume multi-node deployment, using different configuration files, accessing different back-end devices, inserting Driver code by each storage vendor to interact with devices, completing device capacity and capability information collection, volume operation, etc.

Cinder's default backend driver is LVM

typical workflow

Cinder-volume periodically collects the capacity information of the underlying backend and notifies the Scheduler to update the Backend information in memory.

The purpose of creating volume types is to filter different backend storage, such as SSD,STAT, high performance, low performance. By creating different custom volume types, the appropriate backend storage is automatically filtered when creating volumes.

Cinder API

Check the validity of volume parameters (user input, permissions, existence of resources, etc.) Prepare to create parameter dictionaries, reservations, and commit quotas Create corresponding data records in the database Send requests and parameters to Scheduler via message queues

Extract received request parameters

Filter by configured filter and input parameter backend

Avialability_zone_filterCapactiy_filterCapabilities_filterAffinity_filter

Weighter Computing Backend Weights

CapactiyWeigher/AllocatedCapacityWeigherChanceWeigherGoodnessWeigher

Select the best Backend and queue the request to the specified Backend

Similar to Nova Scheduler, Cinder Scheduler also selects the backend with appropriate conditions after Filter, and then uses Weighter to calculate the backend weight sorting, and finally selects the most appropriate backend storage.

Cinder Volume

Provide the parameters of the received request

Call the corresponding Driver to create the Century volume on the backend

Update records in the database with the model returned by Driver

Cinder mount volume flow

Volume hanging process: When hanging a volume, the remote volume is finally connected to the Host node where the virtual machine is located through the cooperation of Nova and Cinder, and finally mapped to the internal virtual machine through the virtual machine management program.

Nova calls Cinder api to create a volume, passing host information, such as hostname,ISCSI initiator name,FC WWWPNsCinder API passes this information to Cinder VolumeCinder Volume finds the corresponding Cinder DriverCinder Driver notifies the storage to allow the host to access the volume, and returns connection information for the storage (e.g. ISCSI iqn, portal,FC target WWPN, NFS path)Nova calls the code for host identification of disks for different storage types (Cinder provides brick module for reference) to identify disks or file devices Nova notifies Cinder that mounting has been performed Nova passes host device information to hypervisor to implement virtual machine mounting of disks Cinder Main operations

Object Storage Swift Introduction

Swift provides highly available, distributed, and ultimately consistent object storage services

Swift is not a file system or a data storage system, it's called object storage for long-term storage of static data of permanent type, which can be retrieved, adjusted, and updated as necessary.

Examples of the most snake and stored data types are virtual machine mirroring, picture storage, mail storage and archive backups

Because there is no central unit or master node, Swift provides greater scalability, redundancy, and durability.

Swift is often used to store mirrors or backup copies of VM instance volumes

Swift Apps

Mirror storage backend: Combine with mirror service Glance in Openstack to store mirror files for it

Static data storage: due to swift's scalability, it is suitable for storing log files and data backup warehouses

Swift Architecture

The object storage URL in Swift is as follows:

https://swift.example.com/va/account/container/object

URL has two parts: centralized location and storage location

Cluster location: swift.example.com/v1/

Storage location (object):/account/containr/object

There are three storage locations:

/account

Account storage locations are uniquely named storage areas that contain metadata (descriptive information) about the account itself and a list of containers in the account

/account/container

A container storage location is a user-defined storage area within an account that contains metadata for the container itself and the list of objects in the container

/account/container/object

Object storage locations store the location of data objects and metadata.

Swift components

Proxy Server: Provides an object service API to the outside world. Due to the stateless REST request protocol, it can be horizontally extended to Load Balancer

Account Server: A service that provides account metadata and statistics, and maintains a list of containers included. Information for each account is stored in a SQLite database.

Container Server: A service that provides container metadata and statistics, and maintains a list of objects contained therein. Information about each container is also stored in a SQLite database.

Object Server: provides object metadata and content services, the content of each object is stored as a file in the file system, metadata is stored as file attributes

Replicator: checks whether the local partition replica is consistent with the remote replica, pushes the remote replica when it is inconsistent, and ensures that objects marked for deletion are removed from the file system

Updater: When an object cannot be updated immediately due to high load, the task will be serialized and queued in the local file system for asynchronous updates when service is restored

Auditor: Checks the integrity of objects, containers, and accounts, and if bit-level errors are found, the file is quarantined and other copies are copied to overwrite the local corrupt copy; other types of errors are logged

Account Reaper: Remove the account marked for deletion, deleting all containers and objects it contains

Swift API

Swift provides HTTP-based REST service interfaces through Proxy servers, and performs CRUD operations on accounts, containers, and objects.

Swift data model

Three-layer logical structure: Accout/Contianer/Object

There is no limit to the number of nodes per layer, and it can be expanded arbitrarily.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.