Shulou(Shulou.com)06/02 Report--

Chapter 1 introduction to the use of GlusterFS distributed File system 1.1 Glusterfs Overview

GlusterFS is the core of Scale-Out storage solution Gluster. It is an open source distributed file system with strong scale-out ability to support several PB storage capacity and handle thousands of clients. GlusterFS gathers physically distributed storage resources together with TCP/IP or InfiniBand RDMA networks and uses a single global namespace to manage data. GlusterFS is based on a stackable user space design that provides excellent performance for a wide variety of data loads.

GlusterFS supports standard clients running standard applications on any standard IP network. As shown in figure 2, users can access application data using standard protocols such as NFS/CIFS in a globally unified namespace. GlusterFS enables users to get rid of the original independent, high-cost closed storage system, can use ordinary cheap storage devices to deploy storage pools that can be centrally managed, scale out, and virtualized, and the storage capacity can be expanded to the TB/PB level. The main features of GlusterFS are as follows:

1.1.1 scalability and high performance

GlusterFS takes advantage of dual features to provide highly scalable storage solutions from several TB to several PB. The Scale-Out architecture allows storage capacity and performance to be improved by simply adding resources. Disk, computing, and Imax O resources can all be added independently, supporting high-speed network interconnection such as 10GbE and InfiniBand. Gluster Elastic Hash (Elastic Hash) removes the need for metadata server in GlusterFS, eliminates single point of failure and performance bottleneck, and truly realizes parallel data access.

1.1.2 High availability

GlusterFS can automatically copy files, such as mirrored or multiple copies, to ensure that data is always accessible, even in the event of a hardware failure. The self-healing function can restore the data to the correct state, and the repair is performed incrementally in the background, with little performance load. GlusterFS does not design its own private data file format, but uses mainstream disk file systems in the operating system (such as EXT3, ZFS) to store files, so data can be copied and accessed using a variety of standard tools.

1.1.3 Global uniform namespace

The global unified namespace aggregates disk and memory resources into a single virtual storage pool, shielding the underlying physical hardware from upper-level users and applications. Storage resources can be flexibly expanded in the virtual storage pool as needed, such as expanding or shrinking. When storing virtual machine images, there is no limit to the number of virtual image files stored, and thousands of virtual machines share data through a single mount point. Virtual machine Imando O can automatically load balance on all servers in the namespace, eliminating the access hotspots and performance bottlenecks that often occur in the SAN environment.

1.1.4 Elastic Hash algorithm

Instead of using centralized or distributed metadata server indexes, GlusterFS uses an elastic hashing algorithm to locate data in the storage pool. In other Scale-Out storage systems, metadata servers usually cause performance bottlenecks and single point of failure problems. In GlusterFS, all storage systems in the Scale-Out storage configuration can intelligently locate arbitrary data fragments without looking at the index or querying other servers. This design mechanism completely parallelizes data access and achieves real linear performance expansion.

1.1.5 Elastic volume management

Data is stored in logical volumes, which can be logically partitioned independently from virtualized physical storage pools. The storage server can be added and removed online without interrupting the application. Logical volumes can grow and shrink in all configuration servers, can be migrated across different servers for capacity balancing, or add and remove systems, all online. File system configuration changes can also be made and applied online in real time to adapt to changes in workload conditions or online performance tuning.

1.1.6 based on standard protocol

Gluster storage services support NFS, CIFS, HTTP, FTP and Gluster native protocols and are fully compatible with the POSIX standard. Existing applications can access data in Gluster without making any modifications or using a dedicated API. This is useful when deploying Gluster in a public cloud environment, where Gluster abstracts the cloud service provider-specific API and then provides a standard POSIX interface.

1.1.7 introduction of terms

A storage unit in Brick:GFS that is an export directory of a server in a trusted storage pool. Can be identified by hostname and directory name, such as' SERVER:EXPORT'

Client: the device with the GFS volume mounted

Extended Attributes:xattr is a file system feature that allows users or programs to associate files / directories and metadata.

FUSE:Filesystem Userspace is a loadable kernel module that allows non-privileged users to create their own file systems without modifying kernel code. The code that runs the file system in user space is bridged with the kernel through FUSE code.

Geo-Replication

Each file or directory in a GFID:GFS volume is associated with a unique 128bit data, which is used to simulate inode

Namespace: each Gluster volume exports a single ns as the mount point for POSIX

Node: a device with several brick

RDMA: remote direct memory access, which supports direct memory access without the OS of both parties.

RRDNS:round robin DNS is a method that returns different devices through DNS rotation for load balancing.

Self-heal: used for background runs to detect inconsistencies between files and directories in the replica volume and resolve these inconsistencies.

Split-brain: cerebral fissure

Translator:

The configuration file for the Volfile:glusterfs process, usually located in / var/lib/glusterd/vols/volname

Volume: a logical set of bricks

1.1.18 composition of Glusterfs architecture

1.2 GlusterFS Technical Features 1.2.1 full Software implementation (Software Only)

GlusterFS believes that storage is a software problem and cannot be solved by limiting users to specific vendors or hardware configurations. GlusterFS uses an open design and widely supports industry-standard storage, network, and computer devices, rather than bundling with customized dedicated hardware devices. For commercial customers, GlusterFS can be delivered as a virtual appliance, packaged with a virtual machine container, or an image deployed in a public cloud. In the open source community, GlusterFS is widely deployed on various operating systems based on cheap idle hardware to form a centralized and unified pool of virtual storage resources. In short, GlusterFS is an open, full-software implementation, completely independent of hardware and operating systems.

1.2.2 complete storage operating system stack (Complete Storage Operating System Stack)

GlusterFS not only provides a distributed file system, but also provides many other important distributed functions, such as distributed memory management, Imax O scheduling, soft RAID, and self-healing. GlusterFS draws lessons from the micro-kernel architecture, draws lessons from the design idea of the GNU/Hurd operating system, and implements a complete storage operating system stack in the user space.

1.2.3 user space implementation (User Space)

Unlike traditional file systems, GlusterFS is implemented in user space, which makes it particularly easy to install and upgrade. In addition, it also greatly reduces the threshold for ordinary users to modify GlusterFS based on source code, requiring only general C programming skills and no special kernel programming experience.

1.2.4 Modular Stack Architecture (Modular Stackable Architecture)

GlusterFS uses a modular, stack-like architecture, and can support highly customized application environments through flexible configuration, such as large file storage, massive small file storage, cloud storage, multi-transfer protocol applications, etc. Each function is implemented in the form of a module, and then a simple combination of building blocks can achieve complex functions. For example, Replicate module can achieve RAID1,Stripe module can achieve RAID0, through the combination of the two can achieve RAID10 and RAID01, while achieving high performance and high reliability.

1.2.5 Raw data format storage (Data Stored in Native Formats)

GlusterFS stores data in original data formats (such as EXT3, EXT4, XFS, ZFS) and implements a variety of automatic data repair mechanisms. As a result, the system is so flexible that files can be accessed through other standard tools even offline. If users need to migrate data from GlusterFS, they can still use the data completely without any modification.

1.2.6 metadata Free Service Design (No Metadata with the Elastic Hash Algorithm)

One of the biggest challenges for Scale-Out storage systems is to record the mapping relationship between data logic and physical location, that is, data metadata, which may also include information such as attributes and access rights. Traditional distributed storage systems use centralized or distributed metadata services to maintain metadata. Centralized metadata services will lead to single point of failure and performance bottlenecks. Distributed metadata services have performance load and metadata synchronization consistency problems. Especially for the application of a large number of small files, the problem of metadata is a great challenge.

GlusterFS uniquely uses the design of no metadata service, instead of using algorithms to locate files, metadata and data are not separated but stored together. All the storage system servers in the cluster can intelligently locate the file data fragments, only according to the file name and path and using the algorithm, without the need to query the index or other servers. This makes data access completely parallelized, resulting in true linear performance expansion. No metadata server greatly improves the performance, reliability and stability of GlusterFS.

1.3 Glusterfs installation and deployment 1.3.1 Experimental Environment

System version: Linux CentOS release 6.9 (Final)

System kernel: 2.6.32-696.23.1.el6.x86_64

Glusterfs Software version: 3.10.12

Node1:192.168.1.111

Node2:192.168.1.112

1.3.2 Server-side deployment

Yum-y install centos-release-gluster310 # install glusterfs yum source

Yum install-y libgssglue libtirpc rpcbind lvm2-develyum list glusterfs libaio # installation dependency

Yum install-y atop fio iperf # install testing tools

Yum install-y glusterfs-libs glusterfs glusterfs-api glusterfs-cli glusterfs-client-xlators glusterfs-fuse glusterfs-server # install glusterfs server software

Mkfs.ext4 / dev/sdb # formats the file system

Mount / dev/sdb / storage/brick1/ mount file system

Mount # View

Vim / etc/fstab # set up boot auto mount

/ dev/sdb / storage/brick1 exit4 defaults 0 0

Service glusterd start # start the gluster service

Chkconfig glusterd on # gluster service boot self-startup

Gluster peer probe node2 # join hosts to the cluster

Gluster peer status # View node status

=

Create a distributed hash volume:

Gluster volume create testol node2:/storage/brick1/b1 # create a volume testol to the / storage/brick1/b1 directory

Gluster volume start testol # starts the volume testol

Gluster volume rebalance testol fix-layout start # Hash range reallocation. Usually execute this command after data migration or adding a new brick to do the load.

=

Delete a volume (it is best to unmount the client mounted by mount before deleting it):

Gluster volume stop testol # stop the volume first

Gluster volume delete testol # and then delete

PS: after deleting the volume, the data is still there. If you don't think it is necessary, you can delete the data.

=

Delete a brick

Gluster volume remove-brick [replica]...

=

Load balancing: (load balancing for other volumes after deleting brick)

Gluster volume rebalance VOLNAME {start | stop |} [force]

=

Create a replication volume:

Gluster volume create testvol replica 2 node2:/storage/brick1/b2 # create a replication volume

1.3.3 client software installation

Yum install-y glusterfs-fuse glusterfs-libs glusterfs # software to be installed by the client

Mount-t glusterfs node1:testvol / mnt # client mount, pay attention to do a good hosts resolution.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.