Shulou(Shulou.com)05/31 Report--

This article mainly introduces the meaning of GlusterFS, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand.

1. Related concepts of GlusterFS:

GlusterFS is not only an open source distributed file system, but also the core of Gluster, a Scale-Out storage solution, which has a strong ability to scale out in storing data. GlusterFS is mainly composed of storage server, client and NFS/Samba storage gateway (optional components). The biggest design feature of GlusterFS architecture is that there are no metadata server components, that is, there is no master / slave server, and each node can be a master server.

.

1. The relevant reference documents for Gluster are as follows (my following configuration is based on the local yum configuration. If you need to build the latest version, you can directly follow the documentation link below):

.

Gluster official website: https://wiki.centos.org/SpecialInterestGroup/Storage/gluster-Quickstart

Official documentation for installing Gluster based on centos7/Redhat: https://blog.csdn.net/daydayup_gzm/article/details/52748812

.

2. GlusterFS related terms:

.

Brick (storage block): refers to the dedicated partition provided by the host for physical storage in the trusted host pool.

Volume (logical volumes): a logical volume is a collection of Brick. A volume is a logical device for data storage.

FUSE: a kernel module that allows users to create their own file systems without modifying kernel code.

Glusterd (background Management process): runs on every node in the storage cluster.

VFS: the interface provided by kernel space to user space to access the disk.

3. Volume type of GlusterFS:

Distributed volumes: equivalent to spanned volumes in Windows, but with expanded disk space and no fault tolerance

Stripe volume: equivalent to a striped volume in Windows, which belongs to RAID 0 level. A file will be read and written on multiple disks. The larger the file, the higher the reading and writing efficiency, but it does not have fault tolerance.

Replication volume: equivalent to a mirror volume in Windows, belongs to the RAID 1 level, with fault tolerance and high read performance, but write performance is poor because the same file is written synchronously on multiple Brick.

Distributed stripe volume: the number of brick server is a multiple of the number of stripes (the number of brick distributed in data blocks), which has the characteristics of both distributed volume and stripe volume.

Distributed replication volumes: the number of brick server is a multiple of the number of mirrors (the number of copies of data), which has the characteristics of both distributed and replicated volumes.

Stripe replication volume: similar to RAID 10, with the characteristics of both stripe volume and replication volume.

Distributed stripe replication volume: a composite volume of three basic volumes, usually used in map reduce applications.

Some of the above volume types may not be fully understood, but it doesn't matter. In a production environment, most companies consider disk utilization and use RAID5, or RAID 10. For the configuration of RAID5 volumes, please see: GlusterFS Dispersed Volume (error Correction Volume) summary.

4. The following describes the characteristics of some volume types of GlusterFS (excluding RAID5):

1. Distributed volumes (similar to spanned volumes in Windows):

.

Distributed volumes are the default volumes for GlusterFS, and when creating volumes, the default option is to create distributed volumes. In this mode, the file is not divided into blocks, and the file is stored directly on a server node.

.

Distributed volumes have the following characteristics:

.

1. Files are distributed on different servers and do not have redundancy.

2. Expand the size of the volume more easily and cheaply.

3. A single point of failure will cause data loss.

4. Rely on the underlying data protection.

2. Stripe volume (similar to striped volume in Windows, also known as RAID 0):

.

The stripe mode is equivalent to RAID 0, in which the file is divided into N blocks (N stripe nodes) according to the offset, and is polled stored in each Brick Server node. The node stores each block as an ordinary file in the local file system, and records the total number of blocks and the sequence number of each block through extended attributes. "specify that the number of stripes must be equal to the number of storage servers contained in the Brick in the volume, especially when storing large files, but without redundancy."

.

Stripe volumes have the following characteristics:

.

1. The data is divided into smaller blocks and distributed to different stripes in the block server farm.

2. Distribution reduces load and smaller files speed up access.

3. No data redundancy.

3. Copy the volume (similar to the mirror volume in Windows, also known as RAID 1)

.

Replication mode, that is, one or more copies of the same file are kept, and the same content and directory structure are kept on each node. In the replication mode, the disk utilization is low because the copy is saved. If the storage space on multiple nodes is inconsistent, then the capacity of the lowest node is taken as the total capacity of the volume according to the bucket effect. Replication volumes are redundant, and even if one node is damaged, it does not affect the normal use of the data.

.

Replication volumes have the following characteristics:

1. All servers in the volume keep a complete copy.

2. The number of copies of the volume can be determined by the customer when it is created.

There are at least two block servers or more servers.

4. Redundancy.

4. Distributed replication volumes (also known as RAID 10):

Distributed replication volumes take into account the functions of both distributed volumes and replication volumes, and are mainly used in situations where redundancy is needed.

.

Second, the deployment of various volume types of GlusterFS and the use of client mount:

.

The environment is as follows:

Related information:

Disk mount information:

.

Preparatory work:

1. Operate on all nodes: add disks according to the above table, partition them with fdisk command, format them with mkfs, create a corresponding mount directory, mount the formatted disks to the corresponding directories, and finally modify the / etc/fstab configuration file to make it permanently mounted.

Please refer to the blog post: https://blog.51cto.com/14154700/2350748

2. Configure firewall to release traffic

3. Local yum warehouse: https://pan.baidu.com/s/1ZzURDjGsue7cPnVPdKFCvA (extraction code: 309q)

Start deployment:

.

Configuration of node1:

Configure the hosts file [root@node1 /] # vim / etc/hosts # to add parsing of four nodes. / / omit part of the content 192.168.1.10 node1192.168.1.20 node2192.168.1.30 node3192.168.1.40 node4 [root@node1 /] # mount / dev/sr0 / media/ # is hung in the local yum warehouse mount: / dev/sr0 is write-protected Mounting read-only [root@node1 /] # rm-rf / etc/yum.repos.d/* # Delete the original yum configuration file (or remove it) [root@node1 /] # vim / etc/yum.repos.d/a.repo # write yum configuration file [yum] baseurl= file:///mediagpgcheck=0[root@node1 /] # yum-y install glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma # yum install gluster All nodes need to do [root@node1 /] # systemctl start glusterd # startup service [root@node1 /] # systemctl enable glusterd

Configuration of node2:

.

Because most of the configurations of all nodes are the same, we can use the scp command to copy the configured node files for use

[root@node2 /] # scp root@192.168.1.10:/etc/hosts / etc/ # copy the hosts parsing file to The authenticity of host '192.168.1.10 (192.168.1.10)' can't be established.ECDSA key fingerprint is 68:df:0f:ac:c7:75:df:02:88:7d:36:6a:1a:ae:27:23.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.1.10' (ECDSA) to the list of known hosts.root@192.168.1.10's password: # enter the password hosts 100% 234 0.2KB/s 00:00 [root@node2 /] # rm-rf / etc/yum.repos.d/* [root@node2 /] # scp root @ 192.168.1.10 a.repo 0.0KB/s * / root@node2 /] # mount / dev/sr0 / media # Mount the yum warehouse [root@node2 / ] # yum-y install glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma # installation software [root@node2 /] # systemctl start glusterd # startup service [root@node2 /] # systemctl enable glusterd

Node3 and node4 can operate according to node2 configuration.

.

Add a node (all the next configuration can be done on any node, which I do here on node1):

[root@node1 /] # gluster peer probe node1peer probe: success. Probe on localhost not needed # means that it is on node1 and already exists. There is no need to add [root@node1 /] # gluster peer probe node2peer probe: success. [root@node1 /] # gluster peer probe node3peer probe: success. [root@node1 /] # gluster peer probe node4peer probe: success. [root@node1 /] # gluster peer status # check the cluster status Number of Peers: 3Hostname: node2Uuid: b25b967a-0ec6-4e2e-841f-b13368ec6d73State: Peer in Cluster (Connected) # if a node displays Disconnected, check the hosts configuration file Hostname: node3Uuid: 3ce5c3aa-c1da-4112-825d-998e71220423State: Peer in Cluster (Connected) Hostname: node4Uuid: 0c543cae-ec0c-4249-8600-605e0f552b11State: Peer in Cluster (Connected)

Create a volume:

1. Create a distributed volume:

[root@node1 /] # gluster volume create dis-volume node1:/e6 node2:/e6 force # force: mandatory; where "dis-volume" is the volume name, no type is specified, and distributed volumes are created by default. Volume create: dis-volume: success: please start the volume to access data [root@node1 /] # gluster volume info dis-volume # View information about this volume You can also view the information of other volumes by changing the volume name: Volume Name: dis-volumeType: DistributeVolume ID: 09a6443c-a544-475d-9084-28892ce0093fStatus: StartedSnapshot Count: 0Number of Bricks: 2Transport-type: tcpBricks:Brick1: node1:/e6Brick2: node2:/e6Options Reconfigured:transport.address-family: inetnfs.disable: on [root@node1 /] # gluster volume start dis-volume # enable the volume volume start: dis-volume: success

2. Create a stripe volume:

[root@node1 /] # gluster volume create stripe-volume stripe 2 node1:/d5 node2:/d5 forcevolume create: stripe-volume: success: please start the volume to access data# creates stripe volumes and specifies that the number of stripe volumes is 2. " Stripe-volume "specifies stripe for the volume name #, with a value of 2, followed by 2 brick server, so the stripe volume [root@node1 /] # gluster volume start stripe-volume # enables the volume volume start: stripe-volume: success

3. Create a replication volume:

[root@node1 /] # gluster volume create rep-volume replica 2 node3:/d5 node4:/d5 forcevolume create: rep-volume: success: please start the volume to access data# specifies a type of "replica", a value of "2", followed by two brick server, so create a replication volume [root@node1 /] # gluster volume start rep-volume # enable the volume volume start: rep-volume: success

4. Create a distributed stripe volume:

[root@node1 /] # gluster volume create dis-stripe stripe 2 node1:/b3 node2:/b3 node3:/b3 node4:/b3 forcevolume create: dis-stripe: success: please start the volume to access data# specifies a type of stripe, a value of 2, followed by four brick server, so the distributed stripe volume [root@node1 /] # gluster volume start dis-stripe # enables volume start: dis-stripe: success

5. Create a distributed replication volume:

[root@node1 /] # gluster volume create dis-rep replica 2 node1:/c4 node2:/c4 node3:/c4 node4:/c4 forcevolume create: dis-rep: success: please start the volume to access data# specifies that the type is replica, the value is 2, and it is followed by four brick server, which is twice the number of 2, so the distributed replication volume [root@node1 /] # gluster volume start dis-repvolume start: dis-rep: success is created.

At this point, the required volumes are created.

.

Deploy the Gluster client:

.

1. Install client software

[root@client /] # rm-rf / etc/yum.repos.d/* [root@client /] # scp root@192.168.1.10:/etc/yum.repos.d/* / etc/yum.repos.d/root@192.168.1.10's password: a.repo 100% 39 0.0KB/s 00:00 # Mount the local yum Warehouse [root@client /] # yum-y install glusterfs glusterfs-fuse # yum install client

2. Create a mount directory:

[root@client /] # mkdir-p / test/ {dis,stripe,rep,dis_and_stripe,dis_and_rep} [root@client /] # ls / test/dis dis_and_rep dis_and_stripe rep stripe

3. Modify hosts file: (direct scp copy is also possible)

[root@client /] # scp root@192.168.1.10:/etc/hosts / etc/The authenticity of host '192.168.1.10 (192.168.1.10)' can't be established.ECDSA key fingerprint is 68:df:0f:ac:c7:75:df:02:88:7d:36:6a:1a:ae:27:23.Are you sure you want to continue connecting (yes/no)? YesdWarning: Permanently added '192.168.1.10' (ECDSA) to the list of known hosts.root@192.168.1.10's password: hosts 100% 234 0.2KB/s 00:00 [root@client /] # cat / etc/hosts. / / omitted part 192.168.1.10 node1192.168.1.20 node2192.168.1.30 node3192.168.1.40 node4

4. Mount the Gluster file system:

[root@client /] # mount-t glusterfs node1:dis-volume / test/dis [root@client /] # mount-t glusterfs node1:stripe-volume / test/stripe [root@client /] # mount-t glusterfs node1:rep-volume / test/rep/ [root@client /] # mount-t glusterfs node1:dis-stripe / test/dis_and_stripe/ [root @ client /] # mount-t glusterfs node1:dis-rep / test/dis_and_rep/ [root @ client /] # df- H # View mount information If the mount is not successful, check the hosts parsed file system capacity available available% mount point. / / omit some node1:dis-volume 12G 65M 12G 1% / test/disnode1:stripe-volume 10G 65M 10G 1% / test/stripenode1:rep-volume 5.0G 33M 5.0G 1% / test/repnode1:dis-stripe 12G 130M 12G 2% / test/dis_and_stripenode1:dis-rep 8.0G 65M 8.0G 1% / test/dis_and_rep

5. Modify the fstab file and set automatic mount:

[root@client /] # vim / etc/fstab. / / omit part node1:stripe-volume / test/stripe glusterfs defaults,_netdev 0 0node1:rep-volume / test/rep glusterfs defaults,_netdev 0 0node1:dis-stripe / test/dis_and_stripe glusterfs defaults,_netdev 0 0node1:dis-rep / test/dis_and_rep glusterfs defaults,_netdev 0 0node1:dis-volume / test/dis glusterfs defaults _ netdev 0 0

GlusterFS maintenance commands:

[root@node1 /] # gluster volume list / / View the list of volumes dis-repdis-stripedis-volumerep-volumestripe-volume [root@node1 /] # gluster volume info / / View the information of all volumes Volume Name: dis-repType: Distributed-ReplicateVolume ID: f57c0228-4a5f-4995-8fac-87f9bc034c33Status: StartedSnapshot Count: 0Number of Bricks: 2 x 2 = 4Transport-type: tcpBricks:Brick1: node1:/c4Brick2: node2:/c4Brick3: node3:/ C4Brick4: node4:/c4Options Reconfigured:transport.address-family: inetnfs.disable: on. / / omit part [root@node1 /] # gluster volume status / / View the status of the volume Status of volume: dis-repGluster process TCP Port RDMA Port Online Pid -Brick node1:/c4 49155 0 Y 13656Brick node2:/c4 49155 0 Y 13443Brick node3:/c4 49154 0 Y 13480Brick node4:/c4 49154 0 Y 13334Self-heal Daemon on localhost N/A N/A Y 13676Self-heal Daemon on node2 N/A N/A Y 13463Self-heal Daemon on node4 N/A N/A Y 13354Self-heal Daemon On node3 Nacha Nachet A Y 13500Task Status of Volume dis-rep---- [root@node1 ~] # gluster Volume stop dis-stripe # stop a volume Stopping volume will make its data inaccessible. Do you want to continue? Yvolume stop: dis-stripe: success [root@node1 ~] # gluster volume delete dis-stripe # Delete a volume Deleting volume will erase all information abe. Do you want to continue? Yvolume delete: dis-stripe: success [root@node1 ~] # gluster volume set dis-rep auth.allow 192.168.1. The setting allows only clients with specific network segments to access the questionnaire dis-repvolume set: success Thank you for reading this article carefully. I hope the article "what is the meaning of GlusterFS" shared by the editor will be helpful to you, and I also hope you will support it. Pay attention to the industry information channel, more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.