How to build a highly available file sharing environment based on cman+rgmanage+gfs2+iscsi+rbd

How to build a highly available file sharing environment based on cman+rgmanage+gfs2+iscsi+rbd? in view of this problem, this article introduces the corresponding analysis and solutions in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

I. Architecture Diagram

Description: rbd is exported as shared storage in iscsi mode. Iscsi target is 10.20.18.111 iscsi intinator is 10.20.15.163 and 10.20.15.164. Cman+rgmanager manages the nfs-server cluster, and gfs2 is the cluster file system, which manages the parallel operation of files in the cluster.

Second, iscsi target configuration 1. Install target# rpm-ivh scsi-target-utils-1.0.38-48.bf6981.ceph.el6.x86_64.rpm# service tgtd start2 that supports rbd. Check whether rbd# sudo tgtadm-- lld iscsi-- op show-- mode systemSystem:State: readydebug: offLLDs:iser: erroriscsi: readyBacking stores:rbd (bsoflags sync:direct) rdwr (bsoflags sync:direct) sscnullbsgsgsheepdogDevice types:passthroughtapechangercontrollerosdcd/dvddiskiSNS:iSNS=OffiSNSServerIP=iSNSServerPort=3205iSNSAccessControl=Off3. Configure rbd-tgt (get Pool and rbd img ready) # vi / etc/tgt/targets.confdriver iscsibs-type rbdbacking-store iscsi/iscsi-rbd # / 4. Load configuration # service tgtd reloadUpdating target framework daemon configuration III, configure the iscsi client (163Universe 165machines) 1. Install iscsi-initiator-utils # yum install iscsi-initiator-utils2. It is found that the target side shared device # iscsiadm-m discovery-t sendtargets-p 10.20.18.110.20.18.111 iqn.2015-04.rbdstore.example.com:iscsi3.Map target# iscsiadm-m node-T iqn.2014-04.rbdstore.example.com:iscsi-p 10.20.18.111 target 3260-loginLogging in to [iface: default, target: iqn.2014-04.rbdstore.example.com:iscsi, portal: 192.168.1.102 multiple 3260] (multiple) Login to [iface: default Target: iqn.2014-04.rbdstore.example.com:iscsi, portal: 192.168.1.102] successful.4. Check the disk device (one more sda device) fdisk-lDisk / dev/sda: 1073 MB, 1073741824 bytes34 heads, 61 sectors/track, 1011 cylindersUnits = cylinders of 2074 * 512 = 1061888 bytesSector size (logical/physical): 512 bytes / 4194304 bytesI/O size (minimum/optimal): 4194304 bytes / 4194304 bytesDisk identifier: 0x00000000 IV, configure cman+rgmanager1. Configure hosts# vi / etc/hosts10.20.15.165 DEV-L000355710.20.15.163 CNSZOSD012. Ssh mutual trust # ssh-keygen-t rsa-P "" # ssh-copy-id-I. ssh / id_rsa.pub root@ CNSZOSD013. Ntp synchronization

(both nodes are configured)

4. Install cman and rgmanager# yum install cman rgmager-y5. Configure the cluster (one node executes) # ccs_tool create tcluster# ccs_tool addnode CNSZOSD01-n "ccs_tool addnode DEV-L0003557-n" vi / etc/cluster/cluster.conf6.copy cluster configuration # scp / etc/cluster/cluster.conf initiator2:/etc/cluster/7. Two nodes start the service # service cman start# service rgmanager start8. Check the cluster status (all online means ok) # clustatCluster Status for tcluster @ Fri May 8 13:53:40 2015Member Status: QuorateMember Name ID Status- CNSZOSD01 1 OnlineDEV-L0003557 2 Online, Local 5, configure gfs21. Install gfs2# yum install gfs2-utils2. Format the / dev/sda device as the gfs file system # mkfs.gfs2-j 2-p lock_dlm-t tcluster:sda dev/sda

Note: gfs is a log file system, and each node must have a log. In this experiment, there are two nodes, so create two logs. Tcluster is the cluster name and sda is the lock table name.

3. Two nodes mount the gfs2 file system # mount-t gfs2-o _ netdev,nodiratime,noatime / dev/sda / mnt4. You can test file read and write on two nodes. CLVM configuration (two) 1. Install # yum install-y lvm2-cluster2. Enable cluster LVM# lvmconf-- enable-cluster3. Start the clvm service # service clvmd start4. Create cluster logical volume (operation before / dev/sda is not formatted) # pvcreate / dev/sda create physical volume # vgcreate clustervg/ dev/sda create volume group # lvcreate-L 500m-n clusterlv clustervg create logical volume # mkfs.gfs2-j 2-p lock_dlm-t tcluster:lv / dev/clustervg/clusterlv# mount / dev/clustervg/clusterlv / MNT VII, vip configuration (brief) 8, nfs configuration 1.NFS server-side configuration # yum-y install nfs-utils rpcbind# yum install nfs-utils

2. Configuration file definition of NFS: / etc/exports

# vim / etc/exports / mnt 10.20.room.* (rw,no_root_squash,no_all_squash,sync)

Note: profile description:

/ mnt is the shared directory, that is, the directory where the rbd is attached, and the absolute path .10.20. *. * allows the client to access the IP. () is the permission control.

3. Server-side startup service # service rpcbind start# service nfs start4. Client mount (any host with network segment 10.20.room.*) # mkdir / usr/local/nfstest

# configuration takes effect

# exportfs-rv5. Client test mount # showmount-e 10.20.16.1776. Mount # mount-t nfs 10.20.16.177:/mnt / usr/local/ nfstest 9, cifs\ smb configuration 1. Install samba # yum install samba samba-client samba-swat2. Configure samba configuration

(simple configuration without client password authentication)

# vi / etc/samba/smb.conf [global] / / this setting is related to the overall running environment of the Samba service Its setup project is for all shared resources #-Network Related Options-# # workgroup = NT-Domain-Name or Workgroup-Name, eg: MIDEARTH## server string is the equivalent of the NT Description field## netbios name can be used to specify a server name not tied to the hostnameworkgroup = WORKGROUP / / defines the workgroup That is, the workgroup concept in windows server string = DavidSamba Server Version% v / / defines a brief description of the Samba server netbios name = DavidSamba / / defines the computer name shown in windows #-- Logging Options-- # # Log File let you specify Where to put logs and how to split them up.log file = / var/log/samba/log.%m / / define the log file for the Samba user % m represents the client hostname / / the Samba server will establish a different log file for each login host in the specified directory #-Standalone Server Options-# # Scurity can be set to user, share (deprecated) or server (deprecated) security = share / / share level Users do not need an account and password to access # = = Share Definitions = = [public] / / Settings are for individual settings of shared directories, and only work on current shared resources comment = Public Stuff / / A pair of shared directory description files. You can define the description information path = / mnt / / to specify the shared directory. The required option public = yes / / can be viewed by everyone, which is equivalent to guest ok = yes3.windows client access\\ 10.20.15.163\ public. This is the answer to the question on how to build a highly available file sharing environment based on cman+rgmanage+gfs2+iscsi+rbd. I hope the above content can be of some help to everyone. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.