Shulou(Shulou.com)06/03 Report--

MFS working principle distributed principle

Distributed file system is to assemble some shared folders scattered on multiple computers into a shared folder. When users want to access these folders, they only need to open a folder. You can see all the shared folders linked to this folder.

MFS principle

MFS is a fault-tolerant network distributed file system, which stores data on multiple physical servers and presents a unified resource to users.

The composition of MFS

Metadata server (Master): responsible for managing the file system and maintaining metadata throughout the system, but does not currently support high availability.

Metadata log server (MetaLogger): back up the change log files of the Master server. When the master server is damaged, you can recover the files from the log server.

Data storage server (Chunk Server): the more servers that actually store data, the larger the capacity, the higher the reliability, and the better the performance.

Client (Client): you can mount the MFS file system like NFS.

The process of reading data by MFS

The client issues a read request to the metadata server

The metadata server informs the client where the required data is stored (the IP address and Chunk number of Chunk Server)

The client sends data to a known Chunk Server request

Chunk Server sends data to the client

The process of writing

The client sends a write request to the metadata server

The metadata server interacts with Chunk Server, but the metadata server only creates a new partitioned Chunks on some servers. After the creation is successful, the hunk Servers informs the metadata server that the operation is successful.

The metadata server tells the client which Chunk Server and which Chunks the data can be written to

The client writes data to the specified Chunk Server

The Chunk Server synchronizes data with other Chunk Server. After the synchronization is successful, the Chunk Server informs the client that the data is written successfully.

The client informs the metadata server that this write is complete.

Experimental environment: master 192.168.45.13 "close the firewall [root@localhost] # systemctl stop firewalld.service [root@localhost ~] # setenforce" installation component [root@localhost] # yum install zlib-devel gcc gcc-c++-y # mount the necessary software package for mfs [root] @ localhost ~] # mount.cifs / / 192.168.100.3/lzp / mntPassword for root@//192.168.100.3/lzp: [root@localhost ~] # cd / mnt/mfs# unzip installation package [root@localhost mfs] # tar zxvf mfs-1.6.27-5.tar.gz-C / opt [root@localhost mfs] # cd / opt/mfs-1.6.27/ [root@localhost mfs-1.6.27] # useradd-s / sbin/nologin Mfs [root@localhost mfs-1.6.27] #. / configure\-- prefix=/usr/local/mfs\ / / specify the installation directory-- with-default-user=mfs\ / / specify the program running user-- with-default-group=mfs\ / / specify the program running group-- disable-mfschunkserver\ / / disable the Chunk function-- disable-mfsmount / / disable mfsmount function [root@localhost mfs-1.6.27] # make & & make install# copy files Configure mfs file [root@localhost mfs-1.6.27] # cd / usr/local/mfs/etc/mfs/ [root@localhost mfs] # cp mfsmaster.cfg.dist mfsmaster.cfg# permissions [root@localhost mfs] # cp mfsexports.cfg.dist mfsexports.cfg# log [root@localhost mfs] # cp mfsmetalogger.cfg.dist mfsmetalogger.cfg [root@localhost mfs] # cd / usr/local/mfs/var/mfs/ [root@localhost mfs] # cp metadata.mfs.empty metadata.mfs# Modify permissions and launch [root@localhost mfs] # chown-R mfs.mfs / usr/local/mfs [root@localhost mfs] # / usr/local/mfs/sbin/mfsmaster start# to check whether the mfs port is enabled [root@localhost mfs] # ps-ef | grep mfs# starts monitoring After all has been built, start [root@localhost mfs] # / usr/local/mfs/sbin/mfscgiserv to check whether the port is open.

Log log 192.168.45.13 "disable firewall [root@localhost ~] # systemctl stop firewalld.service [root@localhost ~] # setenforce" install necessary components [root@localhost ~] # yum install zlib-devel gcc gcc-c++-y # create user [root@localhost ~] # useradd mfs-s / sbin/nologin # mount package [root@localhost ~] # mount.cifs / / 192.168.100.3/lzp / mnt [root@localhost ~] # cd / Mnt/mfs# unzip installation package [root@localhost mfs] # tar zxvf mfs-1.6.27-5.tar.gz-C / opt [root@localhost mfs] # cd / opt/mfs-1.6.27/ [root@localhost mfs-1.6.27] #. / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfschunkserver\-- disable-mfsmount [root@localhost mfs-1.6.27 ] # make & & make install [root@localhost mfs-1.6.27] # cd / usr/local/mfs/etc/mfs/ [root@localhost mfs] # cp mfsmetalogger.cfg.dist mfsmetalogger.cfg [root@localhost mfs] # vim mfsmetalogger.cfg# enable the function on line 15 Then the address points to the master server address MASTER_HOST = 192.168.45.13 modify permissions and start [root@localhost mfs] # chown-R mfs.mfs / usr/local/mfs/ [root@localhost mfs] # / usr/local/mfs/sbin/mfsmetalogger start to check the port opening.

Storage node 192.168.45.136root@localhost ~] # yum install zlib-devel gcc gcc-c++-y [root@localhost ~] # useradd-s / sbin/nologin mfs [root@localhost ~] # mount.cifs / / 192.168.100.3/lzp / mnt [root@localhost ~] # cd / mnt/mfs# decompression installation package [root@localhost mfs] # tar zxvf mfs-1.6.27-5.tar.gz-C / opt [root@localhost mfs] # cd / opt/ Mfs-1.6.27/ [root@localhost mfs-1.6.27] #. / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfsmaster\-- disable-mfsmount [root@localhost mfs-1.6.27] # make & & make install [root@localhost mfs-1.6.27] # cd / usr/local/mfs/etc//mfs/ [root@localhost mfs] # cp mfschunkserver.cfg.dist Mfschunkserver.cfg [root@localhost mfs] # cp mfshdd.cfg.dist mfshdd.cfg# modification On line 12, turn it on and point the address to the master address [root@localhost mfs] # vim mfschunkserver.cfg MASTER_HOST = 192.168.45.133 # set the mount directory [root@localhost mfs] # vim mfshdd.cfg/data [root@localhost mfs] # mkdir / data [root@localhost mfs] # chown-R mfs:mfs / data [root@localhost mfs] # / usr/local/mfs/sbin/mfschunkserver start to check the port opening

Storage node 192.168.45.134 [root @ localhost ~] # yum install zlib-devel gcc gcc-c++-y [root@localhost ~] # useradd-s / sbin/nologin mfs [root@localhost ~] # mount.cifs / / 192.168.100.3/lzp / mnt [root@localhost ~] # cd / mnt/mfs# unzip installation package [root@localhost mfs] # tar zxvf mfs-1.6.27-5.tar.gz-C / opt [root@localhost mfs] # cd / Opt/mfs-1.6.27/# compilation and installation [root@localhost mfs-1.6.27] # / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfsmaster\-- disable-mfsmount [root@localhost mfs-1.6.27] # make & & make install# copy and modify [root@localhost mfs-1.6.27] # cd / usr/local/mfs/etc//mfs/ [root@localhost mfs] # cp mfschunkserver.cfg.dist mfschunkserver.cfg [root@localhost mfs] # cp mfshdd.cfg.dist mfshdd.cfg [root@localhost mfs] # vim mfschunkserver.cfg MASTER_HOST = 192.168.45.133 [root@localhost mfs] # vim mfshdd.cfg/data [root@localhost mfs] # mkdir / data [root@localhost mfs] # chown-R mfs:mfs / data [root@localhost mfs] # / usr/local/mfs/sbin/mfschunkserver start View Port Open status

Client 192.168.45.13 disable firewall [root@manager ~] # systemctl stop firewalld.service [root@manager ~] # setenforce "install necessary components [root@manager ~] # yum install gcc gcc-c++ zlib-devel-y # mount package [root@manager ~] # mount.cifs / / 192.168.100.3/lzp / mnt [root@manager ~] # cd / mnt/mfs/# decompression package [root@manager mfs] # tar zxvf Fuse-2.9.2.tar.gz-C / opt# compile and install [root@manager mfs] # cd / opt/fuse-2.9.2/ [root@manager fuse-2.9.2] #. / configure [root@manager fuse-2.9.2] # make & & make install# modify the system file [root@manager fuse-2.9.2] # vim / etc/profileexport PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH# duplicate Newly load [root@manager fuse-2.9.2] # source / etc/profile# create user [root@manager fuse-2.9.2] # useradd-s / sbin/nologin mfs# extract installation package [root@manager fuse-2.9.2] # cd / mnt/mfs/ [root@manager mfs] # tar zxvf mfs-1.6.27-5.tar.gz-C / opt# compile and install [root@manager mfs] # cd / opt/mfs-1.6. 27 / [root@manager mfs-1.6.27] #. / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfsmaster\-- disable-mfschunkserver\-- enable-mfsmount [root@manager mfs-1.6.27] # make & & make install# create mount point [root@manager mfs-1.6.27] # mkdir / opt/mfs [root@manager mfs-1.6.27] # modprobe fuse # Mount [root@manager mfs-1.6.27] # / usr/local//mfs/bin/mfsmount / opt/mfs-H 192.168.45.13 optimized client [root@manager mfs-1.6.27] # vim / etc/profileexport PATH=/usr/local/mfs/bin:$PATH [root@manager mfs-1.6.27] # source / etc/profile# backup [root@manager mfs-1.6.27] # mfsgetgoal-r / opt/mfs/ View Mount

View the distributed pattern

Access http://192.168.45.133:9425 in a browser

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.