Shulou(Shulou.com)06/02 Report--

MFS principle

MFS is a fault-tolerant network distributed file system, which stores data on multiple physical servers and presents a unified resource to users.

1) composition of MFS file system

The composition of the MFS file system is shown in the figure.

Metadata server (Master): responsible for managing the file system and maintaining metadata throughout the system. Metadata log server (MetaLogger): back up the change log file of the Master server, the file type is changelog_ml.*.mfs. When the Master server data is lost or corrupted, you can get the file from the log server and recover it. Data storage server (Chunk Server): the server that actually stores data. When a file is stored, the file is saved in blocks and copied between data servers. The more data servers, the more "capacity" you can use, the higher the reliability, and the better the performance. Client (Client): the MFS file system can be mounted just like NFS, and the operation is the same.

2) the process of reading data by MFS. The client issues a read request to the metadata server. The metadata server informs the client where the required data is stored (the IP address and Chunk number of the Chunk Server).

The client sends data to a known Chunk Server request.

Chunk Server sends data to the client. 3) the process of writing data by MFS. The client sends a write request to the metadata server. The metadata server interacts with the Chunk Server (only if the required chunked Chunks exists), but the metadata server only creates a new chunked Chunks on some servers, and the Chunk Servers informs the metadata server that the operation is successful. The metadata server tells the client which Chunk Server and which Chunks the data can be written to. The client writes data to the specified Chunks Server. The Chunk Server synchronizes data with other Chunk Server. After the synchronization is successful, the Chunk Server informs the client that the data is written successfully. The client informs the metadata server that the write is complete. The main software of host IP address in the experimental environment is Master Server192.168.58.10mfs-1.6.27-5.tar.gzMetaLogger Server192.168.58.11mfs-1.6.27-5.tar.gzChunk Server1192.168.58.12mfs-1.6.27-5.tar.gzChunk Server2192.168.58.13mfs-1.6.27-5.tar.gzClient192.168.58.14mfs-1.6.27-5.tar.gz fuse-2.9.2.tar.gz1. Set up Master Serversystemctl stop firewalld.service / / turn off firewall and enhance security function systemctl disable firewalld.servicesetenforce 0yum install-y zlib-devel gcc gcc-c++ / / install package useradd-s / sbin/nologin-M mfs / / create user mkdir / abc / / create compressed directory mount.cifs / / 192.168.58.22/share / abc / / mount cd / abctar zxvf mfs-1.6.27-5 .tar.gz-C / opt/ / decompression package cd / opt/mfs-1.6.27 / / installation package. / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfschunkserver\-- disable-mfsmountmake & & make install cd / usr/local/mfs/etc/mfs / / copy configuration file cp mfsmaster.cfg.dist mfsmaster.cfgcp Mfsexports.cfg.dist mfsexports.cfgcp mfstopology.cfg.dist mfstopology.cfgcd / usr/local/mfs/var/mfscp metadata.mfs.empty metadata.mfs/usr/local/mfs/sbin/mfsmaster start / start Master Serverps-ef | grep mfs/ / check whether to start the / usr/local/mfs/sbin/mfsmaster-s / / stop command (optional) 2. Set up MetaLogger Serversystemctl stop firewalld.service / / turn off firewall and enhance security function systemctl disable firewalld.servicesetenforce 0yum install-y zlib-devel gcc gcc-c++ / / install package useradd-s / sbin/nologin-M mfs / / create user mkdir / abc / / create compressed directory mount.cifs / / 192.168.58.22/share / abc / / mount cd / abctar zxvf mfs-1.6.27-5 .tar.gz-C / opt/ / decompression package cd / opt/mfs-1.6.27 / / installation package. / configure\-prefix=/usr/local/mfs\-with-default-user=mfs\-with-default-group=mfs\-disable-mfschunkserver\-disable-mfsmountmake & & make installcd / usr/local/mfs/etc/mfscp mfsmetalogger.cfg.dist mfsmetalogger.cfgvi mfsmetalogger.cfg...MASTER_HOST = 192.168 .58.10 / / modified to the IP address of Master Server. / usr/local/mfs/sbin/mfsmetalogger startps-ef | grep mfs/usr/local/mfs/sbin/mfsmetalogger-s / / stop MetaLogger Server (optional) 3. Build Chunk Server1, Chunk Server2systemctl stop firewalld.service / / turn off firewall and enhanced security features systemctl disable firewalld.servicesetenforce 0yum install-y zlib-devel gcc gcc-c++ / / install package useradd-s / sbin/nologin-M mfs / / create user mkdir / abc / / create compressed directory mount.cifs / / 192.168.58.22/share / abc / / mount cd / abctar zxvf mfs-1.6.27-5.tar .gz-C / opt/ / decompression package cd / opt/mfs-1.6.27 / / installation package. / configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- disable-mfsmaster\ / Note the difference between this and the previous-- disable-mfsmountmake & & make installcd / usr/local/mfs/etc/mfscp mfschunkserver.cfg .Dist mfschunkserver.cfgcp mfshdd.cfg.dsit mfshdd.cfgvi mfschunkserver.cfg...MASTER_HOST = 192.168.58.10 / / modified to the IP address of Master Server. Vi mfshdd.cfg/data / / add a line at the end The / data here is a partition for MFS mkdir / data / create the corresponding / data directory chown-R mfs:mfs / data / / modify the master group / usr/local/mfs/sbin/mfschunkserver start / / start Chunk Server/usr/local/mfs/sbin/mfschunkserver-s / / close Chunk Server (optional) 4. Client configuration systemctl stop firewalld.service / / turn off the firewall and enhance security features systemctl disable firewalld.servicesetenforce 0yum install-y zlib-devel gcc gcc-c++ / / install package 4.1.install FUSEtar xzvf fuse-2.9.2.tar.gz-C / opt / / Mount it first Then decompress cd / opt/fuse-2.9.2. / configuremake & & make installvi / etc/profileexport PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH / / add to the end of source / etc/profile4.2 install the MFS client useradd-s / sbin/nologin-M mfscd / abctar zxvf mfs-C / optcd / opt/mfs./configure\-prefix=/usr/local/mfs\-- with-default-user=mfs\-- with- Default-group=mfs\-- disable-mfschunkserver\-- disable-mfsmaster\-- disable-mfschunkserver\-- enable-mfsmount / / enable MFS mount function make & & make installmkdir / opt/mfs / / create mount point modprobe fuse / / load fuse module into kernel / usr/local/mfs/bin/mfsmount / opt/mfs-H 192.168.58.10df-hT / / View mount Load vi / etc/profile / / optimize client export PATH=/usr/local/mfs/bin:$PATH / / add / usr/local/mfs/bin to environment variable source / etc/profile5.MFS monitoring

Mfscgiserv is a Web server written in python. Its listening port is 9425, which can be started by command / usr/local/mfs/sbin/mfscgiserv on Master Server. Users can use the browser to fully monitor all customer mounts, Chunk Server, Master Server, and various client operations.

Http://192.168.58.10:9425 is accessed through a browser on the client

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.