Shulou(Shulou.com)06/02 Report--

Https://pan.baidu.com/s/1RY1emqQHM-Mq3nq1NP5PUw / / Software Baidu network disk connection

MFS distributed file system

File system: ext4, xfs, ext3

MooseFS (moose Moose) is a network distributed file system. It distributes data across multiple servers, but for users, all they see is a source. MFS, like other UNIX-like file systems, contains a hierarchical structure (directory tree) that stores file attributes (permissions, last access and modification time), common special files (block devices, character devices, pipes, sockets), symbolic links, hard links.

MooseFS [MFS] is a network distributed file system with fault tolerance. It distributes data on multiple physical servers, but presents users with a unified resource

When the capacity of our storage server reaches the bottleneck, then we need to use a distributed file system to expand the storage capacity, similar to Baidu Cloud storage, but also using distributed storage.

MFS characteristics:

1. Chromatographic structure (directory tree)

2. Store file attributes (permissions, access and modification time)

3. Support special files (block devices, character devices, pipes)

4. Symbolic links, soft and hard links

5. Access to the file system can be restricted by IP address or password

6. High reliability (multiple copies of data are stored on different computers)

7. Dynamic expansion of capacity can be achieved by adding a new computer or hard disk.

8. Deleted files can be retained according to a configurable time period

9. Coherent snapshots of files that are not affected by access and writes

Application scenario: application scenario of distributed file system

1. Large-scale and highly concurrent data storage and access (small files, large files)

2. Large-scale data processing, such as log analysis

Official website: https://moosefs.com

MFS distributed file system deployment scenario:

Moosefs is a distributed file system, and the MooseFS file system structure consists of the following four roles:

1. Management server managing server (master)

Responsible for the management of each data storage server, file read and write scheduling, file space recovery and recovery, multi-node copy

2. Metadata Log Server Metalogger Server (Metalogger)

Responsible for backing up the change log file of the master server, the file type is changelog_ml.*.mfs, so that it can take over the work when something goes wrong with master server.

3. Data storage server data servers (chunk servers)

Obey the scheduling of the management server, provide storage space, and provide data transmission for customers. A server that actually stores user data.

When storing files, first divide the files into blocks, and then copy these blocks between the data service area chunk server (the number of copies can be specified manually, it is recommended to set the number of copies to 3). There can be multiple data servers, and the greater the number, the greater the "disk space" that can be used and the higher the reliability.

4. Client computers is used for client mount.

The client mounts the storage shared by the remote mfs server and uses it.

The hard disk shared by the data storage server managed on the management server is mounted through the fuse kernel interface.

The usage of a shared file system is similar to nfs.

The client that uses the MFS file system to store and access the host is called MFS. After the MFS file system is mounted successfully, the virtual storage can be shared as before using NFS.

Principle of distributed file system

Distributed file system means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through the computer network. Is to put some scattered (distributed on various computers in the local area network) shared folders, gathered into a folder (virtual shared folder). For users, when they want to access these shared folders, as long as they open the virtual shared folder, they can see all the shared folders linked to the virtual shared folder. Users do not feel that these shared files are scattered on individual computers.

Advantages of distributed file systems:

Centralized access

Simplify operation

Data disaster recovery

Improve file access performance and online expansion

Principle of reading data by MFS

1. The client issues a read request to the metadata server

two。 The metadata server informs the client where the required data is stored (the IP address and Chunk number of Chunk Servert)

3. The client sends data to a known Chunk Serve request

4.Chunk Server sends data to the client

Process:

1. The client sends a write request to the metadata server

two。 The metadata server interacts with the Chunk Server (only if the required chunked Chunks exists), but the metadata server only creates a new chunked Chunks on some servers, and informs the metadata server that the operation is successful

3. The metadata server tells the client which Chunk Server and which Chunks the data can be written to

4. The client writes data to the specified Chunk Server

5. The Chunk Serve synchronizes data with other Chunk Serve, and then the Chunk Server informs the client that the data was written successfully

6. The client informs the metadata server that this write is complete.

Port number:

9420:MFS master and MFS chunk communication port

Communication port between 9421:MFS master and MFS client

Communication port between 9419:MFS master and MFS metalogger

Communication port between 9422:MFS chunk and MFS client

The 9425:MFS master web interface listens to the port to view the overall running status.

System environment

Host operating system ip address package mastercentos7.4192.168.80.100mfsmetaloggercentos7.4192.168.80.101mfschunk01centos7.4192.168.80.102mfschunk02centos7.4192.168.80.103mfschunk03centos7.4192.168.80.104mfsclientcentos7.4192.168.80.105mfs, fuse

First, set up master server1, install relevant compilers, toolkit (all servers must be installed) turn off firewall and SElinuxsystemctl stop firewalldsetenforce 0yum install gcc gcc-c++ make zlib-devel fuse-devel-yyum install lrz*-y2, create process user useradd-M-s / sbin/nologin mfs3, Install mfs package yum install unzip-yunzip moosefs-master.zip-d / opt cd / opt/moosefs-master. / configure-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs make & & make install# compilation and installation 4. Modify related file information cd / usr/local/mfs/etc/mfs/

Cp mfsexports.cfg.dist mfsexports.cfg # output directory configuration file, define mount and permission settings file cp mfsmaster.cfg.dist mfsmaster.cfg # main configuration file cp mfstopology.cfg.dist mfstopology.cfg # metadata log file cd / usr/local/mfs/var/mfs/ cp metadata.mfs.empty metadata.mfs # master metadata file

5. Explain the main configuration file cd / usr/local/mfs/etc/mfs/ vi mfsmaster.cfg # in this file, the annotated symbol is the default configuration # WORKING_USER = mfs # user running master service # WORKING_GROUP = mfs # group running master service # SYSLOG_IDENT = mfsmaster # indicates in syslog that the log is generated by mfsmaster and identified in syslog Indicates whether the # LOCK_MEMORY = 0 # generated by the master service executes mlockall () to avoid master process overflow (default is 0) # NICE_LEVEL =-19 # running priority (default is-19 if possible) Note that the process must be started by root) # EXPORTS_FILENAME = / usr/local/mfs/etc/mfs/mfsexports.cfg # the location of the mounted directory and its permission control files # TOPOLOGY_FILENAME = / usr/local/mfs/etc/mfs/mfstopology.cfg # DATA_PATH = / usr/local/mfs/var/mfs # data store location # BACK_LOGS = 50 # metadata the number of log files changed (default is 50) # BACK_META_KEEP_PREVIOUS = 1 # REPLICATIONS_DELAY_INIT = 30 REPLICATIONS_DELAY_DISCONNECT = 3600 # MATOML_LISTEN_HOST = * # IP address for metalogger listening (default is * For any IP) # MATOML_LISTEN_PORT = 9419 # Port address for metalogger listening (default is 9419) # MATOML_LOG_PRESERVE_SECONDS = 9419 # MATOCS_LISTEN_HOST = * # IP address for chunkserver connections (default is * Represents any IP) # MATOCS_LISTEN_PORT = 9420 # Port IP address for chunkserver connections (default is 9420) # MATOCL_LISTEN_HOST = * # MATOCL_LISTEN_PORT = 9421 # CHUNKS_LOOP_MAX_CPS = 100000 # CHUNKS_LOOP_MIN_TIME = 100000 # chunks Loopback Frequency (default is 300 seconds) # CHUNKS_SOFT_DEL_LIMIT = 1 "CHUNKS_HARD_DEL_LIMIT = 2" CHUNKS_WRITE_REP _ LIMIT = 2 # maximum number of chunk copied to a chunkserver in a loop # CHUNKS_READ_REP_LIMIT = 10 # maximum number of chunk copied from a chunkserver in a loop # ACCEPTABLE_DIFFERENCE = 0.1 # SESSION_SUSTAIN_TIME = 8640 REJECT_OLD_CLIENTS = replication deprecated:# CHUNKS_DEL_LIMIT-use CHUNKS_SOFT_DEL_LIMIT instead# LOCK_FILE-lock system has been changed And this option is used only to search for old lockfile

6. Explain the mounted directory and permission configuration file cd / usr/local/mfs/etc/mfs/ vi mfsexports.cfg # Allow everything but "meta". * / rw,alldirs,maproot=0 # Allow "meta". *. Rw# details according to the display of the file, you can see Each entry in the file is divided into three parts: the first part: the IP address of the client part II: the mounted directory part III: the IP address part of the client with permissions held by the client *: represents all IP addresses 192.168.100.71: represents a single IP address 192.168.100.0amp 24: represents the entire network segment 192.168.100.71-192.168.100.100: represents The directory to which the network segment is mounted /: identifies the MFS root.: identifies the permissions owned by the MFSMETA file system client part ro: represents read-only mode rw: represents read-write sharing alldirs: allows mounting of any specified subdirectory admin: administrator permissions maproot: maps to root Or the specified user Password: specify the client password 7, authorization, optimization path chown-R mfs.mfs / usr/local/mfsln-s / usr/local/mfs/sbin/* / usr/local/bin/8, start the service mfsmaster start

Netstat-anpt | grep mfs

9 、 Edit startup script (I use method 2) method 1: vi / lib/systemd/system/mfs.service [Unit] Description=mfsAfter=network.target [Service] Type=forkingExecStart=/usr/local/mfs/sbin/mfsmaster start # start service ExecStop=/usr/local/mfs/sbin/mfsmaster stop # close service PrivateTmp=true [Install] WantedBy=multi-user.target systemctl enable mfs.service # service boot self-start systemctl stop mfs.service # shut down service systemctl start mfs.service # start service Method 2: echo "/ usr/local/mfs/sbin/mfsmaster start" > > / etc/rc.d/rc.localchmod + x / etc/rc.d/rc.local

Build MetaLogger Server (metadata log server) 1, install relevant compilers, toolkit yum install gcc gcc-c++ make zlib-devel fuse-devel-yyum install lrz*-y2, and create process user useradd-s / sbin/nologin-M mfs

3. Install mfs package yum install-y unzipunzip moosefs-master.zip-d / opt cd / opt/moosefs-master./configure\-- prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs make & & make install4, Modify the main configuration file cd / usr/local/mfs/etc/mfs/ cp mfsmaster.cfg.sample mfsmaster.cfgcp mfsexports.cfg.sample mfsexports.cfgcp mfsmetalogger.cfg.sample mfsmetalogger.cfg cd / usr/local/mfs/var/mfscp metadata.mfs.empty metadata.mfs cd / usr/local/mfs/etc/mfs/vi mfsmetalogger.cfg modify the following: MASTER_HOST = 192.168.80.100 execute the ip address of the master server Remove comment symbol META_DOWNLOAD_FREQ = 24 # backup frequency time 5, authorization, optimize path chown-R mfs.mfs / usr/local/mfsln-s / usr/local/mfs/sbin/* / usr/local/bin/6, start service mfsmetalogger start

Netstat-anpt | grep mfsmetalogger

Echo "/ usr/local/mfs/sbin/mfsmetalogger start" > > / etc/rc.d/rc.localchmod + x / etc/rc.d/rc.local

3. Build Chunk Server (the steps for building three Chunk Server hosts are the same)

Install data servers (chunkservers) with appropriate spare space on their disks and operating systems that follow POSIX standards (verified: Linux, FreeBSD, Mac OS X and OpenSolaris).

When chunkserver stores data, it stores blocks or fragments (chunks/fragments) as files on a common file system such as ext4. You can't see the complete file on chunkserver.

1. Install relevant compiler, toolkit yum install gcc gcc-c++ make zlib-devel fuse-devel-yyum install lrz*-y2, create program user useradd-s / sbin/nologin-M mfs3, install mfs package yum install unzip-yunzip moosefs-master.zip-d / opt cd / opt/moosefs-master. / configure\-prefix=/usr/local/mfs\-with-default-user=mfs\-with-default-group=mfs make & & make install4, Modify the corresponding configuration file cd / usr/local/mfs/etc/mfs/ cp mfschunkserver.cfg.sample mfschunkserver.cfgcp mfshdd.cfg.sample mfshdd.cfg vi mfschunkserver.cfg modify the following: MASTER_HOST = 192.168.80.100 points to the ip address of the master server Remove the annotation symbol MASTER_PORT = 9420 vi mfshdd.cfg and add a line: / data this is a partition for MFS It is best to use a separate partition in the production environment, or mount the disk to this directory. 5. Create the MFS partition directory mkdir / data chown-R mfs:mfs / data/6, license, optimize the path chown-R mfs.mfs / usr/local/mfsln-s / usr/local/mfs/sbin/* / usr/local/bin/7, start the service mfschunkserver start

Netstat-anpt | grep mfschunkserver to see whether port 9422 is working Note: if it is not displayed, try the following command netstat-anpt | grep 9422

Echo "/ usr/local/mfs/sbin/mfschunkserver start" > > / etc/rc.d/rc.localchmod + x / etc/rc.d/rc.local

IV. Configure the client

-

1. Install the relevant compiler, toolkit yum install gcc gcc-c++ make zlib-devel fuse-devel-y4, and create process user useradd-s / sbin/nologin-M mfsyum install lrz*-y

5. Install mfs software package yum install unizp-yunzip moosefs-master.zip-d / opt cd / opt/moosefs-master. / configure\-prefix=/usr/local/mfs\-- with-default-user=mfs\-- with-default-group=mfs\-- enable-mfsmount make & & make install ln-s / usr/lcoal/mfs/bin/* / usr/local/bin6, Mount MFS file system mkdir / opt/mfs # create mount point modprobe fuse # mount fuse module to kernel mfsmount / opt/mfs/-H 192.168.80.100 # Mount MFS

Df-hT # to check the mount

Unzip moosefs-master.zip-d / opt cd / opt/moosefs-master echo "modprobe fuse" > > / etc/rc.d/rc.localecho "/ usr/local/mfs/bin/mfsmount / opt/mfs-H 192.168.80.100" > > / etc/rc.d/rc.local View on chunkserver: yum install tree-ytree / data7, enable web monitoring service mfscgiserv start on master

8. Access MFS monitoring tools

Http://192.168.80.100:9425 # client accesses web

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.