Shulou(Shulou.com)06/02 Report--

Description:

Architecture planning

Metadata server mfs-master-1 172.16.100.2

Backup server mfs-metalogger 172.16.100.4

Data Storage Server mfs-chunkserver-1 172.16.100.5

Data storage server mfs-chunkserver-2 172.16.100.6

Data Storage Server mfs-chunkserver-3 172.16.100.7

Node description:

Master Server:

Since Master Server controls the components of the entire MooseFS and is responsible for providing services to the outside world, we must ensure that the Master Server is in a very stable state. For example, Master Server uses dual power supply and two-way configuration, and multiple disks use RAID1 or RAID10 for redundancy. As mentioned earlier, Master Server keeps all accessed metadata information in memory to provide user access. As a result, when the number of files increases, so does memory usage. According to official data, 1 million files of chunk information, about 300 megabytes of memory space to do. For disks, Master Server does not use a lot of disks, depending on the number of files and chunk blocks used (recorded in the main metadata file) and the number of operations on files (recorded in the metadata change log). In general, 20G can be used to store 25 million file change records for up to 50 hours. From this point of view, as a Master Server memory is large enough is the top priority.

Metalogger Server:

In the design of MooseFS, although Metalogger Server is only used to collect backups of metadata (changes in file changes) of the MooseFS master server, the hardware requirements should not be higher than the backup of the master server. However, it should be noted that if Master Server is not highly available, after the primary server goes down, we need to enable Metalogger Server to replace the primary server. So, from this point of view, Metalogger Server is at least the same configuration as Master Server, and keep in mind!

Chunk Server:

For Chunk Server, it is the real carrier for storing data. Therefore, our requirements for it are simple and rough, as long as the performance of the hard drive is guaranteed. If it is an ordinary business, you can choose multiple disks to do RAID5, of course, RAID0 or RAID10 is OK. It is important to note that due to the problem with MooseFS's default load balancing algorithm, I recommend that the disk sizes of all Chunk Server remain the same. In this way, we can ensure that the data usage of each MooseFS node is roughly the same during the use of Chunk. Otherwise, the use of Chunk Server with large disk capacity will increase, while the usage of Chunk Server with small disk capacity will become smaller. Remember, remember! Of course, if the employees of the company have the ability, they can also improve the algorithm of adding carry variables each time in MooseFS's load balancing algorithm, to avoid the shortcomings of the default algorithm, so that the stored data can be evenly distributed on each Chunk Server.

Deployment:

Deploy Master Server

1. Parameter introduction

-- disable-mfsmaster # is not created as a management server (for installation of pure nodes)

-- disable-mfschunkserver # do not create a data storage chunkserver server

-- disable-mfsmount # does not create mfsmount and mfstools (if installed with a development package, they will be created by default)

-- enable-mfsmount # make sure to install mfsmount and mfstools (if

-- prefix=DIRECTORY # locks the installation directory (default is / usr/local)

-- sysconfdir=DIRECTORY # Select the profile directory (default is ${prefix} / etc)

-- localstatedir=DIRECTORY # Select variable data directory (default is ${prefix} / var, MFS metadata is stored in the subdirectory of mfs, default is ${prefix} / var/mfs)

-- with-default-user # the user running the daemon. If no user is set in the configuration file, it defaults to nobody user.

-- with-default-group # the user group that runs the daemon. If no user group is set in the configuration file, it defaults to nogroup user group.

2. Install Master Server

Yum install zlib-devel-y

Groupadd-g 1000 mfs

Useradd-u 1000-g mfs-s / sbin/nologin mfs

Cd / usr/local/src

Wget http://moosefs.org/tl_files/mfscode/mfs-1.6.27-5.tar.gz

Tar zxf mfs-1.6.27-5.tar.gz

Cd mfs-1.6.27

. / configure-prefix=/usr/local/mfs-1.6.27-with-default-user=mfs-with-default-group=mfs-disable-mfschunkserver-disable-mfsmount

Make

Make install

Ln-s / usr/local/mfs-1.6.27 / usr/local/mfs

Official documents:

Https://moosefs.com/download/centosfedorarhel.html

Version 6:

Curl "http://ppa.moosefs.com/MooseFS-3-el6.repo" > / etc/yum.repos.d/MooseFS.repo"

Curl "http://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > / etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS"

Version 7:

Curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > / etc/yum.repos.d/MooseFS.repo"

Install Master Server

Yum install moosefs-master moosefs-cli moosefs-cgi moosefs-cgiserv

/ etc/init.d/moosefs-master start

View log, port, firewall open port 9419 9420 9421

Install Chunkservers

Yum install moosefs-chunkserver

Install Metaloggers

Yum install moosefs-metalogger

Install Clients

Yum install moosefs-client

Boot and mount automatically:

Yum install fuse

Vim / etc/fstab

Mfsmount / mnt/mfs fuse defaults 0 0

Mfsmaster.host.name: / mnt/mfs moosefs defaults 0 0

Start the service

Service moosefs-master start

Service moosefs-chunkserver start

Write file test

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.