Shulou(Shulou.com)06/03 Report--

Brief introduction

FastDFS is an open source distributed file system similar to Google FS, which is implemented in pure C language and supports UNIX systems such as Linux, FreeBSD, AIX and so on. It can only access files through proprietary API, does not support POSIX interface, and cannot be used by mount. To be exact, Google FS and Google FS like FastDFS, mogileFS, HDFS, TFS are not system-level distributed file systems, but application-level distributed file storage services.

There can also be multiple Tracker server in a FastDFS cluster, and there is no single point of problem for both Tracker server and Storage server. There is a peer-to-peer relationship between Tracker server and between Storage server within a group. In the traditional Master-Slave structure, Master is a single point, and the write operation is only for Master. If Master fails, you need to upgrade Slave to Master, and the implementation logic will be complicated. Compared with the Master-Slave structure, the status of all nodes in the peer-to-peer structure is the same, each node is Master, there is no single point problem.

I. Environmental preparation

Centos7-1: tracker 192.168.142.66

Centos7-2: storage+nginx 192.168.142.77

2. Start installation

(1) install libfastcommon service (all nodes are installed)

Install the base environment package

Yum-y install libevent libevent-devel perl make gcc zlib zlib-devel pcre pcre-devel gcc-c++ openssl-devel

Install libfastcommon

[root@storage ~] # wget https://github.com/happyfish200/libfastcommon/archive/V1.0.38.tar.gz

[root@storage] # tar zxf V1.0.38.tar.gz-C / opt/

[root@storage ~] # cd / opt/libfastcommon-1.0.38/

Compile and establish soft links to facilitate system identification

[root@storage libfastcommon-1.0.38] #. / make.sh &. / make.sh install

[root@storage libfastcommon-1.0.38] # ln-s / usr/lib64/libfastcommon.so / usr/local/lib/libfastcommon.so

[root@storage libfastcommon-1.0.38] # ln-s / usr/lib64/libfdfsclient.so / usr/local/lib/libfdfsclient.so

[root@storage libfastcommon-1.0.38] # ln-s / usr/lib64/libfdfsclient.so / usr/lib/libfdfsclient.so~~

(2) install FastDFS service (all nodes are installed)

Download and install

[root@storage ~] # wget https://github.com/happyfish200/fastdfs/archive/V5.11.tar.gz

[root@storage] # tar zxf V5.11.tar.gz-C / opt/

[root@storage ~] # cd / opt/fastdfs-5.11/

Compile

[root@storage fastdfs-5.11] #. / make.sh &. / make.sh install

Make changes to the template file

[root@storage fastdfs-5.11] # cd / etc/fdfs/ configuration file template path

[root@storage fdfs] # cp tracker.conf.sample tracker.conf

[root@storage fdfs] # cp storage.conf.sample storage.conf

[root@storage fdfs] # cp client.conf.sample client.conf

3) configuration of tracker monitoring side

Establish a directory for storing data files and log files

Mkdir-m 755-p / opt/fastdfs

Modify tracker configuration file

[root@tracker ~] # vim / etc/fdfs/tracker.conf

# modify the following configuration

The default port of port=22122 / / tracker service is 22122.

Base_path=/opt/fastdfs / / tracker stores the following paths of data and log, which must be created in advance

Start the http service process on the http.server_port=8080 / / tracker server without ignoring it

Start the service

[root@tracker ~] # fdfs_trackerd / etc/fdfs/tracker.conf start

[root@tracker ~] # netstat-atnp | grep 22122

Tcp 0 0 0.0.0. 0. 0. 0. 0. 0. 22. 0. 0. 0. 0. 0. 0

[root@tracker ~] # systemctl stop firewalld

[root@tracker ~] # setenforce 0

(4) storage server modification

Establish a directory for storing data files and log files

Mkdir-m 755-p / opt/fastdfs

Modify storage configuration file

[root@storage ~] # vim / etc/fdfs/storage.conf

# modify the following configuration

Group_name=group1 / / default group name, which is modified according to the actual situation

Port=23000 / / storge defaults to 23000, and the storage port number of the same group must be the same.

Root path of base_path=/opt/fastdfs / / storage log file

Store_path_count=1 / / is the same as the number of lower diameters. The default is 1.

Storage path provided by store_path0=/opt/fastdfs / / (stored with log file by default)

Tracker_server=192.168.142.66:22122 / / your own tracker server IP (key point! )

The port of http.server_port=80 / / http access file defaults to 8888. The listening port configured in Ginx keeps the same.

Start the service

# enable the service (command supports start | stop | restart)

[root@storage ~] # fdfs_storaged / etc/fdfs/storage.conf start

[root@storage ~] # netstat-atnp | grep 23000

Tcp 00 0.0.0.0 23000 0.0.0.015 * LISTEN 40430/fdfs_storaged

[root@storage ~] # systemctl stop firewalld

[root@storage ~] # setenforce 0

Check whether the association with the tracker monitoring side is successful

[root@storage ~] # fdfs_monitor / etc/fdfs/storage.conf

Storage 1:

Id = 192.168.142.77

Ip_addr = 192.168.142.77 (storage) ACTIVE

Http domain =

. Omit

(5) install nginx service (storage:192.168.142.77)

Here, in order to reduce the number of virtual machines turned on, nginx is installed on the storage side.

Install nginx and its components

[root@storage] # tar zxf nginx-1.12.0.tar.gz-C / opt/

# download fastdfs-nginx-module installation package

[root@storage ~] # wget https://github.com/happyfish200/fastdfs-nginx-module/archive/V1.20.tar.gz

[root@storage] # tar zxf V1.20.tar.gz-C / opt/

Compile & install

[root@storage ~] # cd / opt/nginx-1.12.0

[root@storage nginx-1.12.0] #. / configure\

-- prefix=/usr/local/nginx\

-- add-module=/opt/fastdfs-nginx-module-1.20/src/ fastdfs-nginx-module module

[root@storage nginx-1.12.0] # make & & make install

Be careful

Possible errors:

/ usr/include/fastdfs/fdfs_define.h:15:27: fatal error: common_define.h: No such file or directory

Solution:

Modify the fastdfs-nginx-module-1.20/src/config file, then reconfigure, compile, and install

Ngx_module_incs= "/ usr/include/fastdfs / usr/include/fastcommon/"

CORE_INCS= "$CORE_INCS / usr/include/fastdfs / usr/include/fastcommon/"

Configure the fastdfs-nginx-module module

[root@storage nginx-1.12.0] # cd fastdfs-nginx-module-1.20/src

[root@storage nginx-1.12.0] # cp mod_fastdfs.conf / etc/fdfs/ move its configuration file to the fdfs directory

# modify fastdfs-nginx-module module configuration file mod-fasts.conf

[root@storage nginx-1.12.0] # cd / etc/fdfs

[root@storage fdfs] # vim mod_fastdfs.conf

# check the configuration

Base_path=/opt/fastdfs / / the path where data files and logs are stored

The address of the tracker_server=192.168.142.66:22122 / / tracker side (key point! )

Url_have_group_name = whether true / / url contains a group name

Storage_server_port=23000 / / needs to be the same as the storage configuration

The number of store_path_count=1 / / storage paths, which needs to match the number of store_path

Location of store_path0=/opt/fastdfs / / file storage

Modify nginx configuration file

[root@storage fdfs] # vim / usr/local/nginx/conf/nginx.conf

# add at blank line

Location ~ / M00 {

Root / opt/fastdfs/data

Ngx_fastdfs_module

}

Copy the http.conf and mime.types in the fastdfs decompression directory

# failure to do this may lead to an error report

[root@storage fdfs] # cd / opt/fastdfs-5.11/conf/

[root@storage conf] # cp mime.types http.conf / etc/fdfs/

(6) modify the client (this experiment is the storage)

Modify the configuration file

[root@storage ~] # vim / etc/fdfs/client.conf

# check the following configuration

Base_path=/opt/fastdfs / / tracker server file path

Tracker_server=192.168.142.66:22122 / / tracker server IP address and port number

The http port number of the http.tracker_server_port=8080 / / tracker server, which must be set with the tracker

III. Test

Upload test file abc.txt

/ usr/bin/fdfs_upload_file / etc/fdfs/client.conf abc.txt

Download the file abc.txt

/ usr/bin/fdfs_download_file / etc/fdfs/client.conf group1/M00/00/00/wKiOTV354W2AIf7GAAAAEh4TEws726.txt abc.txt

Delete the file abc.txt

Fdfs_delete_file / etc/fdfs/client.conf group1/M00/00/00/wKiOTV354W2AIf7GAAAAEh4TEws726.txt

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.