Shulou(Shulou.com)06/03 Report--

Background

FastDFS is an open source lightweight distributed file system, tailored for the Internet, fully considering redundant backup, load balancing, linear expansion and other mechanisms, and paying attention to high availability, high performance and other indicators to solve the problems of mass storage and load balancing. It is especially suitable for online services based on files, such as photo albums, video websites and so on. For Internet applications, compared with other distributed file systems, the advantage is very obvious.

For the sake of brevity, FastDFS does not store files in blocks, so it is not suitable for distributed computing scenarios.

Once again, thanks to Taobao senior architect Yu Qing for opening up such an excellent lightweight distributed file system, this article records the installation and configuration of the latest version of FastDFS 5.11 in CentOS7.

Related links

Download address: http://sourceforge.net/projects/fastdfs/files

Environmental preparation

Operating system: CentOS7

Two servers: 192.168.53.141, 192.168.53.142

Install the package:

Fastdfs-5.11.tar.gzlibfastcommon-1.0.36.tar.gzfastdfs-nginx-module-master.zipnginx-1.14.0.tar.gz

Server Planning:

Tracker server: 192.168.53.141

Storage server: 192.168.53.142

Installation package acquisition

This installation is the latest version of the current update, and the installation packages used are from the author's last releases on github.

Get the FastDFS installation package

At present, the author's last releases is on June 3, 17, and the latest version is 5.11, which is downloaded directly from Yu Da's GitHub:

Https://github.com/happyfish200/fastdfs/releases

Get the libfastcommon installation package

At present, the author's last releases is on April 5, 17, and the latest version is 1.0.36, which is downloaded directly from Yu Da's GitHub:

Https://github.com/happyfish200/libfastcommon/releases

Get the fastdfs-nginx-module installation package

Get the latest code from github

Https://github.com/happyfish200/fastdfs-nginx-module

Get the nginx installation package

Http://nginx.org/en/download.html

After obtaining the latest installation package, upload it to the / usr/local/src/ directory.

Installation instructions

FastDFS is divided into tracker (tracker) and storage (memory)

The tracker tracker mainly does scheduling work and plays the role of load balancing on access.

Storage storage, files and meta data are all stored on the storage server.

The group group and volume can be expanded horizontally outside the group, and the files on the servers in the group backup each other, which is exactly the same.

Both servers need to have libfastcommon and FastDFS installed, with different configurations for tracker and storage.

Install FastDFS (all)

The two servers we prepared need to install the FastDFS base environment.

Installation dependency

# installation depends on yum-y install gcc-c++ perl

Install the libfastcommon class library

FastDFS 5.x cancels the dependence on libevent and adds dependence on libfastcommon. To install FastDFS, you must install the libfastcommon class library.

Tar-zxvf libfastcommon-1.0.36.tar.gzcd libfastcommon-1.0.36./make.sh./make.sh install

Execute the. / make.sh command to compile. If there is no error information, the compilation is successful. Finally, execute the. / make.sh install to install. When you see a message similar to the following, libfastcommon has been installed successfully:

Create a soft connection

Because the lib directory set by the FastDFS main program is / usr/local/lib, you need to create a soft link (the installation of the FastDFS main program depends on this kind of library, and the installation will fail if you do not create the soft connection)

Ln-s / usr/lib64/libfastcommon.so / usr/local/lib/libfastcommon.so

Install FastDFS

Tar-zxvf fastdfs-5.11.tar.gzcd fastdfs-5.11./make.sh./make.sh install

Generally speaking, if there is no error message, the installation is successful. The installation log is:

Related files and directories

We install it by default, and the corresponding files and directories are as follows:

Service script:

/ etc/init.d/fdfs_storaged

/ etc/init.d/fdfs_tracker configuration file (sample profile):

/ etc/fdfs/client.conf.sample

/ etc/fdfs/storage.conf.sample

/ etc/fdfs/storage_ids.conf.sample

The / etc/fdfs/tracker.conf.sample command tool is located in the / usr/bin/ directory:

Fdfs_appender_test

Fdfs_appender_test1

Fdfs_append_file

Fdfs_crc32

Fdfs_delete_file

Fdfs_download_file

Fdfs_file_info

Fdfs_monitor

Fdfs_storaged

Fdfs_test

Fdfs_test1

Fdfs_trackerd

Fdfs_upload_appender

Fdfs_upload_file

Stop.sh

Restart.sh configure FastDFS Tracker (192.168.53.141)

Copy tracker.conf.sample and rename it to tracker.conf

Cd / etc/fdfs/cp tracker.conf.sample tracker.conf

Edit configuration file

Vi / etc/fdfs/tracker.conf

The main information and changes are as follows:

Disabled=false # defaults to false. If it is not modified, it is not enabled. = false is enabled. This is the design, so don't complain for the time being. The port number of port=22122 # tracker is 22122 by default. The production environment can be modified by itself. Here, I temporarily use the default port base_path=/fastdfs/tracker # directory address to set according to the actual mounted data disk. The root directory must exist and the subdirectory will be created automatically.

You can keep the default configuration for other parameters. For more information, please see the official documentation:

Http://bbs.chinaunix.net/thread-1941456-1-1.html

Create a basic data directory (the path corresponding to base_path)

Mkdir-p / fastdfs/tracker

Open the tracker port in the firewall (default is 22122)

Firewall-cmd-permanent-zone=public-add-port=22122/tcpfirewall-cmd-reload

Start tracker

/ etc/init.d/fdfs_trackerd start

Check to see if tracker starts successfully

Ps-ef | grep fdfs

The following stop and set boot start is not a necessary operation, I do not want to restart the server every time manually start again, add boot boot!

Stop tracker

/ etc/init.d/fdfs_trackerd stop

Set up boot boot

Vi / etc/rc.d/rc.local

Add the following:

# # FastDFS Tracker/etc/init.d/fdfs_trackerd start

Because in centos7, the permissions of the / etc/rc.d/rc.local file have been reduced and there is no execute permission, you need to add executable permissions to it.

Chmod + x / etc/rc.d/rc.local

Configure FastDFS storage (192.168.53.142)

Copy storage.conf.sample and rename it to storage.conf

Cd / etc/fdfs/cp storage.conf.sample storage.conf

Edit configuration file

Vi / etc/fdfs/storage.conf

The main information and changes are as follows:

Disabled=false # defaults to false. If it is not modified, it is not enabled. = false is enabled. The design is like this, and I won't complain here for the time being. Port=23000 # memory port number. The default is 23000. The production environment can modify it. Here, I temporarily use the default port base_path=/fastdfs/storage # directory address to set according to the actual mounted data disk. The root directory must exist, and the subdirectory will automatically create store_path0=/fastdfs/storage # storage server supports multiple paths when storing files. If base_path0 is not configured, Over there, it is the same as the path corresponding to base_path. The list of tracker_server=192.168.53.141:22122 # tracker_server should be written (again, active connection to tracker_server) the http.server_port=8888 # HTTP service port defaults to 8888, which can be modified by the production environment. Here, I will temporarily use the default port.

You can keep the default configuration for other parameters. For more information, please see the official documentation:

Http://bbs.chinaunix.net/thread-1941456-1-1.html

Create a basic data directory (the path corresponding to base_path)

Mkdir-p / fastdfs/storage

Open the tracker port in the firewall (default is 23000)

Firewall-cmd-permanent-zone=public-add-port=23000/tcpfirewall-cmd-reload

Start storaged

/ etc/init.d/fdfs_storaged start

(initial startup generates data and logs directories under the / fastdfs/storage directory)

Check to see if storage starts successfully

Ps-ef | grep fdfs

The following stop and set boot start is not a necessary operation, I do not want to restart the server every time manually start again, add boot boot!

Stop storaged

/ etc/init.d/fdfs_storaged stop

Set up boot boot

Vi / etc/rc.d/rc.local

Add the following:

# # FastDFS Storaged/etc/init.d/fdfs_storaged start

Because in centos7, the permissions of the / etc/rc.d/rc.local file have been reduced and there is no execute permission, you need to add executable permissions to it.

Chmod + x / etc/rc.d/rc.local

Test file upload (192.168.53.141)

Modify the client profile in the Tracker server

Cp / etc/fdfs/client.conf.sample / etc/fdfs/client.confvi / etc/fdfs/client.conf

The main information and changes are as follows:

Base_path=/fastdfs/tracker

Tracker_server=192.168.53.141:22122

Perform upload naming

/ usr/bin/fdfs_upload_file / etc/fdfs/client.conf / usr/local/src/fastdfs-5.11.tar.gz

Return ID number: group1/M00/00/00/wKg1jlrfY6-Ad3ssAAUkKwe5sE4.tar.gz

(the first parameter is the upload configuration file, and the second parameter is the file to be uploaded. The ID of the above file can be returned, indicating that the file was uploaded successfully)

Install nginx on the storage node

Install nginx and fastdfs-nginx-modulemok modul

Extract the file

Cd / usr/local/src/unzip fastdfs-nginx-module-master.ziptar-zxvf nginx-1.14.0.tar.gz

Install the dependency packages required to compile Nginx

Yum-y install pcre* zlib zlib-devel

Compile and install Nginx (add fastdfs-nginx-module module)

. / configure-- add-module=/usr/local/src/fastdfs-nginx-module-master/srcmake & & make install

Copy the configuration file in the fastdfs-nginx-module source code to the / etc/fdfs directory and modify

Cp / usr/local/src/fastdfs-nginx-module-master/src/mod_fastdfs.conf / etc/fdfs/vi / etc/fdfs/mod_fastdfs.conf

Modify the following configuration:

Connect_timeout=10 # defaults to 2 to 102 seconds too few base_path=/tmptracker_server=192.168.53.141:22122 # tracker tracker address storage_server_port=23000 # storage memory port group_name=group1 # group name url_have_group_name = true # url hava group name default to false to truestore_path0=/fastdfs/storage # memory storage address

Copy some of the configuration files from FastDFS to the / etc/fdfs directory

Cd / usr/local/src/fastdfs-5.11/conf/cp http.conf mime.types / etc/fdfs/

Create a soft connection under the / fastdfs/storage file storage directory and link it to the directory where the data is actually stored

Ln-s / fastdfs/storage/data/ / fastdfs/storage/data/M00

Configure nginx

Sample brief configuration:

User root;worker_processes 1 leading events {worker_connections 1024;} http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65 leading server {listen 8888; server_name localhost; location ~ / group ([0-9]) / M00 {# alias / fastdfs/storage/data; ngx_fastdfs_module;} error_page 500502 503 504 / 50x.hml; location = / 50x.html {root html }}}

Firewall opens port 8888

Firewall-cmd-permanent-zone=public-add-port=8888/tcpfirewall-cmd-reload

Access the address of the file you just tested and uploaded after starting nginx

Http://192.168.53.142:8888/group1/M00/00/00/wKg1jlrfY6-Ad3ssAAUkKwe5sE4.tar.gz

The file can be downloaded normally, and the installation and configuration of the fdfs single node is complete.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.