Shulou(Shulou.com)06/03 Report--

1. Introduction to FastDFS

1.1 introduction

FastDFS is an open source, high-performance distributed file system, its main functions include: file storage, synchronization and access, design based on high availability and load balancing, FastDFS is very suitable for file-based services, such as picture sharing and video sharing sites.

FastDFS is a lightweight open source distributed file system, which mainly solves the problems of large-capacity file storage and high concurrent access, and achieves load balancing during file access.

Support online expansion of storage server, support saving only one copy of the same file, saving disk.

FastDFS can only be accessed through Client API, and POSIX access is not supported.

FastDFS is suitable for medium and large websites to store resource files (such as pictures, documents, videos, etc.)

1.2 FastDFS composition

Tracker server

Tracking server: used to schedule requests from the client. And record the information status of all storage groups and storage servers in memory.

Storage server

Storage server: used to store files (data) and file attributes (metadata)

Client

Client: the initiator of the business request, which interacts with tracker and storage server based on TCP protocol through a dedicated interface

Group

Group, also known as volume: the files on the same group are exactly the same

File identification

It consists of two parts: group name and file name (including path)

Meta data

File related attributes: key-value pair (Key Value Pair) mode

Fid

File identifier: (for example: group1/M00/00/00/CgEOxVegXB2AdYafAAAB0b8tBbQ9155303)

Group1: the group name of the storage group. After the upload is completed, the client needs to save it.

M00: virtual path configured by the server, corresponding to the disk option store_path#

000000A two-level directory named after two-digit hexadecimal numbers

CgEOxVegXB2AdYafAAAB0b8tBbQ9155303: file name, which is not the same as the original file name; generated by storage server based on specific information. The file name includes the IP address of the source storage server, file creation timestamp, file size, random number and file extension, etc.

1.3 FastDFS synchronization mechanism

Storage server in the same group is peer-to-peer, and file upload, deletion and other operations can be performed on any storage server.

File synchronization is only carried out between storage server in the same group, using push mode, that is, the source server synchronizes to the target server

The source data needs to be synchronized, and the backup data does not need to be synchronized again, otherwise it will form a loop.

Tip: an exception to the second rule above is that when a new storage server is added, all existing data (including source data and backup data) will be synchronized to the new server by an existing storage server.

1.4 FastDFS characteristics

Index

Description

System conciseness

Very concise, only two roles: tracker, storage system performance is very high, do not use the database, file synchronization directly peer-to-peer, without tracker transit system stability

High, C language development, which can support high concurrency and load

RAID mode

Grouping (intra-group redundancy) with high flexibility

Communication protocol proprietary protocol, download file support HTTP

Attribute (meta,data)

Only one copy of the same content file is supported.

Support file offset when downloading files

Support

2. Background

Here, I deploy with a cluster of three servers, all of which deploy nginx+fastdfs and related modules. The deployment methods of the three servers are all the same, except that some configurations of storage servers will be different. In the process of deployment, please see clearly where to operate. The process is a bit troublesome. Please be patient and deploy OK step by step.

2.1 Environment description

172.18.18.111 (tracker server): nginx, fastdfs (running tracker, storage), fastfds other modules

172.18.18.112 (storage server): nginx, fastdfs (running storage), fastfds other modules

172.18.18.113 (storage server): nginx, fastdfs (running storage), fastfds other modules

2.2 package, installation path

Prepared software packages:

Nginx-1.9.3.tar.gz

Lua-5.1.4.tar.gz

Libfastcommon-1.0.35.zip

Fastdfs-master.zip

LuaJIT-2.1.0-beta2.tar.gz

V0.2.19.tar.gz

GraphicsMagick-1.3.25.tar.gz

GraphicsMagick-1.3.25.tar

Path of installation:

Service Directory nginx/usr/local/nginx/fastdfs configuration

/ etc/fdfs/

Fastdfs

/ usr/local/fastDFS/

Storage_data

/ usr/local/fastDFS/storage/ installation package / usr/src

3. Deploy FastDFS

Install libfastcommon (tracker and storage server operation)

Before installing fastDFS, download the latest version of libfastcommon for installation and upload it to the / usr/src directory. Download address https://github.com/happyfish200/libfastcommon/releases

Extract, configure, and compile:

# cd / usr/src/

# unzip libfastcommon-1.0.35.zip

# cd libfastcommon-1.0.35/

#. / make.sh

#. / make.sh install

Tip: after confirming that there are no errors in make, perform the installation, and the 64-bit system will be copied to / usr/lib64 by default.

Set environment variables and create soft links:

# export LD_LIBRARY_PATH=/usr/lib64/

# ln-s / usr/lib64/libfastcommon.so / usr/local/lib/libfastcommon.so

Install fastdfs (tracker and storage server operation)

Download the latest version of fastDFS and upload it to the / usr/src/ directory. Download address: https://github.com/happyfish200/fastdfs

Decompress, compile:

# cd / usr/src/

# unzip fastdfs-master.zip

# cd fastdfs-master/

#. / make.sh

#. / make.sh install

Tip: after confirming that there are no errors in make, perform the installation, which will be installed into / usr/bin by default, and 4 files ending in .sample will be generated in / etc/fdfs, as shown below:

# ll / etc/fdfs/

-rw-r--r-- 1 root root 1461 Aug 12 15:48 client.conf.sample

-rw-r--r-- 1 root root 7927 Aug 12 15:48 storage.conf.sample

-rw-r--r-- 1 root root 105 Aug 12 15:48 storage_ids.conf.sample

-rw-r--r-- 1 root root 7389 Aug 12 15:48 tracker.conf.sample

# ll / usr/bin/fdfs_*

-rwxr-xr-x 1 root root 317432 Aug 12 15:48 / usr/bin/fdfs_appender_test

-rwxr-xr-x 1 root root 317208 Aug 12 15:48 / usr/bin/fdfs_appender_test1

-rwxr-xr-x 1 root root 304064 Aug 12 15:48 / usr/bin/fdfs_append_file

-rwxr-xr-x 1 root root 303800 Aug 12 15:48 / usr/bin/fdfs_crc32

-rwxr-xr-x 1 root root 304120 Aug 12 15:48 / usr/bin/fdfs_delete_file

-rwxr-xr-x 1 root root 304856 Aug 12 15:48 / usr/bin/fdfs_download_file

-rwxr-xr-x 1 root root 304448 Aug 12 15:48 / usr/bin/fdfs_file_info

-rwxr-xr-x 1 root root 322360 Aug 12 15:48 / usr/bin/fdfs_monitor

-rwxr-xr-x 1 root root 1111640 Aug 12 15:48 / usr/bin/fdfs_storaged

-rwxr-xr-x 1 root root 327376 Aug 12 15:48 / usr/bin/fdfs_test

-rwxr-xr-x 1 root root 326592 Aug 12 15:48 / usr/bin/fdfs_test1

-rwxr-xr-x 1 root root 453880 Aug 12 15:48 / usr/bin/fdfs_trackerd

-rwxr-xr-x 1 root root 305048 Aug 12 15:48 / usr/bin/fdfs_upload_appender

-rwxr-xr-x 1 root root 306072 Aug 12 15:48 / usr/bin/fdfs_upload_file

3.3.Configuring tracker server (operating on tracker server)

Copy the tracker sample configuration file and rename

# cd / etc/fdfs/

# cp tracker.conf.sample tracker.conf

Modify tracker configuration file

# vim tracker.conf

Bind_addr=172.18.18.111 # bind service IP. If left empty, it represents all

Port=22122 # Port on which services are provided

Base_path=/usr/local/fastDFS # Root directory where logs and data are stored

Store_group=group1 # sets this parameter to be a group name that exists in the system when the previous parameter is set to 1 (store_lookup=1, that is, when a group name is specified). If you choose another upload method, this parameter will not work.

Store_server=1 # which storage server is selected for upload operation. 1 indicates that the first server (the one with the lowest IP address) is selected according to the ip address.

Download_server=1 # chooses which storage server is used as the download server. 1 means that the source storage server will be used.

Http.server_port=80 # HTTP service port. The default is 8080. You don't have to change it, but you need to access the test with port 8080.

Other parameters remain in the default configuration. For more information, please see the official documentation: http://bbs.chinaunix.net/thread-1941456-1-1.html.

Create bash_path to specify the path

# mkdir / usr/local/fastDFS/

# mkdir-p / usr/local/fastDFS/storage/data

3.4.Configuring storage servers (tracker and storage server operations)

Copy the sample storage configuration file and rename:

# cd / etc/fdfs/

# cp storage.conf.sample storage.conf

Modify tstorage configuration file

# vim storage.conf

Group_name=group1 # specify the group to which this storage server belongs

Bind_addr= # bind service IP, which is not required (if it is a tracker server, you can also configure the IP address of the tracker server itself)

Base_path=/usr/local/fastDFS # stores the root directory of logs and data

Store_path0=/usr/local/fastDFS/storage # 1st storage directory

IP and port of tracker_server=172.18.18.111:22122 # tracker server, if there are multiple tracker servers, you can write multiple lines

Http.server_port=80 # HTTP service port. The default is 8888. You don't have to change it, but you need to access the test with port 8080.

Other parameters remain in the default configuration. For more information, please see the official documentation: http://bbs.chinaunix.net/thread-1941456-1-1.html.

3. 5 modify tracker server client profile (tracker server operation)

Copy the client sample configuration file and rename

# cd / etc/fdfs/

# cp client.conf.sample client.conf

Modify the client.conf configuration file:

# cd / etc/fdfs/

# cp client.conf.sample client.conf

# vim client.conf

Base_path=/usr/local/fastDFS # modify path

Tracker_server=172.18.18.111:22122 # tracker server IP and port, multiple tracker servers can write multiple configurations

4. File upload test

4.1 start the tracker service (tracker server operation)

# / usr/bin/fdfs_trackerd / etc/fdfs/tracker.conf

Check to see if FastDFS Tracker Server starts successfully:

# ps-ef | grep fdfs_trackerd

Root 27077 25738 0 15:11 pts/2 00:00:00 grep-color=auto fdfs_trackerd

Root 31854 1 0 Nov20? 00:02:34 / usr/bin/fdfs_trackerd / etc/fdfs/tracker.conf

4.2 start the storage service (tracker and storage server operations)

When I get here, some friends may ask me, why do I have to operate on the tracker server when I start the storage service? Have you noticed that in step 2. 1, I declared that the tracker server is running both (tracker and storage), so don't be surprised.

# / usr/bin/fdfs_storaged / etc/fdfs/storage.conf

# ps-ef | grep fdfs_storaged

Root 27234 25738 0 15:15 pts/2 00:00:00 grep-color=auto fdfs_storaged

Root 31863 1 0 Nov20? 00:03:37 / usr/bin/fdfs_storaged / etc/fdfs/storage.conf

4.3 perform file upload (tracker server operation)

I will first put a test picture (test.jpg) to the / home directory:

# ll / home

Drwx-. 14 admin admin 4096 Apr 22 2017 admin

-rw-r--r-- 1 root root 4121391 Dec 7 17:13 test.jpg

Use the fdfs_test command to test the upload in the following format:

# fdfs_test / etc/fdfs/client.conf upload / home/test.jpg

This is FastDFS client test program v5.11

Copyright (C) 2008, Happy Fish / YuQing

FastDFS may be copied only under the terms of the GNU General

Public License V3, which may be found in the FastDFS source kit.

Please visit the FastDFS Home Page http://www.csource.org/

For more detail.

[2017-12-27 15:20:21] DEBUG-base_path=/usr/local/fastDFS, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

Tracker_query_storage_store_list_without_group:

Server 1. Group_name=, ip_addr=172.18.18.111, port=23000

Group_name=group1, ip_addr=172.18.18.111, port=23000

Storage_upload_by_filename

Group_name=group1, remote_filename=M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957.jpg

Source ip address: 172.18.18.111

File timestamp=2017-12-27 15:20:21

File size=4121391

File crc32=4134714274

Example file url: http://172.18.18.111/group1/M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957.jpg

Storage_upload_slave_by_filename

Group_name=group1, remote_filename=M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957_big.jpg

Source ip address: 172.18.18.111

File timestamp=2017-12-27 15:20:21

File size=4121391

File crc32=4134714274

Example file url: http://172.18.18.111/group1/M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957_big.jpg

The following figure shows that the file ID can be returned, indicating that the image was uploaded successfully:

Note: just uploaded the picture of test.jpg to fastdfs, if you want to use a browser to access the url address given above, you also need to install the nginx service.

Use the fdfs_delete_file command to test the deletion (I will not do the deletion test here, but I will use the uploaded image below), as follows:

# fdfs_delete_file / etc/fdfs/client.conf group1/M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957.jpg

4. Install LuaJIT, nginx-lua-module, GraphicsMagick modules (tracker and storage server operations)

4.1 install LuaJIT

Download the latest LuaJIT and upload it to the / usr/src directory:

# cd / usr/src

# wget http://luajit.org/download/LuaJIT-2.1.0-beta2.tar.gz

Decompression, configuration:

# tar zxf LuaJIT-2.1.0-beta2.tar.gz

# cd LuaJIT-2.1.0-beta2

# make

# make install

4.2 install the nginx_devel_kit (NDK) module

Download the latest nginx_devel_kit module and upload it to the / usr/src directory:

# cd / usr/src

# wget http://luajit.org/download/LuaJIT-2.1.0-beta2.tar.gz

Decompression, configuration:

# tar zxf LuaJIT-2.1.0-beta2.tar.gz

# cd LuaJIT-2.1.0-beta2

# make

# make install

4.3 install the lua_nginx_module module

Download the latest nginx_devel_module module and upload it to the / usr/src directory:

# cd / usr/src

# wget https://github.com/openresty/lua-nginx-module/archive/v0.10.2.tar.gz

4.4 install GraphicsMagick module (free software for picture editing, compositing, etc.)

Download the latest GraphicsMagick module and upload it to the / usr/src directory:

# cd / usr/src

# wget ftp://ftp.graphicsmagick.org/pub/GraphicsMagick/1.3/GraphicsMagick-1.3.5.tar.gz

Install the ImageMagick dependency package:

# yum-y install libpng-devel libpng

# yum-y install libjpeg-devel libjpeg

# yum-y install libtool-ltdl libtool-ltdl-devel

# yum-y install ImageMagick ImageMagick-devel

Compile, configure:

# cd GraphicsMagick-1.3.25

#. / configure

# make & & make install

PS: the corresponding files will be installed by default in the following directory:

/ usr/local/bin

/ usr/local/include

/ usr/local/lib

/ usr/local/share

Tip: step 4 can add these modules according to the company's business situation. If your company does not need these modules, this step can be omitted.

5. Install the fastdfs-nginx-module module (installed on all storage nodes, equivalent to operating on tracker and storage servers)

5.1 description of the function of fastdfs-nginx-module

FastDFS stores files on the Storage server through the Tracker server, but file replication is required between the storage servers in the same group, so there is a problem of synchronization delay. Suppose the Tracker server uploads the file to ip01, and the file ID has been returned to the client after the upload is successful. At this point, the FastDFS storage cluster mechanism will synchronize this file to the same group storage ip02. If the file has not been copied yet, if the client uses this file ID to fetch the file on ip02, an error will occur that the file cannot be accessed. On the other hand, fastdfs-nginx-module can redirect the file to connect to the source server to retrieve the file, so as to avoid the file inaccessible error caused by the replication delay on the client. (the unzipped fastdfs-nginx-module is used during nginx installation.)

5.2 download fastdfs-nginx-module

Download the latest fastdfs-nginx-module and upload it to the / usr/src directory:

# cd / usr/src

# wget https://github.com/happyfish200/fastdfs-nginx-module/archive/master.zip

# unzip master

5.3 install the nginx dependency package

# yum-y install gcc gcc-c++ make automake autoconf libtool pcre* zlib openssl openssl-devel

5.3.Compiler and install nginx (add just a few module compilations)

# cd / usr/src

# tar-zxvf nginx-1.9.3.tar.gz

# cd nginx-1.9.3

#. / configure-prefix=/usr/local/nginx-user=www-group=www\

-- with-http_stub_status_module\

-- with-http_realip_module\

-- with-pcre\

-- with-http_ssl_module\

-- with-stream\

-- add-module=/usr/src/fastdfs/ngx_devel_kit-0.2.19/\

-- add-module=/usr/src/fastdfs/lua-nginx-module-0.10.2/\

-- add-module=/usr/src/fastdfs/fastdfs-nginx-module-master/src

# make

# make install

If the following error occurs:

. / configure: error: ngx_http_lua_module requires the Lua library.

Resolve:

Dependency on installing lua: yum install readline-dev readline-devel

Install lua5.1:

Download lux5.1 's package

# tar-zxvf lua-5.1.4.tar.gz

# cd lua-5.1.4

# make linux test

# make install

And then re -:

. / configure

Make & & make install

5.4 copy the configuration file in the fastdfs-nginx-module source code to the / etc/fdfs directory and modify

# cp / usr/src/fastdfs-nginx-module/src/mod_fastdfs.conf / etc/fdfs/

# vim / etc/fdfs/mod_fastdfs.conf

Modify the following:

Base_path=/usr/local/fastDFS # stores the database root directory

Tracker_server=172.18.18.111:22122 # tracker server IP and port

Url_have_group_name = true # url contains group name

Store_path0=/usr/local/fastDFS/storage

Log_filename=/usr/local/fastDFS/logs/mod_fastdfs.log # mod_fastdfs diary storage path

Group_count = 1

Add at the end of the file:

[group1]

Group_name=group1

Storage_server_port=23000

Store_path_count=1

Store_path0=/usr/local/fastDFS/storage

# manually create mod_fastdfs.log log files

# touch / usr/local/fastDFS/log/mod_fastdfs.lo

5.5 copy some configuration files of fastdfs-mast to the / etc/fdfs directory

# cd / usr/src/fastdfs-master/conf

# cp http.conf mime.types / etc/fdfs/

5.6 nginx configuration, startup

Create a www user

# groupadd www

# useradd-g www www

Modify the group permissions of the fastDFS directory:

# chown-R www:www / usr/local/fastDFS/

Modify the nginx.conf file:

# vim / usr/local/nginx/conf/nginx.conf

User www

Worker_processes auto

Server {

Listen 80

Server_name localhost

Location / {

Root html

Index index.php index.html index.htm

Location / group1/M00 {

Ngx_fastdfs_module

Alias / usr/local/fastDFS/storage/data

Set $image_root "/ usr/local/fastDFS/storage/data"

If ($uri ~ "/ ([a-zA-Z0-9] +) / ([a-zA-Z0-9] +) / (. *)") {

Set $image_dir "$image_root/$3/$4/"

Set $image_name "$5"

Set $file "$image_dir$image_name"

}

If ($image_name ~ "([a-zA-Z0-9\ -] +) _ ([0-9] + x [0-9] + | water)? (Q [0-9] {1 a-zA-Z0 2})? ([a-zA-Z0-9] +)") {

Set $a "$1"

Set $b "$2"

Set $c "$3"

Set $d "$4"

Set $e "$5"

Set $f "$6"

Set $file "$file"

}

}

}

Description:

For the configuration http.server_port=8888 in An and / etc/fdfs/storage.conf, the default port is 8888, and I will change it to 80. If you do not modify this default port, then the port that the nginx configuration listens on will also be changed to 8888.

If there are multiple group corresponding to B and Storage, the access path will have a group name, such as / group1/M00/00/00/xxx, and the corresponding Nginx will be configured as follows:

Location ~ / group ([0-9]) / M00 {

Ngx_fastdfs_module

}

Start the nginx service:

# / usr/local/nginx/sbin/nginx-t

Ngx_http_fastdfs_set pid=6715

Nginx: the configuration file / usr/local/nginx/conf/nginx.conf syntax is ok

Nginx: configuration file / usr/local/nginx/conf/nginx.conf test is successful

# / usr/local/nginx/sbin/nginx

6. Use http to test the picture you just uploaded

Http://172.18.18.111/group1/M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957_big.jpg

The test is shown in the following figure:

Here is a little embarrassed, the above is the author's own picture, took a picture on the computer and uploaded, the only picture, of course, readers feel very handsome if you can also collect, !

Then, let's test whether the image can be cropped (if you configure the procedure 4.4), change the red font below to the cropped size, and then visit it in the browser:

Http://172.18.18.111/group1/M00/00/12/rBISb1pDSbWAeUGaAD7jL_Zyu6I957_400x400.jpg

As you can see from the above picture, I can cut the uploaded image here, and the picture is really smaller, it is OK.

7. FastDFS some simple operation commands

# restart the storage service:

# / usr/bin/fdfs_storaged / etc/fdfs/storage.conf restart

# restart the tracker service:

# / usr/bin/fdfs_trackerd / etc/fdfs/tracker.conf restart

# disable the storage service:

# killall fdfs_storaged

# disable the tracker service:

# killall fdfs_trackered

# View cluster status:

# / usr/bin/fdfs_monitor / etc/fdfs/storage.conf

# upload operation:

# specific url of fdfs_test / etc/fdfs/client.conf upload files

# delete operation:

# path after fdfs_delete_file / etc/fdfs/client.conf upload

For example:

# fdfs_delete_file / etc/fdfs/client.conf group1/M00/00/00/ZciEZlepkl6Abj28AAAPOSSdASU225_big.gif)

8. Java API client configuration

1. Go to GitHub to download the Java_client code. Https://github.com/fzmeng/fastdfs.client2. Add the file fastdfs_client.conf under your project src/java/resources

Note to modify the Ip address of the tracker server:

Connect_timeout = 2

Network_timeout = 30

Charset = ISO8859-1

Http.tracker_http_port = 80

Http.anti_steal_token = no

Tracker_server=172.18.18.111:22122

Default_group_name=group1

Of course, there are php and other clients connected to fastdfs, I will not introduce them one by one. Interested friends can discuss on their own Baidu or leave messages. Finally, I would like to thank Master Zhang Qiufang in order to successfully complete this manuscript.

Reference article:

Https://blog.51cto.com/xinzong/1834466

Https://github.com/qieangel2013/nginxLuaGmFastdfs

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.