Shulou(Shulou.com)06/02 Report--

This article is to share with you what are several common problems in the use of FastDFS, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article. Let's take a look at it with the editor.

1. Scenarios suitable for FastDFS and scenarios that are not applicable

FastDFS is a distributed file storage system tailor-made for Internet applications, which is very suitable for storing pictures, audio, video, documents and other files. For Internet applications, simple and efficient FastDFS has obvious advantages over other distributed file systems. For details, you can consult the relevant introduction documents, such as: FastDFS architecture design documents and so on.

For the sake of brevity, FastDFS does not store files in blocks, so it is not suitable for distributed computing scenarios.

two。 Server time must be consistent

Because FastDFS's ingenious design does not need to store file indexes, FastDFS compares timestamps to determine whether files are synchronized. Therefore, the time of the servers in the cluster should be the same, and the time difference between each server should not exceed 1 second. It is recommended to use NTP timing service.

3. Too many open files error resolution

The error of opening too many files is reported in the log because the system allows a process to open too many files. For the solution in Linux environment, modify the file / etc/security/limits.conf and add the following code at the end of the file (if it already exists, modify the corresponding value):

Root soft nofile 65535

Root hard nofile 65535

* soft nofile 65535

* hard nofile 65535

Note: just configure the last two lines, why configure it separately for root users? After checking the information on the Internet, it is said that wildcards like * are not valid for root users, so root needs to be configured separately (well, Aliyun ECS is configured with the above four lines).

4. Start and stop FastDFS service

The FastDFS server program comes with start, stop, and restart instructions, and the command line example is as follows:

/ usr/bin/fdfs_trackerd / etc/fdfs/tracker.conf [start | stop | restart]

/ usr/bin/fdfs_storaged / etc/fdfs/storage.conf [start | stop | restart]

You can use kill or killall to kill fdfs_trackerd and fdfs_storaged processes normally, but

Do not add the-9 parameter forced kill, otherwise it may lead to problems such as binlog data loss.

5. Does FastDFS support resuming upload from breakpoints?

Both uploading and downloading files are supported.

For file upload, you need to upload a file of type appender first, and then use the apend method.

If you want to upload a large file that exceeds 1GB, it is recommended to upload multiple files using append, such as 64MB each time. You need to create a file of type appender first, and you can create an empty appender file.

For super-large files, if you want to support multi-thread upload to speed up the upload speed, you can use the following three steps:

1) upload files of appender type

2) call the truncate method to set the appender file to the final file size

3) call the modify method to upload file fragments concurrently.

For file downloads, FastDFS can specify the file offset and the size of the file content obtained. Using this feature, file downloading can achieve breakpoint continuation and multithreaded downloading.

6. Java SDK is not thread safe

The Java SDK provided by FastDFS is not thread-safe, and someone has stepped on it. The three classes, including TrackerClient, which is responsible for interacting with tracker server, StorageClient, and StorageClient1, which communicate directly with storage server, are all non-thread safe.

Why do two class names with the word StorageClient appear? The two implement exactly the same function, StorageClient is the use of the separation of group and filename, and StorageClient1 is the combination of group and filename (file ID). It is usually fine to use StorageClient1.

These are several common problems in the use of FastDFS. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.