Shulou(Shulou.com)06/03 Report--

In the previous article, high availability and high performance distributed file system FastDFS advanced keepalived+nginx for high availability hot backup of multi-tracker has been introduced to build a highly available distributed file system architecture.

That how to call in the program, in fact, there are many chestnuts on the Internet, here on the basis of them to make a brief introduction.

Download the source code and join the local warehouse

Official website Java client source code: https://github.com/happyfish200/fastdfs-client-java

After opening the source code, execute maven install to type the code into jar to the local maven repository (you can google this step by yourself)

Sample source code

Then create a Demo project, which is created in spring mvc mode.

Add references to pom and rely on jar packages in maven

Org.csource fastdfs-client-java 1.27-SNAPSHOTfastdfs-client.properties

Create the configuration file fastdfs-client.properties in the properties folder of resources

Fastdfs.connect_timeout_in_seconds = 5fastdfs.network_timeout_in_seconds = 30fastdfs.charset = UTF-8fastdfs.http_anti_steal_token = falsefastdfs.http_secret_key = FastDFS1234567890fastdfs.http_tracker_http_port = 80fastdfs.tracker_servers = 10.0.11.201Vl22122, 10.0.11.202Vl22122, 10.0.11.203For FastDFSClient tool class package com.james.utils;import org.csource.common.MyException;import org.csource.common.NameValuePair;import org.csource.fastdfs.*;import java.io.BufferedOutputStream Import java.io.IOException;import java.net.URLDecoder;/** * Created by James on 2015-11-14. * FastDFS file upload * / public class FastDFSClientUtils {private TrackerClient trackerClient = null; private TrackerServer trackerServer = null; private StorageServer storageServer = null; public FastDFSClientUtils (String conf) throws Exception {if (conf.contains ("classpath:")) {String path = this.getClass (). GetResource ("/"). GetPath () Conf = conf.replace ("classpath:", URLDecoder.decode (path, "UTF-8"));} ClientGlobal.init (conf); trackerClient = new TrackerClient (); trackerServer = trackerClient.getConnection (); storageServer = null; storageClient = new StorageClient1 (trackerServer, storageServer);} / * * File upload method *

Title: uploadFile

*

Description:

* * @ param fileName file full path * @ param extName file extension does not contain (.) * @ param metas file extension information * @ return * @ throws Exception * / public String uploadFile (String fileName, String extName, NameValuePair [] metas) {String result = null; try {result = storageClient.upload_file1 (fileName, extName, metas) } catch (IOException e) {e.printStackTrace ();} catch (MyException e) {e.printStackTrace ();} return result } / * upload the file, and upload the disk path name of the fileName * * @ param fileName file, for example: D:/image/aaa.jpg * @ return null is failed * / public String uploadFile (String fileName) {return uploadFile (fileName, null, null) } / * * @ param fileName file disk path name such as: D:/image/aaa.jpg * @ param extName file extension such as txt jpg, etc. * @ return null is failed * / public String uploadFile (String fileName, String extName) {return uploadFile (fileName, extName, null);} / * * upload file method *

Title: uploadFile

*

Description:

* * @ param fileContent file contents, byte array * @ param extName file extension * @ param metas file extension * @ return * @ throws Exception * / public String uploadFile (byte [] fileContent, String extName, NameValuePair [] metas) {String result = null; try {result = storageClient.upload_file1 (fileContent, extName, metas) } catch (IOException e) {e.printStackTrace ();} catch (MyException e) {e.printStackTrace ();} return result } / * * upload file * * @ param fileContent file byte array * @ return null is failed * @ throws Exception * / public String uploadFile (byte [] fileContent) throws Exception {return uploadFile (fileContent, null, null) } / * * upload file * * @ byte array of param fileContent files * @ param extName file extension such as txt jpg png, etc. * @ return null is failed * / public String uploadFile (byte [] fileContent, String extName) {return uploadFile (fileContent, extName, null) } / * File downloaded to disk * * @ param path Picture path * @ param output output stream contains the path to disk * @ return-1 failed, 0 succeeded * / public int download_file (String path, BufferedOutputStream output) {/ / byte [] b = storageClient.download_file (group, path); int result =-1 Try {byte [] b = storageClient.download_file1 (path); try {if (b! = null) {output.write (b); result = 0 }} catch (Exception e) {} / / the user may have cancelled the download of finally {if (output! = null) try {output.close () } catch (IOException e) {e.printStackTrace ();} catch (Exception e) {e.printStackTrace ();} return result } / * get the path of the file array * * @ param path file, such as group1/M00/00/00/wKgRsVjtwpSAXGwkAAAweEAzRjw471.jpg * @ return * / public byte [] download_bytes (String path) {byte [] b = null; try {b = storageClient.download_file1 (path) } catch (IOException e) {e.printStackTrace ();} catch (MyException e) {e.printStackTrace ();} return b } / * * delete file * * @ param group group name such as: group1 * @ param storagePath path name without group name such as: M00/00/00/wKgRsVjtwpSAXGwkAAAweEAzRjw471.jpg * @ return-1 failed, 0 succeeded * / public Integer delete_file (String group, String storagePath) {int result =-1 Try {result = storageClient.delete_file (group, storagePath);} catch (IOException e) {e.printStackTrace ();} catch (MyException e) {e.printStackTrace ();} return result } / * * @ param storagePath file all paths such as: group1/M00/00/00/wKgRsVjtwpSAXGwkAAAweEAzRjw471.jpg * @ return-1 failed, 0 succeeded * @ throws IOException * @ throws Exception * / public Integer delete_file (String storagePath) {int result =-1; try {result = storageClient.delete_file1 (storagePath) } catch (IOException e) {e.printStackTrace ();} catch (MyException e) {e.printStackTrace ();} return result;}}

Java client file upload, download, delete, and metadata acquisition tests:

Package com.james.fdfs;import org.junit.Test;import java.io.File;import java.util.HashMap;import java.util.Iterator;import java.util.Map;import com.james.utils.FastDFSClientUtils;public class FastDFSClientUtilsTest {/ * File upload Test * / @ Test public void testUpload () {File file = new File ("C:\\ Users\\ James\\ Pic\\ share.jpg"); Map metaList = new HashMap () MetaList.put ("width", "1024"); metaList.put ("height", "768"); String fid = FastDFSClientUtils.uploadFile (file,file.getName (), metaList); System.out.println ("upload local file" + file.getPath () + "ok, fileid=" + fid) / / File returned after successful upload: ID: group1/M00/00/00/wKgAyVgFk9aAB8hwAA-8Q6_7tHw351.jpg} / * * File download test * / @ Test public void testDownload () {int r = FastDFSClientUtils.download_file ("group1/M00/00/00/wKgAyVgFk9aAB8hwAA-8Q6_7tHw351.jpg", new File ("DownloadFile_fid.jpg")) System.out.println (r = 0? "download success": "download failed");} / * * File deletion test * / @ Test public void testDelete () {int r = FastDFSClientUtils.delete_file ("group1/M00/00/00/wKgAyVgFk9aAB8hwAA-8Q6_7tHw351.jpg"); System.out.println (r = = 0? "deletion succeeded": "deletion failed");}}

If there are no problems, you will see the printed log.

Net version

For the net version, please refer to another netizen's code:

Https://github.com/huanzui/fastdfs.client.net

problem

Now that the distributed file platform has been built and code tested, there are still several problems in practice:

1, upload to the platform file names are irregular 64base encoded strings, so if only as images and other files stored as no problem, because we do not care about the file name, but if you want to download the content, such as attachments, or installation package, download if still encoded that can not intuitively know what this file is to do, is to convert to the correct file name.

Solution: on the Internet, there is a way to spell out the url through nginx, using the domain name and FID, and then add a parameter after the url to specify the original file name.

For example: http://121.14.161.48:9030/group2/M00/00/89/eQ6h4FKJf_PRl8p4AUz4wO8tqaA688.apk?attname=filename.apk

Configure the following on Nginx so that Nginx intercepts the parameter attname in url and adds the field Content-Disposition "attachment;filename=$arg_attname" to the Http response header.

Here is only one plan, the specific content actually needs a space to introduce, have time to write it.

2. In practice, we want to put different files in different folders according to the business, such as chat files, document files, and temporary files, which sometimes need to be cleaned regularly, but the distributed file platform cannot specify folders.

Solution: the most commonly used approach is to implement a file correspondence library, which corresponds to the uploaded file name, time, corresponding business and other information to the final file path, so that you can do any logic, but the disadvantage is very troublesome.

FastDFS does not see the API saved to the specified level 2 directory, but it can be saved to the specified group, and you can specify which business to use a group for. However, this will destroy the distributed structure of the entire FastDFS, resulting in a group that is very large and not easy to expand. In actual use, there will be other business content into this group.

3. How to continue the transmission of large files at breakpoints?

If the file is greater than 100m, then you need to resume the function of breakpoint. FastDFS is a bit of a struggle for large files, but it can still be achieved. According to the solution provided by netizens, customers are required to upload slices, and the slice byte size is less than or equal to the buff_size configured by storage. The default is 256k. This piece needs to be implemented by itself.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.