Shulou(Shulou.com)06/02 Report--

SpringBoot how to upload files to FastDFS, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

What is FastDFS

FastDFS is an open source lightweight distributed file system, which solves the problems of large data storage and load balancing, especially suitable for small and medium-sized files (recommended range: 4 KB

< file_size < 500 MB）为载体的在线服务，如相册网站、视频网站等 FastDFS 由 C 语言开发，支持 Linux、FreeBSD 等 UNIX 系统类 Google FS，不是通用的文件系统，只能通过专有 API 访问，目前提供了 C、Java 和 PHP API，为互联网应用量身定做，解决了大容量文件存储问题，追求高性能和高扩展性，FastDFS 可以看做是基于文件的 Key Value Pair 存储系统，称作分布式文件存储服务会更合适。 FastDFS 特性 文件不分块存储，上传的文件和 OS 文件系统中的文件一一对应 支持相同内容的文件只保存一份，节约磁盘空间 下载文件支持 HTTP 协议，可以使用内置 Web Server，也可以和其他 Web Server 配合使用 支持在线扩容 支持主从文件 存储服务器上可以保存文件属性（meta-data）V2.0 网络通信采用 libevent，支持大并发访问，整体性能更好 FastDFS 相关概念 FastDFS 服务端有三个角色：跟踪服务器（Tracker Server）、存储服务器（Storage Server）和客户端（Client） Tracker Server：跟踪服务器，主要做调度工作，起负载均衡的作用。在内存记录集群中所有存储组和存储服务器的状态信息，是客户端和数据服务器交互的枢纽。相比 GFS 中的 Master 更为精简，不记录文件索引信息，占用的内存量很少。 Storage Server：存储服务器（又称存储节点或数据服务器），文件和文件属性（Meta Data）都保存到存储服务器上。Storage Server 直接利用 OS 的文件系统调用管理文件。 Client：客户端，作为业务请求的发起方，通过专有接口，使用 TCP/IP 协议与跟踪器服务器或存储节点进行数据交互。FastDFS 向使用者提供基本文件访问接口，如 upload、download、append、delete 等，以客户端库的方式提供给用户使用。

Tracker is equivalent to the brain of FastDFS. Whether uploading or downloading, resources are allocated through Tracker. Clients can generally use static servers such as Ngnix to call or do part of the cache. The storage server is divided into volumes (or groups). There is a parallel relationship between volumes and volumes, which can be increased at any time according to the use of resources. Server files in the volume are backed up synchronously to achieve the purpose of disaster recovery.

Upload mechanism

First, the client requests the Tracker service to obtain the IP address and port of the storage server

Then the client requests to upload the file based on the returned IP address and port number

After receiving the request, the storage server produces the file, and writes the contents of the file to the disk and returns to the client file_id, path information, file name and other information. The client saves the relevant information and uploads it.

Download mechanism

The client takes the file name information and requests the Tracker service to obtain the IP address and port of the storage server

Then the client requests to download the file according to the returned IP address and port number, and the storage server receives the request and returns the file to the client.

II. Spring Boot Integration FastDFS

The sample project in this section will be built and developed on the basis of the project in the previous section. After uploading the file from the front end to the background, it is passed directly to the FastDFS cluster, and the address of the file storage is returned.

POM package configuration

The Java client that introduced FastDFS:

Org.csource fastdfs-client-java 1.27-SNAPSHOTFastDFS configuration

Add the fdfs_client.conf file to the project resources directory, and the configuration file sets the connection timeout, encoding format, and tracker_server address and other information.

Connect_timeout = 60 # connection timeout network_timeout = 60 # Network timeout charset = UTF-8 # Encoding format http.tracker_http_port = 8080 # tracker Port http.anti_steal_token = no # token hotlink Protection http.secret_key = 123456 # key # tracer server list, if there are multiple tracer server, the branch lists tracker_server = 192.168.53.85:22122tracker_server = 192.168.53.8622122 encapsulated FastDFS upload tool class

Encapsulate FastDFSFile, file basic information including file name, content, file type, author and so on.

Public class FastDFSFile {private String name; private byte [] content; private String ext; private String md5; private String author; / / omit getter, setter}

Next, encapsulate the FastDFSClient class, and FastDFSClient mainly encapsulates the most basic operations, including upload, download, deletion and other methods. The FastDFSFile task can be the encapsulation of FastDFS uploaded files, and each file corresponds to an instance during operation.

First, the configuration information is read and initialized when the class is loaded.

The static {try {String filePath = new ClassPathResource ("fdfs_client.conf") .getFile () .getAbsolutePath (); / / ClientGlobal.init method reads the configuration file and initializes the corresponding properties. ClientGlobal.init (filePath);} catch (Exception e) {logger.error ("FastDFS Client Init Fail!", e);}}

File upload

Use the client storageClient provided by FastDFS to upload the file, and finally return the upload result.

Public static String [] upload (FastDFSFile file) {logger.info ("File Name:" + file.getName () + "File Length:" + file.getContent () .length); / / File attribute information NameValuePair [] meta_list = new NameValuePair [1]; meta_list [0] = new NameValuePair ("author", file.getAuthor ()); long startTime = System.currentTimeMillis (); String [] uploadResults = null; StorageClient storageClient=null Try {/ / get storageClient = getStorageClient (); / / upload uploadResults = storageClient.upload_file (file.getContent (), file.getExt (), meta_list);} catch (IOException e) {logger.error ("IOException when uploadind the file:" + file.getName (), e) } catch (Exception e) {logger.error ("Non IO Exception when uploadind the file:" + file.getName (), e);} logger.info ("upload_file time used:" + (System.currentTimeMillis ()-startTime) + "ms"); / / verify the upload result if (uploadResults = = null & storageClient upload null) {logger.error ("upload file fail, error code:" + storageClient.getErrorCode ()) } / / groupName will be returned if the file is uploaded successfully. Logger.info ("upload file upload file fullyarmed!" + "group_name:" + uploadResults [0] + ", remoteFileName:" + "" + uploadResults [1]); return uploadResults;}

NameValuePair, which mainly stores some basic attributes of the file, such as author information, creation time, etc.

GetStorageClient (), which encapsulates the method of getting the client.

First get the TrackerServer information and use TrackerServer to build the client instance StorageClient for each operation. The detailed code is as follows:

Private static StorageClient getStorageClient () throws IOException {TrackerServer trackerServer = getTrackerServer (); StorageClient storageClient = new StorageClient (trackerServer, null); return storageClient;}

The following is the method for encapsulation to obtain TrackerServer:

Private static TrackerServer getTrackerServer () throws IOException {TrackerClient trackerClient = new TrackerClient (); TrackerServer trackerServer = trackerClient.getConnection (); return trackerServer;}

Get Fil

Get file information based on groupName and file name. Group can also be called a volume. The files on the servers in the same group are exactly the same, and the storage server in the same group is peer-to-peer. File upload, deletion and other operations can be performed on any storage server.

Public static FileInfo getFile (String groupName, String remoteFileName) {try {storageClient = new StorageClient (trackerServer, storageServer); return storageClient.get_file_info (groupName, remoteFileName);} catch (IOException e) {logger.error ("IOException: Get File from Fast DFS failed", e);} catch (Exception e) {logger.error ("Non IOException: Get File from Fast DFS failed", e);} return null;}

Download a file

Get the byte stream of the file according to the API of storageClient and return:

Public static InputStream downFile (String groupName, String remoteFileName) {try {StorageClient storageClient = getStorageClient (); byte [] fileByte = storageClient.download_file (groupName, remoteFileName); InputStream ins = new ByteArrayInputStream (fileByte); return ins;} catch (IOException e) {logger.error ("IOException: Get File from Fast DFS failed", e) } catch (Exception e) {logger.error ("Non IO Exception: Get File from Fast DFS failed", e);} return null;}

Delete a file

Delete the corresponding file according to the file name and group; when using FastDFS, you can directly call the method corresponding to FastDFSClient.

Public static void deleteFile (String groupName, String remoteFileName) throws Exception {StorageClient storageClient = getStorageClient (); int I = storageClient.delete_file (groupName, remoteFileName); logger.info ("delete file uploaded fullylogged classes!" + I);} write upload control classes

Read the file information from MultipartFile, then upload the file to the FastDFS cluster using FastDFSClient, encapsulate a saveFile () method to call the FastDFS utility class encapsulated above, upload the MultipartFile file to FastDFS, and return the address information of the uploaded file.

Public String saveFile (MultipartFile multipartFile) throws IOException {String [] fileAbsolutePath= {}; String fileName=multipartFile.getOriginalFilename (); String ext = fileName.substring (fileName.lastIndexOf (".") + 1); byte [] file_buff = null; InputStream inputStream=multipartFile.getInputStream (); if (inputStreamstreams null) {int len1 = inputStream.available (); file_buff = new byte [len1]; inputStream.read (file_buff);} inputStream.close () FastDFSFile file = new FastDFSFile (fileName, file_buff, ext); try {fileAbsolutePath= FastDFSClient.upload (file); / / upload to fastdfs} catch (Exception e) {logger.error ("upload file Exception!", e);} if (fileAbsolutePath==null) {logger.error ("upload file failed,please upload again!");} String path=FastDFSClient.getTrackerUrl () + fileAbsolutePath [0] + "/" + fileAbsolutePath [1]; return path;}

When the upload request is passed to the backend, the above method saveFile () is called.

@ PostMapping ("/ upload") public String singleFileUpload (@ RequestParam ("file") MultipartFile file, RedirectAttributes redirectAttributes) {if (file.isEmpty ()) {redirectAttributes.addFlashAttribute ("message", "Please select a file to upload"); return "redirect:uploadStatus";} try {String path=saveFile (file) RedirectAttributes.addFlashAttribute ("message", "You successfully uploaded'" + file.getOriginalFilename () + "'"); redirectAttributes.addFlashAttribute ("path", "file path url'" + path + "'");} catch (Exception e) {logger.error ("upload file failed", e);} return "redirect:/uploadStatus";}

After the upload is successful, the path of the file is displayed to the page. The effect image is as follows:

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.