Shulou(Shulou.com)06/01 Report--

This article mainly explains the "Glusterfs directory ls performance optimization method is what", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in-depth, together to study and learn "Glusterfs directory ls performance optimization method is what" it!

Purpose and optimization ideas

This paper discusses the existing optimization measures and possible further optimization schemes of glusterfs on the performance of file system crawler rsync/ls directory. The optimization idea is to reduce the metadata operation of the local file system, reduce the load of fuse client, reduce the network polling times of req, reduce the network communication time, cache prefetching, concurrent, asynchronous, bulk transmission.

Fuse readdirplus

The latest kernel of centos 6.4.It supports fuse readdirplus. Fine-tune the mount timeout parameters.

FUSE: Adaptive NFS-like readdirplus support http://lwn.net/Articles/532705/?format=printable

Http://sourceforge.net/p/fuse/mailman/message/29562240/

Readdir-ahead

This has been merged into 3.5release. This paper mainly optimizes the sequential directory reading of single-thread ls. By grabbing the directory item as the cache in advance during opendir (during callback), it will also grab the inode attribute and extended attribute of the directory entry when glusterfs readdirplus is supported (kernel and glusterfs fuse support). When readdir (p), you can directly return or grab the directory entry cache at this time. Performance will be more optimized based on fuse readdirplus. Caching based on the fd lifecycle. The function is not very powerful and needs to be strengthened.

FUSE_AUTO_INVAL_DATA

The-- fopen-keep-cache mount option is enabled based on kernel FUSE_AUTO_INVAL_DATA support. Supported by the new kernel version. The default behavior of glusterfs fuse is that write operations are synchronous and read file operations are read from page cache. When you open a file, the page cache of the original file is disabled. When this option is enabled, no invalidation operation can be performed if the original file page cache content has not changed as needed.

Quick-read

Glusterfs3.4 decomposes quick-read (3. 3 only translaotr) into openbehind and quick-read. It turns out that no matter what the purpose of manipulating the file, the original design is to get the real fd. After refactoring, according to the purpose of the file operation, if it is to modify the contents of the file, open the file in the background and operate. If it is just a similar operation such as fstat, it will be done using anonymous fd instead of waiting for the real fd. In this way, the performance is optimized according to the purpose of operation. When you lookup, set xdata key as needed, and grab the contents of the file in the posix translator layer. The read operation returns the contents of the file when it is executed to the quick-read layer.

Md-cache

It is mainly inode attr and xattr that grab when readdir (p); lookup grabs only the inode attribute of the directory or file being operated at that time, not all directory entries. This translator optimizes the latency caused by stat and extended attribute fetching during ls. However, at present, we generally turn off the support for selinux and acl extended properties, so the ls optimization of extended properties does not work for the time being.

Other possible effects of translator, to be analyzed

Io-threads server and client settings

Libaio

Scatter-gather IO

Further optimization direction

The fuse kernel currently supports a 4k readdir buffer size. You can modify the kernel code to support buffer with a larger chunk. Readdir-ahead uses a glusterfs rpc 128k buffer for bulk fetch, but only prefetches in user space. Brian Foster has carried on the optimization experiment in this aspect.

Strengthen readdir-ahead to make a powerful client cache architecture, first do directory item caching, and then consider others.

Multi-threaded, non-sequential directory reading

Caching is based on inode and persistent caching

Xavier Hernandez proposes a lock-free architecture that replaces inodelk/entrylk to help implement a powerful cache in client. So far, the community has had a brainstorm to discuss the caching architecture. I'm following up.

Dht read directories are in order (read by a brick). We should analyze whether this restriction can be relaxed.

Small files are merged into transtlaotr of large files. You can refer to the implementation of hystack and tfs.

Refer to the central cache architecture of hdfs, do not do the real cache in client, but cache on the brick side, client only does routing. Or both client and brick do caching.

Tiered storage. This glusters is already working on it.

Thank you for reading, the above is the "Glusterfs directory ls performance optimization method is what" the content, after the study of this article, I believe you on the Glusterfs directory ls performance optimization method is what this problem has a deeper understanding, the specific use of the situation also needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.