Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of common file systems under Linux, which has a certain reference value, and interested friends can refer to it. I hope you can learn a lot after reading this article.

History

File system creator created the platform ext2R é my Card1993Linux,HurdXFSSGI1994IRIX, Linux, FreeBSDext3Dr. Stephen C. Tweedie1999LinuxZFSSun2004Solarisext4 many developers 2006LinuxBtrfsOracle2007Linux

From the creation time, we can see that they live in different times, because the implementation of Btrfs is borrowed from ZFS, so ZFS is also listed here as a reference.

Size limit

File system * * File name length * File size * Partition size ext2255 bytes2 TB16 TBext3255 bytes2 TB16 TBext4255 bytes16 TB1 EBXFS255 bytes8 EB8 EBBtrfs255 bytes16 EB16 EB

* the file and partition size is affected by the block size (block size) used to format the partition. The larger the block, the larger the supported * * file and partition, the more likely it is to waste disk space. The data listed in the above table is based on the block size of 4K.

Code size

From the code size, you can see the functional richness and complexity of the file system. The data listed below comes from kernel-4.1-rc8, which is simply counted by wc-l, without filtering blank lines, comments, and so on.

File system source file (.c) header file (.h) ext283631016ext3164961567ext4446504522XFS8960515091Btrfs1052547933

Btrfs is still in the process of rapid development, and the number of lines of code may change greatly.

Both XFS and Btrfs use B-tree

Ext2

The advantage of ext is that it is relatively simple and has better performance when there are fewer files. It is more suitable for scenarios with fewer files. The main disadvantages are as follows.

The number of inode is fixed. You can specify the proportion of space occupied by inode and data blocks when formatting the partition, but once it is formatted, it cannot be changed later.

When the block size is 4K, the single file size cannot exceed 2TB, and the partition size cannot exceed 16TB. (currently, hard disk sizes are usually only a few TB, so it's not a big problem.)

There can be at most 32000 subdirectories in a directory.

Because the files and subdirectories stored in the directory are organized in a linear manner, it is not efficient to traverse the directory, especially when the number of files under the directory reaches more than 10K, the speed will be significantly slower.

When the underlying disk partition space becomes larger (which is common when using LVM), ext2 cannot dynamically expand to use the increased space

There is no Journal function, so the security of the data is not high

Ext3

Ext3 implements the following functions on the basis of ext2, while others remain unchanged, that is, ext3 also has the disadvantages of ext2.

Journal function is supported, and the security of data is greatly improved compared with ext2.

When the underlying partition space becomes larger, ext3 can automatically expand to use the increased space

Use HTree to organize the files and subdirectories in the directory, so that the number of files and subdirectories in the directory is no longer limited by performance (there will be no performance problems if the number exceeds 10K)

Ext4

Ext4 draws lessons from some current mature file system technologies, adds some functions to ext3, and makes some performance improvements, the main changes are as follows

When the block size is 4K, the supported * * file and * * partition sizes reach 16TB and 1EB, respectively.

No longer subject to the limit of 32000 subdirectories, an unlimited number of subdirectories is supported

Support Extents to improve the operation performance of large files

The internal implementation supports the allocation of multiple data blocks at a time, which is better than that of ext3.

Supports deferred allocation (that is, supports the fallocate function) (fallocate is a function of libc, and on file systems that do not support this function, libc creates a file that takes up disk space)

Support for online fast scanning

Support for online defragmentation (individual files or entire partitions)

Log (Journal) supports check code (checksum), and the security of data is further improved.

No logging (No Journaling) mode is supported (ext3 does not support this feature), which, like ext2, eliminates the impact of logging on performance

Support for nanosecond timestamps

The creation time of the file is recorded. Because the relevant application layer tools are not supported yet, the creation time of the file can only be seen through debug.

Here is an example of viewing the creation time of the file / etc/fstab (the file exists on the / dev/sda1 partition):

Dev@ubuntu:~$ ls-I / etc/fstab 10747906 / etc/fstab dev@ubuntu:~$ sudo debugfs-R 'stat' / dev/sda1 Inode: 10747906 Type: regular Mode: 0644 Flags: 0x80000 Links: 1 Blockcount: 8 ctime: 0x5546dc54:6e6bc80c-- Sun May 3 22:41:24 2015 atime: 0x55d1b014:8bcf7b44-- Mon Aug 17 05:57:40 2015 mtime: 0x5546dc54:6e6bc80c-- Sun May 3 22:41:24 2015 crtime: 0x5546dc54:6e6bc80c-Sun May 3 22:41:24 2015 Size of extra inode fields: 28 EXTENTS: (0): 46712815

Extents: in the original ext2 file system, data blocks are all managed separately. Pointers to data blocks are stored in inode. There are as many pointers (multi-level) in inode as many data blocks are occupied by files. Imagine a 1G file with a block size of 4K. You need (1024 * 1024) / 4cm 262144 data blocks, that is, 262144 pointers are needed. Initialize these pointers when creating a file. These pointers need to be recycled when deleting files, affecting performance. Modern file systems support the function of Extents, to put it simply, Extent is a collection of data blocks. Previously, you can allocate a data block at a time, but now you can allocate an Extent at a time, which contains a lot of data blocks. At the same time, inode only needs to assign pointers to Extent, which greatly reduces the number and level of pointers and improves the performance of large file operations.

Fixed number of inode: in the ext2/3/4 series of file systems, the number of inode is fixed, the disadvantage is that if you store a lot of small files, it may cause inode to be used up, but there is still a lot of remaining space on the disk can not be used, but it also has an advantage that once the disk is damaged, it is relatively easy to recover, because the data layout on the disk is relatively simple.

Xfs

Compared with ext4, xfs does not support the following features

Log (Journal) check code is not supported

No No Journaling mode is supported

File creation time is not supported

Data log (data journal) is not supported, only metadata log (metadata journal)

But xfs has the following features

Supported * * files and partitions have reached 8EB

Inode is allocated dynamically, so it is not limited by the number of inode, and you no longer have to worry about the shortage of inode caused by storing a large number of small files.

Larger xattr (extended attributes) space, ext2/3/4 and btrfs limit the length of xattr to no more than one block (usually 4K), while xfs can reach 64K

The internal Allocation groups mechanism is adopted, and there is no dependency between each group. It supports concurrent operations and performs well in some scenarios in a multi-core environment.

Provides native dump and restore tools and supports online dump

Btrfs

Btrfs is a file system similar to ZFS, which supports a lot of functions. It is said that ext4 will be replaced as the default file system under Linux in the future. Here are some important functions

Supported * files and partitions have reached 16EB

Support for COW (copy on write)

Optimized for small files and SSD

Inode dynamic allocation

Support subpartition (Subvolumes), which can be mounted separately

Support for metadata and data validation (crc32)

Support for compression and de-duplication

Support for multiple disks and partitions, dynamically scalable

Support for LVM,RAID features (with btrfs, lvm and soft raid are no longer needed)

Incremental backup and recovery

Support for snapshots

Convert ext2/3/4 to btrfs (not the other way around)

The disadvantage of btrfs*** is that the fragmentation problem is serious due to the implementation of COW, which is not suitable for frequent writing scenarios, such as databases, disk files of virtual machines, and so on. But don't worry on most occasions. Btrfs has online defragmentation tools.

How to choose

The following table is for reference only

The file system is suitable for scenarios because ext2U disk generally does not store many files, and the files of USB disk are backed up on the computer, so the security requirements are not so high. Because ext2 does not write logs (journal), the performance of writing USB disk is better. Of course, because the compatibility of ext2 is not as good as that of fat, at present, most USB disk formats still use fatext3 where there is a high demand for stability. After having ext4, it seems that there is no reason to use ext3,ext4. The problem now is that it takes a short time to stabilize ext4 small files, fewer ext series file systems do not support inode dynamic allocation, so if there are a large number of small files to store. It is not recommended to use ext4xfs with too many small files or need large xttr space. For example, openstack swift puts the metadata of data files in xttr. Xfs supports inode dynamic allocation, so there is not enough inode, and the length of xttr can reach that 64Kbtrfs does not write frequently, and requires some features of btrfs. Although btrfs is not stable, it supports many functions. If you need these functions and will not write files frequently, choose btrfs.

In addition, the internal structure of ext series file system is relatively simple, and it is relatively easy to recover when something goes wrong.

Thank you for reading this article carefully. I hope the article "sample Analysis of Common File Systems under Linux" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.