Shulou(Shulou.com)06/01 Report--

This article to share with you is about how to ext3, ext4, xfs and btrfs file system performance comparison, Xiaobian think quite practical, so share to everyone to learn, I hope you can read this article after harvest, not much to say, follow Xiaobian to see it.

I only copy the core legend here ha.

1: Single-byte write performance comparison

A: Hard disk block devices are not good at this kind of test.

B: The btrfs system is only 200K/SEC, and the performance of xfs is relatively balanced.

2: Comparison of block write performance (since hard disks are block devices, this comparison is more meaningful)

A: Similar in performance, but the best in efficiency (CPU utilization) is xfs followed by EXT4, EXT3, BTRFS

3: Direct block sequential read and write (turn off any system and file caching)

A: For bypassing system and file buffers (e.g. video recording, some virtual machine software, ECC), EXT3/ 4 is the best choice, followed by BTRFS, and finally XFS.

B: No one file system works for all environments

4: Random seek

A: BTRFS has the worst performance, less than 20 seeks/sec

B: EXT3 performs best, and this file system performs better if the software has a lot of random addressing

5: Create and delete a large number of files (a certain number of files)

The BTRFS system has the worst performance. Below is a comparison of the other three systems without this system.

A: EXT4 is the more efficient and high-performance system, followed by XFS,EXT3

6: Sequential read/write throughput [100 writes/one fsync() without fsync, 1 writes/one fsync() with fsync]

A: 100 writes/one fsync() Each performance is similar

B: EXT3 performs best when 1 writes/one fsync(), followed by XFS,EXT4, BTRFS

C: write + fsync() affects read performance under BTRFS

7: Random read and write throughput

A: 100 seeks/sec 16 KB per block, we get the maximum read speed is 1600 KB/sec, XFS, BTRFS is greater than this value (maybe the data cannot be random or buffering affects the result)

B: EXT3 random write performance is the best, suitable for databases, high-volume loggers and virtual machine systems

8: Write 100,000 lines of data to PostgreSQL

A: BTRFS performance is the best, EXT4 and XFS very low CPU usage but poor performance

9: Read the test

A: 100,000 times of reading test, the performance difference is not significant

10: Complex Reading and Writing and Transaction Testing

A: EXT3 performs best

Therefore, the database is preferably EXT3 unless EXT4 solves the so-called regression problem.

11: Unpacking operations under Linux kernel 2.6.36

A: The best file system for this operation is EXT4.

This operation will be affected by cache and delay allocation, we force synchronization to see the effect

A: XFS is slower FS, EXT3 slower than EXT4 and BTRFS

12: Cat operation

A: The most efficient system for this operation is XFS

B: The system with the fastest execution and the highest CPU consumption is BTRFS, indicating that the system has complex metadata operations.

13: Unzip Linux kernel (32000 files)

A: EXT3, the only system without delay assignment capability, is the worst.

14: Sequential creation of 128 files, each 16 MB long (2 GB total) Fragments generated by various systems

A: BTRFS fragmentation is a serious problem (which explains the poor read performance of previous systems)

Figure after leaving BTRFS system

A: EXT4, XFS This system with delay allocation mechanism produces less fragments than EXT3 (even if one write/one fsync())

15: Random creation of 128 files, each 16 MB long (2 GB total) Fragment generated by various systems

A: Random writing will produce fragments under any system, even if there is a delay allocation.

Linux kernel since 2.6.28

Formal support for the new file system Ext4 begins. Ext4 is an improved version of Ext3, modifying some important data structures in Ext3, not just Ext3 pairs.

Ext2 is just an added logging feature. Ext4 offers better performance and reliability, as well as richer features:

1. Compatible with Ext3. You can migrate from Ext3 to Ext4 online in a few commands without having to reformat the disk or reinstall the system. The old Ext3 data structure remains, Ext4 acts on the new data, and of course the entire file system gets the larger capacity that Ext4 supports.

2. Larger file systems and larger files. Ext4 supports file systems of 1EB (1,048,576TB, 1EB=1024PB, 1 PB = 1024 TB) and files of 16TB, respectively, compared to Ext3's current maximum of 16TB file systems and maximum of 2TB files.

3. Unlimited number of subdirectories. Ext3 currently supports only 32,000 subdirectories, while Ext4 supports an unlimited number of subdirectories.

4. Extents。Ext3

Indirect block mapping is extremely inefficient when working with large files. For example, for a file of 100MB size, 25,600 data blocks (each data block size) are created in Ext3

4KB). Ext4 introduces the popular concept of extents in modern file systems, where each extent

As a set of consecutive data blocks, the file is represented as "the file data is stored in the next 25,600 data blocks," which improves efficiency.

5. Multiple block allocation. when

When writing data to the Ext3 file system, Ext3's block allocator can allocate only one 4KB block at a time, and writing a 100MB file calls 25,600

Subblock allocator, while Ext4's multiblock allocator "multiblock allocator"(mballoc) supports allocating multiple blocks in one call.

6. Delayed distribution. Ext3's block allocation policy is to allocate as soon as possible, while Ext4 and other modern file operating systems 'policy is to delay allocation as much as possible until the file is finished in cache and then start allocating blocks to disk, which optimizes block allocation for the entire file.

7. Fast FSCK. While fsck used to be slow in its first step because it had to check all inodes, Ext4 now adds a list of unused inodes to each group's inode table, so the fsck Ext4 filesystem can skip them and check only those inodes that are in use.

8. Log verification. Logs are the most common part and can easily lead to disk hardware failures, while recovering data from corrupted logs can lead to more data corruption. Ext4's log validation feature makes it easy to determine whether log data is corrupt, and it combines Ext3's two-phase logging mechanism into one phase, increasing security while improving performance.

9. "No Journaling" mode. Logging always has some overhead, and Ext4 allows logging to be turned off so that certain users with special needs can improve performance.

10. Online defragmentation. Although deferred allocations, multiblock allocations, and extents can effectively reduce file system fragmentation, fragmentation is inevitable. Ext4 supports online defragmentation and will provide the e4defrag tool for defragmentation of individual files or entire file systems.

11. Inode related properties. Ext4

Hold larger inode size than Ext3 default inode size of 128 bytes, Ext4 to accommodate more extended attributes in inode (such as nanosecond timestamp)

or inode version), the default inode size is 256 bytes. Ext4 also supports fast extended properties

attributes) and inodes reservation).

12. Persistent preallocation. P2P soft

In order to ensure that there is enough space for downloading files, an empty file of the same size as the downloaded file is often created in advance, so as not to cause download failure due to insufficient disk space in the next few hours or days.

Ext4 implements persistent preallocation at the file system level and provides the corresponding API (posix_fallocate() in libc), which is more effective than the application itself.

Efficiency.

13. Barrier is enabled by default. magnetic

Internal cache on disk to reorder bulk data writes and optimize write performance, so file systems must write commit records after log data is written to disk.

Commit records are written first, and the log may be corrupted, which will affect data integrity. Ext4 enables barrier by default, only data before barrier

Write everything to disk before writing data after barrier. (This feature can be disabled with the "mount -o barrier=0" command.)

The above is how to compare the performance of ext3, ext4, xfs and btrfs file systems. Xiaobian believes that some knowledge points may be seen or used in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.