Shulou(Shulou.com)06/01 Report--

This article mainly explains "the reason why Linux and MacOS do not need defragmentation". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "why Linux and MacOS do not need defragmentation".

I believe that many software engineers today use Linux or macOS systems. Unlike Windows, it is difficult to see the concept of disk defragmentation. From personal experience, the author has not defragmented disks in macOS in the past seven or eight years, and you can't find relevant operations in today's disk tools. You can only use the diskutil command to set whether a disk is on or off defragmentation.

Figure 1-macOS disk tool

In the previous article, we analyzed why the early Windows operating system might need to defragment the disk every once in a while. There are two reasons behind this problem. One is that FAT used by Windows is a very simple file system. The design of the file system determines that the same file may be scattered in different locations on the disk. Second, solid state drives were not popular in ancient times. The random read and write performance of mechanical hard disk is very poor.

The reason why Linux and macOS systems do not require defragmentation is the opposite of why Windows needs defragmentation:

The file systems used by Linux and macOS either reduce the probability of fragmentation or implement automatic defragmentation.

Solid state hard disk and mechanical hard disk have different characteristics. Defragmentation may not only not significantly help to improve read and write performance, but also is not conducive to the service life of hardware.

File system

Linux generally uses Ext2, Ext3, and Ext4 file systems, and most of today's Linux distributions choose Ext4. Unlike Windows, which stores multiple files continuously, Linux scatters files to different places on disk and leaves some space between files to ensure that files are not fragmented when they are modified or updated.

Figure 2-Linux File system [^ 2]

Most of today's macOS uses the APFS file system [^ 3], which is optimized by Apple for devices such as solid state drives. Earlier HFS and HFS+ used an Extent-based design, with each chunk containing a sequence number and a contiguous section of storage space, which looked for several contiguous chunks in the file system to provide the required space.

Figure 3-macOS file system

Whether Linux or macOS, their file systems are designed based on chunks, and the disk space allocation is relatively reasonable, so there are no fragmented disks on the Windows system.

In addition to the file system designed to avoid fragmentation, Linux and macOS also introduce a delayed space allocation strategy, which delays disk writing through buffers as much as possible, which can not only reduce the probability of flushing disk, but also increase the probability that files are written to adjacent blocks. however, this mechanism is not without side effects, and more data may be lost when the system is powered off or crashes.

If fragmentation does occur on disk, Linux and macOS file systems will also try to move fragmented files without the need for additional defragmentation tools, and the user experience of this design will be much better than manually triggering longer defragmentation. The HFS+ system on macOS also supports real-time defragmentation, which triggers defragmentation [^ 4] when the following conditions are met:

File is less than 20 MB

Files are stored in more than 8 blocks

The file has not been updated in the past minute

The system has been activated for three minutes.

In most cases, the proportion of disk fragmentation in these operating systems is very low, and fragmentation begins only when there is insufficient disk space, so what we really need at this time is a larger disk or a newer computer. instead of defragmentation on the disk.

Solid state hard disk

Solid-state drives have been storage media for 30 years, but they have not been popular in data centers and personal computers because they have been very expensive in the past. Even today, the price of mechanical disks has obvious advantages over solid-state drives.

Figure 4-Price comparison between solid State hard drives and Mechanical hard drives

The new type of storage media brings new features and performance, as we introduced in the previous article, because of the mechanical structure of the mechanical hard disk, the performance of its random Imax O may be hundreds of times different from that of sequential Imax O. Defragmentation can merge the data scattered on the disk into one place, and reducing the number of random IPRAO will naturally improve the performance of reading and writing files.

Although there are differences in performance between SSDs and random ICPAs, the differences may be between ten to dozens of times, and the random Imax O latency of SSDs is tens or even thousands of times higher than that of mechanical disks.

Figure 5-Evolution of NAND Flash memory

Although the solid state disk (SSD), as an electronic component, has a good performance, it has a limit on the number of times of cyclic erasure, which is also known as Pmax E. Its life is limited compared with the mechanical hard disk. If the number of erases and writes of a 512 GB solid state disk is 1000 times, it will consume a life each time it is full of data, and the hard disk will be scrapped after the erase and write times reach 1000 times. Defragmentation is actually actively moving the data on the hard disk, which will naturally affect the life of the hardware.

Summary

There is a very interesting phenomenon in software engineering. The engineers of hardware and infrastructure are trying their best to optimize the performance of the system, but the engineers of the application layer often do not care about the small differences in performance. this is also the result of differences in job responsibilities, and different locations determine different concerns.

The evolution and innovation of hardware deeply affect the design of upper software, so it is extremely difficult to design a general-purpose system. If the characteristics of the underlying hardware are not taken into account when designing the file system, we will not be able to make full use of the performance provided by the hardware and get the desired results. Here are a brief summary of two reasons why Linux and macOS do not need defragmentation:

The design of the file system based on block allocation makes the probability of fragmentation on the disk very low. The delayed allocation and automatic defragmentation strategy liberates the users of the operating system. In most cases, there is no need to consider disk fragmentation.

The random read and write performance of solid state disk is much better than that of mechanical hard disk. Although there are differences in performance between random read and write and sequential read and write, there is no great difference between mechanical hard disk and mechanical hard disk, and frequent defragmentation will also affect the service life of solid state hard disk.

In the end, let's take a look at some more open related questions. Interested readers can think carefully about the following questions. Unlike the previous questions, the author will answer these two questions in later articles:

Why is there an upper limit on the number of erases and writes on SSDs?

Under what circumstances are mechanical hard drives more likely to be damaged?

If you have questions about the content of the article or want to know more about the reasons behind some design decisions in software engineering, you can leave a message below the blog, and the author will respond to the questions related to this article in time and choose the appropriate topic as the follow-up content.

Thank you for your reading, the above is the content of "Linux and MacOS do not need defragmentation". After the study of this article, I believe you have a deeper understanding of the reason why Linux and MacOS do not need defragmentation, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.