Shulou(Shulou.com)06/01 Report--

This article mainly shows you the "sample analysis of the SOLARIS ZFS file system", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and study the "sample analysis of the SOLARIS ZFS file system" this article.

ZFS is the world's first 128bit file system launched by SUN, which means it can store 180 billion (18.4 × 10 ^ 18) times more data than the current 64-bit file system. The design of ZFS is so advanced that this limit may never be met at present. It is said: "to fill a 128bit file system will deplete all the storage devices on the planet. Unless you have the energy to boil the entire ocean, you cannot fill it. (Populating 128-bit file systems would exceed the quantum limits of earth-based storage. You couldn't fill a 128-bit storage pool without boiling the oceans.)" [1]

Here are some theoretical limits of ZFS:

* 2 ^ 48-number of snapshots of any file system (2 × 10 ^ 14)

* 2 ^ 48-number of files for any single file system (2 × 10 ^ 14)

* 16 exabytes (2 ^ 64 byte)-maximum file system size

* 16 exabytes (2 ^ 64 byte)-maximum single file size

* 16 exabytes (2 ^ 64 byte)-maximum attribute size

* 3 × 10 ^ 23 petabytes (2 ^ 78 byte)-maximum zpool siz

* 2 ^ 56-the number of attributes in a single file (constrained by the number of ZFS files, actually 2 ^ 4 8)

* 2 ^ 56-the number of files in a single directory (constrained by the number of ZFS files, actually 2 ^ 4 8)

* 2 ^ 64-number of devices in a single zpool

* 2 ^ 64-number of zpools of the system

* 2 ^ 64-number of file systems for a single zpool

As a perceptual understanding of these numbers, assuming that 1000 new files are created per second, it takes about 9, 000 years to reach the ZFS file limit.

In defending the relationship between filling ZFS and boiling the ocean, Bonwick wrote:

Although we all want Moore's Law to last forever, quantum mechanics gives theoretical limits on the rate of calculation (computation rate) and the amount of information on any physical device. For example, an object with a mass of 1 kilogram and a volume of 1 liter performs at most 10 ^ 51 operations per second on 10 ^ 31 bits of information. See Seth Lloyd, "Ultimate physical limits to computation (Ultimate physical limit of Computation)." Nature 406, 1047-1054 (2000)]. A complete 128bit storage pool will contain 2 ^ 128 blocks = 2 ^ 137 bytes = 2 ^ 140 bits; accordingly, at least (2 ^ 140 bits) / (10 ^ 31 bits / kg) = 136 billion kg of material is required to save these data bits.

Add:

Ten reasons to use ZFS

1. No more fsck, scandisk.

Whether you are using Linux,UNIX or Windows, I believe we all have a similar experience: when the system is accidentally powered off or shut down illegally, the file system is found to have an inconsistent problem after the system is restarted, and it needs to be repaired by fsck or scandisk, which is very time-consuming and may not be able to be repaired successfully in the end. To make matters worse, if this is a server that needs to do fsck, it can only be offline (offline), and existing applications are often large hard drives, and the corresponding fsck repair time is also very long, which is almost intolerable for many users who use the server. After using ZFS, you can abandon fsck completely, because ZFS is a file system based on COW (Copy on Write) mechanism. COW does not rewrite existing files on the hard disk, ensuring that all files on the hard disk are valid. So there is no such concept of inconsistent, naturally there is no need for this kind of tool.

two。 Easy to manage

As a brand-new file system, ZFS completely abandons the traditional File System + Volume Manager + Storage architecture. All storage devices are managed through ZFS Pool. As long as all kinds of storage devices are added to the same ZFS Pool, everyone can easily manage the configuration file system in this ZFS Pool. You no longer have to keep in mind all kinds of professional concepts, various commands newfs, metinit and the usage of various Volume Manager. In ZFS, we only need two commands, zpool (needle-to-ZFS Pool management) and zfs (for ZFS file system management), to easily manage 128bit file systems. For example, we often encounter that the system data is growing too fast, the existing storage capacity is not enough, and we need to add a hard disk. If we follow the traditional Volume Manager management method, we need to consider many existing factors in advance and calculate various parameters that need to be configured according to the application in advance. In the case of ZFS, our system administrator can be completely liberated, no longer need this kind of artificial complex consideration and calculation, we can leave these to ZFS, because ZFS Pool will automatically adjust to dynamically adapt to the needs. All we need is a simple command to add a new hard drive to the ZFS Pool:

Zpool add zfs_pool mirror c4t0d0 c5t0d0

Based on this dynamically adjusted ZFS Pool, all file systems can be used immediately on this new hard disk, and the optimized parameters will be automatically selected.

And ZFS also provides a graphical management interface.

3. There is no capacity limit

The ZFS (Zettabyte File System) file system, as its name suggests, can provide truly massive storage, making it almost impossible to encounter capacity problems in reality. Under the existing 64-bit kernel (kernel), it can hold a single file up to 16 Exabytes (264) size, can use 264 storage devices, and can create 264 file systems.

4. Fully guarantee the correctness and integrity of the data

Because all the data operations of ZFS are based on Transaction (transaction), a set of corresponding operations will be parsed into a transaction operation by ZFS, and the transaction operation represents that a group of operations either fail or succeed together. And as mentioned earlier, ZFS is based on COW (Copy on Write) for all operations, thus ensuring that the data on the device is always valid, and that the inconsistent of the data file will never be caused by a system crash or an accidental power outage.

Another potential threat data may come from hardware devices, such as disks, hardware problems with raid cards, or driver bug. Existing file systems often encounter this problem, often simply send the wrong data directly to the upper application, usually we call this problem Silent Data Corruption. On the other hand, in ZFS, a 256bit Checksum is performed on all data, whether it is user data or the metadata data of the file system itself. When ZFS submits the data, it will be verified, which completely eliminates this Silent Data Corruption situation.

5. Provide excellent performance and scalability

Different from the traditional File System + Volume Manager + Storage architecture, ZFS provides all the functions directly based on the storage device, so it has its own unique innovative characteristics, and the performance is naturally extraordinary.

* Dynamic Striping vs. Static Striping

Because ZFS is based on COW and a global dynamic ZFS Pool, any write operation is a write operation to a new block of data (Block). ZFS dynamically picks out an optimal device from ZFS Pool and writes with a transaction (transaction) linear write, which makes full use of the bandwidth of existing devices. We call this feature Dynamic Striping. The corresponding Static Striping is the way used by the traditional file system, Static Striping requires the administrator to correctly calculate the artificial settings for this group of Stripe in advance, and if you add new devices, you need to re-artificial calculation and setting, what is more serious is that if the human calculation error, it will directly affect the performance of the system. After using the feature of Dynamic Striping, we don't need human intervention at all. ZFS will adjust automatically and intelligently provide you with the best device and the fastest mode of operation.

* support multiple block sizes (Multiple Block Size)

ZFS supports block definitions of various sizes, from 512 bytes to 1m bytes. Unlike traditional file systems, which are often fixed size data blocks, ZFS can dynamically calculate according to different size files and dynamically select the best data block.

Because of the different size of data blocks, it directly affects the actual hard disk capacity and reading speed. If you use smaller data blocks, storing files results in less fragmentation and it is faster to read and write small files, but it results in the need to create more metadata, and it takes more time to read and write large files. If you use larger data blocks, you use less metadata, which makes it easier to read and write large files, but results in more fragmentation. According to the actual investigation of the use of existing files, ZFS analyzes an algorithm for selecting data block size, and dynamically determines the best data block according to the actual file size. So ZFS is very smart and can get a self-tuning result without the intervention of the system administrator. Of course, ZFS also supports custom settings for the block size used by a single file or the entire file system.

* Intelligent pre-reading (Intelligent Prefetch)

Most operating systems have this ability to read data in advance, while ZFS is a more intelligent data pre-reading function that is provided directly on the file system. It can not only intelligently identify a variety of reading modes and read data in advance, but also carry out this pre-reading intelligent identification for each read data stream, which is a very good thing for many streaming media providers.

In terms of scalability, unlike most of the existing file systems, which are based on a limited static model, ZFS adopts the dynamic concept of ZFS Pool, its metadata is also dynamic, and its read and write operations are parallel and have the concept of priority, so it can guarantee the linear growth of performance even in the case of large amount of data and multiple devices.

6. Self-repairing function

* ZFS Mirror and RAID-Z

The traditional hard disk Mirror and RAID 4 Grand raid 5 array will encounter the aforementioned problem: Silent Data Corruption. If a physical problem occurs on a hard disk and causes a data error, the existing Mirror, including the RAID 4e raid 5 array, will silently submit the error data to the upper application. If this error occurs in Metadata, it will directly result in the Panic of the system. And there is a more serious situation: in RAID 4 and RAID 5 arrays, if a power outage occurs when the system is calculating Parity values and writing new data and new Parity values again, then all the data stored in the entire array is meaningless.

In ZFS, the corresponding ZFS Mirror and RAID-Z methods are proposed. When it is responsible for reading data, it will automatically check with the 256bit check code, will actively find this kind of Silent Data Corruption, and then get the correct data back to the upper application through the corresponding Mirror hard disk or other hard disks in the RAID-Z array, and automatically repair the Data Corruption of the original hard disk.

* Fault Manager

In Solaris 10, a ZFS diagnostic engine interacts with Solaris's Fault Manager (another new feature of Solaris 10) to diagnose, analyze and report errors in ZFS Pool and storage devices in real time, and users can get a very friendly message in time through Fault Manager. Although the diagnostic engine does not take proactive actions to fix or resolve the problem, it prompts the system administrator for actions that can be taken in the message. Similar to the following ZFS error message, where REC-ACTION is the recommended action:

SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major

EVENT-TIME: Fri Mar 10 11:09:06 MST 2006

PLATFORM: SUNW,Ultra-60, CSN: -, HOSTNAME: neo

SOURCE: zfs-diagnosis, REV: 1.0

EVENT-ID: b55ee13b-cd74-4dff-8aff-ad575c372ef8

DESC: A ZFS device failed. Refer to http://sun.com/msg/ZFS-8000-D3 for more information.

AUTO-RESPONSE: No automated response will occur.

IMPACT: Fault tolerance of the pool maybe compromised.

REC-ACTION: Run 'zpool status-x' and replace the bad device.

7. Safety

In terms of security, ZFS supports ACL (read control list) similar to the NT-style NFSv4 version. And the aforementioned 256bit CAPTCHA code, users can choose a variety of authentication methods, including SHA-256 authentication algorithm, so as to ensure the security of data at the physical storage unit level.

8. Super function

As the "last file system", ZFS covers the basic file system and Volume management functions, while providing many enterprise-level super functions: Quota (quota), Reservation (reservation), Compression (compression), Snapshot (snapshot), Clone (clone). And very fast. With this file system, you don't need any Volume Manager anymore.

9. Compatibility

ZFS is a file system that is fully compliant with the POSIX specification, so the upper application is completely unaffected. ZFS also provides an Emulated Volume module that can use any ZFS file system as a normal block device. At the same time, ZFS can also use the Volume built on Volume Manager as the storage device unit. In this way, without the need to modify the application and the existing file system, it gives everyone the maximum freedom to obtain the various features provided by ZFS.

10. Open source

ZFS is operated by Sun Microsystems as an open source project of OpenSolaris and is completely free to use. Click here (http: / / www.opensolaris.org/os/community/zfs/source/) to browse directly to the ZFS code. This means that we can not only enjoy the high quality of commercial companies at the same time, but also gain the advantages of open source model.

Although currently only Solaris supports this file system, this open source model is bound to promote more ZFS-based applications. Now some foreign developers are porting ZFS to Linux and Mac OS. If you want to experience ZFS, because it is currently bundled with Solaris 10, you need to download the latest version of Solaris 10 6 Universe 06. The above is all the content of the article "sample Analysis of SOLARIS ZFS File system". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.