Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "what is the Linux file distributed system". The editor shows you the operation process through the actual case, and the operation method is simple, fast and practical. I hope this article "what is the Linux file distributed system" can help you solve the problem.

Lustre (www.lustre.org) lustre is a large-scale, secure and reliable cluster file system with high availability, which is developed and maintained by SUN. The main purpose of the project is to develop a next-generation cluster file system that can support more than 10000 nodes and PB storage systems.

Lustre is an open source cluster file system, which adopts the GPL license agreement. at present, in cluster computers, the improvement of data exchange between computers and disks can not keep up with the growth of microprocessors and memory, thus dragging down the performance of applications. a new cluster file system software improves the speed of Imax O, which may reduce the cost of purchasing storage devices and change the way enterprises buy storage. Cluster file systems are already in use in universities, laboratories and supercomputer research centers, and are about to enter the general business computing market. The new cluster file system uses open source lustre technology, developed by the U.S. Department of Energy (Department Of Energy) and commercially supported by HP. It has significantly improved the speed of input and output (I / O), and has produced some success in universities, national laboratories and supercomputing research centers. In the next few years, it is likely to enter the field of ordinary commercial computers.

Run under linux, the development language cAccord +

Hadoop (hadoop.apache.com) hadoop is not just a distributed file system for storage, but a framework designed to execute distributed applications on a large cluster of general-purpose computing devices.

The following figure shows the architecture of hadoop:

Authorization protocol apache, development language java, resource consumption is a bit high.

MogileFs (www.danga.com) MogileFs is an open source distributed file system with key features including

1. Components of the application layer

2. No single point of failure

3. Automatic file replication

4. It has better reliability than RAID

5. It runs under linux without RAID nigukefs support.

FreeNAS (www.openqrm.org) FreeNAS is a dedicated operating system for network attached storage (NAS) services (a simplified version of FreeBSD). Based on m0n0wall firewall, by providing disk management and RAID software, the system allows users to convert PC into NAS server by home, and supports FTP/NFS/RSYNC/CIFS/AFP/UNISON/SSH sourceforge.net/pro protocol. The purpose of this system is to enable people to reuse old hardware.

FastDFS (code.google.com/p/fastdfs) FastDFS is an open source distributed file system, which manages files. Its functions include file storage, file synchronization, file access (file upload, file download), etc., which solves the problems of mass storage and load balancing. It is especially suitable for online services with files as carriers, such as photo album websites, video websites and so on. The FastDFS server has two roles: tracker (tracker) and storage node (storage). The tracker mainly does the scheduling work and plays the role of load balancing on the access. The storage node stores files and completes all the functions of file management:

Store, synchronize, and provide access interfaces, and FastDFS also manages the meta data of files. The so-called meta data of a file is the relevant attributes of the file, expressed in the form of key-value pairs (key value pair), such as: width=1024, in which the key is width,value is 1024. A file meta data is a list of file attributes that can contain multiple key-value trees.

The development language cswab runs under the linux system.

NFS (www.tldp.org/HOWTO/NFS-HOWTO/index.html) network file system is one of the file systems supported by FreeBSD, also known as NFS.

NFS allows a system to share directories and files with others on the network. By using NFS, users and programs can access files on remote systems as if they were local. Its advantages are:

1. The local workstation uses less disk space because the usual data can be stored on a machine and accessible through the network.

2. Users do not have to have a home directory in every machine on the network. The home directory can be placed on a NFS server and is available everywhere on the network.

3. Storage devices such as floppy drives, CDROM, and ZIP can be used by other machines on the network. The number of removable media devices on the entire network can be reduced.

The development language, cswab, can run across platforms.

OpenAFS (www.openafs.org) OpenAFS is an open source distributed file system that allows files and resources to be shared between systems over local and wide area networks. OpenAFS is organized around a set of file servers called cell, and the identity of each server is usually hidden in the file system, and users who log in from AFS clients will not be able to tell which server they are running on, because from the user's point of view, they want to run on a single system that knows the semantics of other Unix file systems.

File system content is usually replicated across cell, so the failure of a hard disk does not damage the operation on the OpenAFS client. OpenAFS requires high-volume client caching up to 1GB to allow access to frequently used files. It is a very secure kerbero-based system that uses access control lists (ACL) for fine-grained access, which is not based on the usual Linux and Unix security models. Development protocol IBM Public, running under linux.

MooseFs (derf.homelinux.org) Moose File System is a network distributed file system with fault tolerance, which distributes data on different servers in the network. MooseFs makes it look like a Unix file system through FUSE. But there is a problem, it still can not solve the problem of single point of failure. Development language perl, which can be operated across platforms.

PNFS (www.pnfs.com) Network File system (Network FileSystem,NFS) is an important part of most local area networks (LAN). But NFS is not suitable for demanding input bookcase-intensive programs in high-performance computing, at least before. The crime modification of the NFS standard incorporates Parallel NFS (pNFS), which is a parallel implementation of file sharing that increases the transfer rate by several orders of magnitude.

The development language, cswab, runs under linu.

GoogleFs is said to be a good extensible distributed file system for large, distributed applications that access large amounts of data. It runs on cheap ordinary hardware, but it can provide fault tolerance, and it can provide high performance services to a large number of users. Google developed it himself.

This is the end of the content about "what is the Linux File distributed system". Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.