Shulou(Shulou.com)06/03 Report--

In-depth understanding of replica storage in HDFS architecture: optimized replica storage strategy is an important feature of HDFS to distinguish other distributed systems. HDFS adopts rack-aware strategy to improve the reliability, availability and network bandwidth utilization of replica data. Through rack awareness, Namenode can determine the rack id to which DataNode belongs. The storage strategy of HDFS is to store one copy on a node of the local rack, one copy on another node of the same rack, and the last copy on a node of a different rack. Reduce the data transmission between racks, improve the efficiency of write operations, and make better use of the bandwidth of multiple racks when reading. After the safe mode Namenode starts, it will enter the special state of safe mode and do not copy the data block. Namenode receives heartbeats and block status reports from all DataNode. The block status report includes a list of all blocks in DataNode. Detect whether each data block reaches the minimum number of copies. File system metadata persistence Namenode uses EditLog's transaction log to record any operations that make changes to the file system metadata. The DataNode space of the entire file system, including block-to-file mapping, file attributes, etc., is stored in the FsaImage file. When NameNode starts, read EditLog and FsImage from the hard disk, and apply all EditLog transactions to the in-memory Image. Save the FsImage and delete the EditLog, a process called a checkpoint that occurs only when the NameNode starts. File system Recycle Bin if the Recycle Bin configuration is enabled, when a user or application deletes a file, the file is renamed and transferred to the / trash directory for later recovery. Save time can be configured, beyond the time, the file will be deleted forever. Hadoop fs-rm / file moves file to the Recycle Bin directory. Hadoop fs-rm-skipTrash / file deletes the file directly.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.