Shulou(Shulou.com)06/03 Report--

This paper mainly introduces the INodeReference mechanism of hdfs in hadoop source code.

In the hdfs2.6 version, many new features have been introduced, and some of the original source code designs have been modified. An important update is the introduction of snapshot functionality. However, when a HDFS file or directory is in a snapshot and the file or directory is renamed or moved to another path, there will be multiple access paths to the file or directory. INodeReference is created to solve this problem.

Problem description

/ an is a normal directory in hdfs, S0 is a snapshot of / a, and there is a file test in the / a directory. According to the definition of the snapshot, we can access the test file through / a/test and / a/snapshot/s0/test.

However, when the user renames the / a/test file to / x/test1, the test file cannot be accessed through the snapshot path / a/snapshot/s0/test, which is not in compliance with the snapshot specification.

Introduction of INodeReference

To solve the above problems, hdfs introduces the INodeReference class. Figure 1-1 shows the inheritance diagram for INodeReference. The WithName,WithCoount,DstReference here is a subclass of INodeReference as well as an inner class of INodeReference. The WithName object is used to replace the INode object in the source path before the rename operation, and the DstReference object is used to replace the INode object in the destination path after the rename operation. WithName and DstReference point to a WithCount object together, and the WithCount object points to the real INode object in the file system directory tree.

Figure 1

INodeReference code implementation

INodeReference is an abstract class, which extends from the INode class, so INodeReference and its subclasses can be added to the file system directory tree to replace the original INodeFile node. INodeReference defines the referred field, which holds the INode node that the current INodeReference class points to, so the WithName and RstReference,referred fields point to the WithCount object and, for WithCount,referred, to the real INode object. INodeReference also defines the getReferredINode () method, which, in the operation of the file system directory tree, calls the getReferredINode () method to get the INode object pointed to by INodeReference if it is determined that the current node is a reference node.

The INode node public INodeReference (INode parent,INode referred) {super (parent) that public abstract class INodeReference extends INode {private INode referred;// points to; this.referred = referred;} public final INode getReferredINode () {/ / get the INode node return referred;} public final void setReferredINode (INode referred) {this.referred = referred;} / /.}

Then, let's look at the implementation of the WithCount class.

The WithCount class defines a collection field withNameList that holds all collections of WithName objects that point to this WithCount object. The WithCount class also defines the addReference () method, which is called by any WithName object and DstReference object that points to the WithCount object to add a pointing relationship. For the DstReference object that points to the WithCount object, the addReference () method sets the object as its own parent INode node, while for the WithName object, the addReference () method saves the object in the withNameList collection.

Public static class WithCount extends INodeReference {/ / saves the collection of all WithName objects pointing to this WithCount object private final List withNameList = new ArrayList (); public WithCount (INodeReference parent,INode referred) {super (parent,referred); / / calls the constructor of the parent class to point to INode Preconditions.checkArgument (! referred.isReference ()) in the file system directory tree; refferred.setParentReferenct (this) / / set the parent node of the real INode to the current WithCount object} public void addReferenct (INodeReference ref) {if (ref instanceof WithName) {/ / if it is a WithName object, add withNameList WithName refWithName = (WithName) ref; int i = Collections.binarySearch (withNameList, refWithName,WITHNAME_COMPARATOR) Preconditions.checkState (I)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.