Shulou(Shulou.com)06/02 Report--

The protocol and NS logical name of the fs.defaultFS hdfs://ns1 default file service, and the corresponding configuration in hdfs-site replaces the fs.default.name,ns1 in 1.0 and the dfs.nameservices configuration item in hdfs-site.xml corresponding to hadoop.tmp.dir / hadoop/1/tmp/hadoop-$ {user.name} ${hue.suffix} A base for other temporary directories. Store some class files after decompressing the jar package, similar to dfs.namenode.name.dir;dfs.datanode.data.dir in the hadoop-unjar3569575687995621667,hdfs-site.xml configuration file Configuration items such as dfs.namenode.checkpoint.dir will use the contents of the configuration items io.compression.codecs org.apache.hadoop.io.compress.DefaultCodec, org.apache.hadoop.io.compress.GzipCodec, com.hadoop.compression.lzo.LzopCodec, org.apache.hadoop.io.compress.BZip2Codec, org.apache.hadoop.io.compress.SnappyCodec Org.apache.hadoop.io.compress.Lz4Codec provides compression / decompression java tool io.file.buffer.size 131072 128m SequenceFiles cache size that can be used in read and write Generally configure 4096 times on x86 systems Determines how much cache ha.zookeeper.quorum nn1.cc.bi:2181,nn2.cc.bi:2181,s1.cc.bi:2181 A list of ZooKeeper server addresses, separated by commas, that are to be used by the ZKFailoverController in automatic failover ipc.client.connection.maxidletime 60000 exceeds the maximum idle period maxidletime of the connection between the client and the server during the read and write process Disconnect the server from the client io.seqfile.compression.type BLOCK Hadoop SequenceFile is a storage file composed of byte streams in binary form key/value, SequenceFile is compressible and segmented It is very suitable for the storage characteristics of hadoop files. SequenceFile is written by SequenceFile.Writer. According to the compression type SequenceFile.Writer, two subclasses SequenceFile.BlockCompressWriter and SequenceFile.RecordCompressWriter are derived. The compression mode is represented by the internal enumeration class CompressionType of the SequenceFile class. Three modes are defined [NONE | RECORD | BLOCK] CompressionType.NONE record information is not compressed CompressionType.RECORD record information compression (single record compression) CompressionType.BLOCK a block will be composed of multiple records, compression is acting on block, than RECODE mode can obtain better compression than topology.script.file.name / usr/local/hadoop/etc/hadoop/topology.sh rack recognition script

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.