Shulou(Shulou.com)06/01 Report--

This article mainly introduces Hadoop how to set the block size of HDFS, the article introduces in great detail, has a certain reference value, interested friends must read it!

HDFS is designed to store and manage big data, so the typical HDFS block size is obviously much larger than the traditional file system block we usually see. The block size setting is used to cut a large file into a data block, and then distribute these data blocks to the cluster. For example, the cluster block size is set to 64MB, and a 128MB file is uploaded to HDFS, and HDFS will split the file into 2 blocks. Then distribute these two pieces of data to the data nodes of the cluster.

Set the block size:

Open hdfs-site.xml, (in the conf folder under the hadoop installation directory)

Procedure:

Set the property values under hdfs-site.xml:

Dfs.block.size

134217728-modified to 128m

Block size

Modification instructions:

Hdfs-site.xml is the configuration file for HDFS, and changing the dfs.block.size attribute value of the hdfs-site.xml configuration file will change the default block size for all files uploaded to HDFS. Changing this value does not affect the block size of any existing files on HDFS, only changes the block size of newly uploaded files.

The above is all the contents of the article "how to set the block size of HDFS by Hadoop". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.