Shulou(Shulou.com)05/31 Report--

This article will explain in detail what the hadoop MapReduce HDFS application scenarios are, and the editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Hadoop is similar to redundant Array of disks (RAID) in terms of storage, which distributes and provides data to provide throughput. Its storage system is HDFS (Hadoop Distuibute Fils System). Computationally speaking, it distributes big data's calculations to multiple computers through the MapReduce model, and then merges the results to reduce computing time.

Hadoop is suitable for:

1. Calculation of super-large data

2. The mode of writing once and reading multiple times

3. Can run on ordinary hardware.

Hadoop is not suitable for:

1. Low latency data access, which is optimized for high data throughput applications

HDFS is not suitable for processing low-latency application requests that users require a short time. It is designed to handle large data set analysis tasks and is mainly designed to achieve high data throughput. This may require high latency as a price [time for space]. The current supplementary solution is to use HBase to make up for this deficiency as much as possible through the upper-level data management project.

2. A large number of small files

The Hadoop client needs to interact with namenode, and the file attributes of datanode are stored in namenode, and they are all in memory. If there are too many small files, namenode cannot store them.

3. Multi-user writes and arbitrarily modifies files.

Hadoop is suitable for one-time computing and multiple-read scenarios, such as search engines, which only support random reads but not random writes. For example, the integration of Hadoop and Lucene cannot be directly integrated, because Lucene supports random writes.

This is the end of this article on "what are the hadoop MapReduce HDFS application scenarios?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.