Shulou(Shulou.com)06/02 Report--

How to treat Hadoop, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

1. 2013 global data total 4.4 ZB estimates 2020 44ZB 1ZB=1000EB=1000 PB 1PB=1024TB2. The data generated by individuals is growing the data generated by the Internet of things 3. Big data is better than a good algorithm

For some applications (such as recommending movies and music based on past preferences), no matter how good the algorithm is, the recommendation effect based on small data is often not as good as that of general algorithms based on a large amount of available data.

Problems encountered in reading multiple hard drives

Hardware failure

How to ensure the correctness of the data

MapReduce

Each query needs to process the vast majority of the entire dataset or at least one dataset

In essence, a batch processing system is not suitable for interactive analysis.

Hard disk development trend: the improvement of addressing time is far less than the improvement of transmission rate. MapReduce is suitable for write-once and multiple-read applications, while relational databases are more suitable for continuously updated data sets. Hadoop is very effective grid computing for unstructured or semi-structured data.

Hadoop tries to store data on computing nodes to achieve fast local access to the data. Data localization is the core of Hadoop data processing, and good performance is obtained as a result.

Three Design goals of Voluntary Computing hadoop

Provide services for assignments that can be completed in just a few minutes or hours

Runs in the same data center with a high-speed network connection

The computers in the data center are all reliable and specialized hardware

After reading the above, have you mastered the way to view Hadoop? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.