Shulou(Shulou.com)05/31 Report--

This article mainly shows you "what are the main components of Hadoop", the content is easy to understand, clear, hope to help you solve your doubts, let the editor lead you to study and learn "what are the main components of Hadoop" this article.

The main components of Hadoop include:

Software framework written by Hadoop:Java to support data-intensive distributed applications

ZooKeeper: distributed Coordination system with High Reliability

MapReduce: a flexible parallel data processing Framework for big data

HDFS:Hadoop distributed file system

Oozie: responsible for MapReduce job scheduling

HBase:Key-value database

Hive: a data Warehouse package built on MapRudece

Pig:Pig is an advanced data processing layer based on Hadoop. The Pig Latin language provides programmers with a more intuitive way to customize data streams.

The scope of application and typical characteristics of Hadoop MapReduce method

A huge amount of data

Little or no data dependency

Contains both structured and unstructured data

Suitable for large-scale parallel processing

Application use case

A batch analyzer fast enough to meet business needs and business reports, such as website traffic and product recommendation analysis.

Iterative analysis using data mining and machine learning algorithms. Such as association rule analysis K-means data aggregation, link analysis (data analysis technology), data mining classification, the famous Bayes algorithm analysis.

Statistical analysis and extraction, such as Web log analysis, data analysis

Behavior analysis, such as clickstream analysis, user video behavior, etc.

Conversion and enhancements such as social media, ETL processing, data standardization, etc.

In general, Hadoop is used in distributed environments. As was the case with Linux, vendors integrate and test components of the Apache Hadoop ecosystem and add their own tools and management capabilities.

These are all the contents of the article "what are the main components of Hadoop". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.