Shulou(Shulou.com)06/02 Report--

This article introduces you how to analyze Hadoop, the content is very detailed, interested friends can refer to, hope to be helpful to you.

Introduction to Hadoop

Hadoop is a distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without knowing the underlying details of the distribution. Make full use of the power of the cluster for high-speed computing and storage.

In a nutshell, Hadoop is a software platform that makes it easier to develop and run software that handles large-scale data.

Hadoop implements a distributed file system (HadoopDistributedFileSystem), referred to as HDFS. HDFS has high fault tolerance (fault-tolerent) and is designed to be deployed on low-cost (low-cost) hardware. And it provides high transfer rate (highthroughput) to access the application's data, which is suitable for applications with very large data sets (largedataset). HDFS relaxes the requirement of (relax) POSIX (requirements) so that data in the file system can be accessed (streamingaccess) by stream.

Hadoop is a distributed computing infrastructure that consists of a series of related subprojects belonging to the Apache Software Foundation (ASF). ASF supports these open source community projects. MapReduce and distributed File system (HDFS) are included in Hadoop, and other subsystems provide some additional functionality, or add some high-level abstraction to core. The following introduction to Hadoop will introduce you to some of the additional functions of the Hadoop subsystem.

Core

Distributed systems and generic IO components and interfaces (serialization, Java remote procedure calls, etc.).

Avro

A data serialization system that supports cross-language procedure calls and persistent data storage.

MapReduce

A distributed data processing model and running environment built on cheap PC machines.

HDFS

The HDFS in the introduction to Hadoop is built on a distributed file system on a cheap PC machine.

Pig

A data flow language and runtime environment that deals with massive datasets. Pig runs on top of HDFS and MapReduce.

HBase

Distributed, column-oriented database. HBase uses HDFS as the underlying storage, while using MapReduce to support batch mode calculations and random queries.

ZooKeeper

Provide distributed and efficient collaboration services. ZooKeeper provides atomic operations such as distributed locks, which can be used to build distributed applications.

Hive

In a distributed data warehouse, Hive uses HDFS to store data and provides a language similar to SQL (converted to MapReduce tasks) to query data.

Chukwa

Distributed data acquisition and analysis system. Use HDFS to store data and use Mapreduce to output analysis reports.

On how to parse Hadoop to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.