Shulou(Shulou.com)05/31 Report--

This article will explain in detail what Hadoop products are, Xiaobian thinks it is very practical, so share it with you for reference, I hope you can gain something after reading this article.

Hadoop series commonly used projects include Hadoop , Hive , Pig , HBase , Sqoop , Mahout , Zookeeper , Avro , Ambari , Chukwa , YARN , Hcatalog , Oozie , Cassandra , Hama , Whirr , Flume , Bigtop , Crunch , Hue and so on.

Product description:

Apache Hadoop is an open source framework for distributed computing from the Apache Open Source Organization that provides a distributed file system subproject (HDFS) and software architecture that supports MapReduce distributed computing.

Apache Hive is a data warehouse tool based on Hadoop, which can map structured data files into a database table and quickly implement simple MapReduce statistics through SQL-like statements without developing special MapReduce applications. It is very suitable for statistical analysis of data warehouses.

Apache Pig: is a large-scale data analysis tool based on Hadoop. It provides SQL-LIKE language called Pig Latin. The compiler of this language converts SQL-like data analysis requests into a series of optimized MapReduce operations.

APache HBase is a high-reliability, high-performance, column-oriented, scalable distributed storage system that uses HBase technology to build large-scale structured storage clusters on cheap PC servers.

Apache Sqoop is a tool for transferring data from Hadoop to relational databases, importing data from a relational database (MySQL, Oracle, Postgres, etc.) into Hadoop HDFS, and importing HDFS data into a relational database.

Apache Zookeeper is a distributed, open-source coordination service designed for distributed applications. It is mainly used to solve some data management problems often encountered in distributed applications, simplify the difficulty of distributed application coordination and management, and provide high-performance distributed services.

Apache Mahout: is a distributed framework for machine learning and data mining based on Hadoop. Mahout implements some data mining algorithms with MapReduce and solves the problem of parallel mining.

Apache Cassandra is an open source distributed NoSQL database system. It was originally developed by Facebook to store data in a simple format, combining Google BigTable's data model with Amazon Dynamo's fully distributed large-scale architecture.

Apache Avro is a data serialization system designed to support data-intensive, high-volume data exchange applications. Avro is a new data serialization format and transport tool that will gradually replace Hadoop's original IPC mechanism.

Apache Ambari is a web-based tool that supports provisioning, management, and monitoring of Hadoop clusters.

Apache Chukwa is an open source data collection system for monitoring large distributed systems, which can collect various types of data into files suitable for Hadoop processing and save them in HDFS for Hadoop to perform various MapReduce operations.

Apache Hama is a HDFS-based BSP (Bulk Synchronous Parallel) parallel computing framework that can be used for large-scale, large-data computing including graph, matrix, and network algorithms.

Apache Flume is a distributed, reliable, and highly compatible system for massive log aggregation, which can be used for log data collection, log data processing, and log data transmission.

Apache Giraph is a scalable distributed iterative graph processing system based on the Hadoop platform, inspired by BSP and Google's Pregel.

Apache Oozie is a workflow engine server for managing and coordinating tasks running on Hadoop platforms (HDFS, Pig, MapReduce).

Apache Crunch is a java library based on Google's FlumeJava library for creating MapReduce programs. Like Hive,Pig, Crunch provides a library of patterns for common tasks such as joining data, performing aggregations, and sorting records.

Apache Whirr is a set of libraries (including Hadoop) running on cloud services that provide a high degree of complementarity. Whirr supports Amazon EC2 and Rackspace services.

Apache Bigtop is a tool for packaging, distributing, and testing Hadoop and its surrounding ecosystem.

Apache HCatalog: Hadoop-based data table and storage management, metadata and schema management within implementation, spanning Hadoop and RDBMS, leveraging Pig and Hive to provide relational views.

Cloudera Hue is a web-based monitoring and management system that implements web-based operations and management of HDFS, MapReduce, YARN, HBase, Hive, Pig.

About "what Hadoop products have" this article is shared here, I hope the above content can be of some help to everyone, so that you can learn more knowledge, if you think the article is good, please share it to let more people see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.