Shulou(Shulou.com)06/02 Report--

1. Development platform:

Hadoop is an open source distributed computing platform under the Apache Software Foundation.

Hadoop with distributed file system (Hadoop Distributed File Syste, HDFS) and MapReduce as the core provides users with the underlying details of the system transparent distributed infrastructure.

The advantages of high fault tolerance and high scalability of HDFS allow users to deploy Hadoop on cheap hardware

MapReduce allows users to develop parallel applications without knowing the underlying details of the distributed system.

Two. Advantages:

High reliability-because Hadoop assumes that computing elements and storage will fail, because it maintains multiple copies of working data and can redistribute and process failed nodes in the event of a failure

High scalability-distribute task data among clusters and easily expand thousands of nodes

High efficiency-under the idea of MapReduce, Hadoop works in parallel to speed up task processing

High fault tolerance-automatically save multiple copies of data and automatically reassign failed tasks

III. Application and development trend:

Yahoo: run Hadoop through the cluster to support advertising system and Web search

FaceBook: (Hive contributor) uses Hadoop to support its data analysis and machine learning

Taobao: the company that uses Hadoop most deeply in China, the whole Taobao and Ali are data-driven.

Huawei: cloud computing platform

Youku: log analysis, recommendation, etc.

Qihoo 360: software, log analysis and data mining in storage software housekeeper

Baidu: the earliest company to use Hadoop in China, such as storage, log analysis, recommendation, machine learning, etc.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.