Shulou(Shulou.com)06/02 Report--

For hadoop beginners, or developers who are using hadoop, building a hadoop environment is not an easy task, and even many blogs say, "Don't spend energy on building the environment." it can be seen that many people will encounter a lot of problems when building the environment, and will spend a lot of time. This article will go through all the "games". I believe you will have a good idea after reading it. Will choose the appropriate construction method according to their own needs.

Deployment mode

advantage

Deficiency

Suitable for the occasion

Apache Hadoop

Single machine

A single machine will do.

simple

Few components are required

Generally not used in production

No HA

Unable to reflect the distribution

Beginner

Development and testing

Small-scale trial

Apache Hadoop

Cluster

Flexible version selection

Self-control is better.

Wide range of applications

Professional management is required.

Poor compatibility between components

Configuration, operation and maintenance are complex.

Study

Development and testing

Production environment

CDH or

HDP

Web Management and Monitoring

Open source vendor support

High compatibility and stability

A lot of configuration is still needed.

Be under the control of the manufacturer

The update version is a little slow

Development and testing

Production environment

CDH from other vendors

Has its own extended features

Vendor support

Not free

Be seriously controlled by the manufacturer

Production environment

Write shell deployment operation and maintenance script

Good self-control

Simple configuration

Good flexibility

Need to write a script

Testing takes time

Need to be perfected constantly

Study

Development and testing

Production environment

Through the comparison above, the summary is as follows:

Beginners to learn hadoop, want to start quickly, using the first apache stand-alone, no foundation in the case of a small can be completed, with linux foundation to remove the installation of virtual machines, linux time, 10 minutes to complete

Used in production environment or test environment, using the third cdh mode, the management cluster is graphical, but lack of in-depth understanding of the internal

For in-depth learners, have some experience and accumulation, you can choose the last one, continue to in-depth understanding of the internal process dependencies, but also improve the level of shell script file programming.

The detailed construction of each environment will be described in several chapters and, if possible, some free videos will be recorded to explain the steps in detail.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.