Shulou(Shulou.com)06/03 Report--

Need to build a huge cdh system//write a blog series from scratch to have their own ideas sublimation!

According to this system, optimize accordingly

Concept:

1. Selection of hardware resources before building cdh//Here, we need to choose what kind of hardware CPU hard disk manufacturers are for the hadoop ecosystem.

2. Set up cdh version control, corresponding information of each version

3. Linux optimization before building cdh production environment

4. Role planning when building cdh clusters

5. Set up cdh production environment

6. HDFS cluster optimization after building cdh

7. Relevant optimization of hbase cluster after cdh establishment (HDFS related configuration optimization, HBase server optimization (GC optimization, Compaction optimization, hardware configuration optimization), column family design optimization, client optimization, etc.)

cdh setup process

The general process is divided into these parts

1. Early optimization

2, mysql build

3、jdk

4、jdbc

5, CDH specific construction

//Here, you need to understand the performance tuning of 4 major components hdfs hbase yarn kafka

Start with hdfs hbase

4. The author summarizes at least the following aspects:

//HDFS related configuration optimization, HBase server optimization (GC optimization, Compaction optimization, hardware configuration optimization), column family design optimization, client optimization, etc.,

Client-side optimization has been discussed through timeout mechanism and retry mechanism in the previous section, and the author will continue to introduce the other three optimization points respectively later.

Important point, need to upgrade to view some parameters of the official website of cdh ****

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.