Shulou(Shulou.com)06/03 Report--

Big data refers to the method that all the data are analyzed and processed instead of a shortcut like random analysis. In the Internet era, every enterprise has to produce huge data every day, store the data, mine, analyze and apply the effective data need to rely on big data development. Big data development course uses real business data sources and the integration of cloud computing and machine learning, so that students have the strength to join first-tier Internet companies.

Today, the editor's technology sharing detailed study of big data's accurate road map, to learn big data also depends on professional tools.

Stage one, the basis of Java

Introduction to Java development, familiar with Eclipse development tools, Java language fundamentals, Java flow control, Java strings, Java arrays and classes and objects, digital processing classes and core technologies, Imando O and reflection, multithreading, Swing programs and collection classes

If you want to learn big data well, you'd better join a good learning environment. You can come to this Q Group 251956502 so that it is more convenient for everyone to learn, and you can also communicate and share materials together.

Stage 2, HTML, CSS and Java

PC website layout, HTML5+CSS3 foundation, WebApp page layout, native Java interactive function development, Ajax asynchronous interaction, jQuery application

Phase III, JavaWeb and database

Database, JavaWeb development core, JavaWeb development insider

Stage IV. LinuxHadoopt system

Linux architecture, Hadoop offline computing outline, distributed database Hbase, data warehouse Hive, data migration tool Sqoop, Flume distributed log framework

Stage 5, actual combat (real project of first-line company)

Data acquisition, data processing, data analysis, data presentation, data application

Stage VI. Spark ecosystem

Python programming language, Scala programming language, Spark big data processing, Spark-Streaming big data processing, Spark-Mlib machine learning, Spark-GraphX graph calculation, practice 1: recommendation system based on Spark (real project of a first-tier company), practice 2: Sina (www.sina.com.cn)

Stage 7. Storm ecosystem

Storm technical architecture, Storm principles and fundamentals, message queuing kafka, Redis tools, zookeeper detailed explanation, practice 1: log alarm system project, practice 2: guess you like to recommend system practice

Stage 8. Big data Analysis-AI (artificial Intelligence)

Data Analyze working environment prepares the basis of data analysis, data visualization, Python machine learning

1. Python machine learning 2, image recognition neural network, natural language processing social network processing, actual combat project: outdoor equipment recognition analysis

Big data is really a magical discipline. It seems that if you learn big data well, you will be able to cover most areas of the Internet. Just like the popular blockchain, artificial intelligence and so on are all closely related to big data's technology. Every partner who wants to learn from big data is a rare talent in the future. Conquer the world with technology.

First, get started with Hadoop to understand what Hadoop is

1. Background of Hadoop production

2. The position and relationship of Hadoop in big data and Cloud Computing

3. Introduction of Hadoop application cases at home and abroad.

4. Analysis of the employment situation of domestic Hadoop and introduction of the curriculum syllabus.

5. Overview of distributed system

6. Brief introduction of Hadoop biosphere and its components.

7. Hadoop core MapReduce example

Second, distributed file system HDFS is a basic course for database administrators.

1. Brief introduction of distributed file system HDFS

2. Introduction to the system composition of HDFS

3. Detailed explanation of the components of HDFS

4. Copy storage policy and routing rules

5 、 NameNode Federation

6. Command line interface

7. Java interface

8. Explain the data flow between client and HDFS

9. Availability of HDFS (HA)

Third, junior MapReduce, becoming a basic course for Hadoop developers

1. How to understand the computing model of map and reduce

2. Analyze the execution process of MapReduce job under pseudo-distributed environment.

3. Yarn model

4. Serialization

5. The type and format of MapReduce

6. Build the MapReduce development environment

7. MapReduce application development

8. More examples, familiar with the principle of MapReduce algorithm

Advanced MapReduce, a key course for senior Hadoop developers

1. Use compression separation to reduce input size

2. Using Combiner to reduce intermediate data

3. Write Partitioner to optimize load balancing.

4. How to customize the collation

5. How to customize grouping rules

6. MapReduce optimization

7. Programming practice

Hadoop Cluster and Management is an advanced course for database administrators.

1. The construction of Hadoop cluster

2. Monitoring of Hadoop cluster

3. Management of Hadoop cluster

4. Run the MapReduce program under the cluster

Basic knowledge of ZooKeeper to build the basic framework of distributed system

1. ZooKeeper embodiment structure

2. Installation of ZooKeeper cluster

3. Operate ZooKeeper

Basic knowledge of HBase, column-oriented real-time distributed database

1. HBase definition

2. Comparison between HBase and RDBMS.

3. Data model

4. System architecture

5. MapReduce on HBase

6. the design of the table

VIII. HBase Cluster and its Management

1. Explain the process of building a cluster.

2. Cluster monitoring

3. Cluster management

IX. HBase client

1. HBase Shell and demo

2. Java client and code demo

Basic knowledge of Pig, another framework for Hadoop computing

1. Overview of Pig

2. Install Pig

3. Use Pig to complete mobile phone traffic statistics.

Hive, a Hadoop framework for computing using SQL

1. Basic knowledge of data warehouse

2. Hive definition

3. Brief introduction of Hive architecture

4. Hive cluster

5. Brief introduction of client

6. HiveQL definition

7. Comparison between HiveQL and SQL

8. Data type

9. The concept of table and table partition

10. Table operation and CLI client demonstration

11. Data import and CLI client presentation

12. Query data and CLI client presentation

13. Data connection and CLI client presentation

14. Development and demonstration of user-defined function (UDF)

12. The framework of data conversion between Sqoop,Hadoop and rdbms

1. Configure Sqoop

2. Import data from MySQL to HDFS using Sqoop

3. Use Sqoop to export data from HDFS to MySQL

XIII. Storm

1. Basic knowledge of Storm: including the basic concepts of Storm and Storm applications.

Scenario, architecture and fundamentals, comparison between Storm and Hadoop

2. Storm cluster building: describe in detail the installation of Storm cluster and common problems during installation.

3. Introduction of Storm components: spout, bolt, stream groupings, etc.

4. Reliability of Storm messages: retransmission of failed messages

5. The integration of Hadoop 2.0 and Storm: Storm on YARN

6. Storm programming practice

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.