Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Big data's basic learning

2025-01-21 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >

Share

Shulou(Shulou.com)06/03 Report--

What is big data?

For example:

1. Commodity recommendation: question:

(1) how to store a large number of orders?

(2) how to calculate a large number of orders?

2. Weather forecast: question:

(1) how to store a large amount of weather data?

(2) how to calculate a large amount of weather data?

If you want to learn big data well, you'd better join a good learning environment. You can come to this Q Group 251956502 so that it is more convenient for everyone to learn, and you can also communicate and share materials together.

What is big data, the essence?

(1) data storage: distributed file system (distributed storage)

(2) data calculation: distributed computing

What is the relationship between Java and big data?

1. Hadoop: based on Java language development

2. Spark: based on Scala language, Scala based on Java language

The basis and route needed to learn from big data

1. The basis for learning big data:

Java fundamentals (JavaSE)-- > classes, inheritance, Igamot O, reflection, generics *

Linux Foundation (operation of Linux)-- > create files, directories, vi editors *

2. Learning route:

(1) Java Foundation and Linux Foundation

(2) Learning of Hadoop: architecture, principle, programming

(*) Phase I: HDFS, MapReduce, HBase (NoSQL database)

(*) Phase 2: data analysis engine-> Hive, Pig

Data acquisition engine-> Sqoop, Flume

(*) Phase III: HUE:Web management tools

ZooKeeper: HA for implementing Hadoop

Oozie: workflow engine

(3) study of Spark

(*) the first stage: Scala programming language

(*) second stage: Spark Core- > Computing based on memory and data

(*) the third stage: Spark SQL-> similar to the SQL statement in Oracle

(*) the fourth stage: Spark Streaming--- >

Carry out real-time calculation (flow calculation) such as waterworks

(4) Apache Storm: similar to Spark Streaming-> perform real-time calculation (flow calculation): for example: waterworks

(*) NoSQL:Redis memory-based database

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 300

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Internet Technology

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report