Shulou(Shulou.com)06/03 Report--

What exactly does Big Data mean? Although we all know the high salary, how to learn big data? What are the learning paths and methods? Today we will take a specific look

What is Big Data?

Let's look at the Wikipedia definition.

Big data (also known as "Big data" or "Megatta") refers to the amount of data involved that is so large that it cannot be intercepted, managed, processed, and organized into information that humans can interpret within a reasonable time.

For the same total data volume, analysis of small data sets combined can yield a lot of additional information and data relationships than analyzing small data sets individually, which can be used to detect business trends, determine the quality of research, prevent the spread of disease, combat crime, or determine real-time traffic conditions; such uses are the reason why large data sets prevail.

The above paragraph seems to be more around, you can look at the popular explanation together:

If you are responsible for product recommendations on Taobao, you want to know whether users who buy jewelry will also buy electronic products, and then decide whether to recommend Samsung.

Under this condition, it is necessary to call the user data of the previous period (for example, one year). Only through the proof of a large amount of data can it be confirmed whether there is correlation between the two. If traditional data processing methods are used, it will take a lot of time. When the positive correlation is confirmed, Samsung's promotion period has passed, and the daily data volume such as Taobao and Jingdong is often counted in TB, so it is necessary to quickly process, analyze and give accurate and appropriate recommendations. This is the role of big data.

. In the process of getting started learning big data, there are encounter learning, industry, lack of systematic learning route, systematic learning planning, welcome you to join my big data learning exchange skirt: ×××, skirt file has my big data learning manual, development tools, PDF documents books, you can download by yourself.

Work related to big data?

In the United States, positions related to big data are collectively referred to as "data scientists"; in China, positions related to big data are subdivided into four categories: data analysis, data mining, data engineer and data architect.

·Data analysis: using tools to extract, analyze, and present data to achieve business significance

Data mining: machine learning, algorithm implementation

·Data Engineer: Developing and using simple data tools to achieve data modeling and other functions, requiring business understanding

·Data Architect: Advanced algorithm design and optimization; data-related system design and optimization, with vertical industry experience

About Big Data Learning

A lot of people are asking how big data processing technology can be learned.

Here, for big data engineers, we give a specific learning path

java basics---linux---hadoop---hive, hbase---scala-spark

First of all, we have to learn Java language and Linux operating system, these two are the basis of learning big data, and the order of learning is not divided.

Java: We all know that Java direction has JavaSE, JavaEE, JavaME, learning big data to learn which direction?

Only need to learn Java standard version of JavaSE can be, like Servlet, JSP, Tomcat, Struts, Spring, Hibernate, Mybatis are JavaEE direction of technology in the big data technology is not used much, just need to understand it;

Of course Java how to connect to the database or to know, like JDBC must master it, some students say Hibernate or Mybites can also connect to the database ah, why not learn about it, I am not here to say that learning these is not good, but that learning these may take you a lot of time, to the final work is not commonly used, I have not seen who uses these two things for big data processing, of course, your energy is sufficient, you can learn Hibernate or Mybites principle, do not only learn API, This will increase your understanding of Java operating databases, since at the heart of both technologies is Java reflection plus various uses of JDBC.

Linux： Because big data-related software is running on Linux, so Linux to learn a solid number, learn Linux for you to quickly master big data-related technologies will be of great help, can let you better understand hadoop, hive, hbase, spark and other big data software operating environment and network environment configuration, can step on a lot of pits, learn shell can understand scripts so that it can be easier to understand and configure big data clusters. It also allows you to learn new big data technologies faster.

The other skills can be learned sequentially.

The other two basic disciplines must also be cultivated:

·Statistics

·Computers (and perhaps some machine learning)

These two disciplines are the foundation of big data foundation, and crossing these two hurdles will qualify you for big data work. So some people say that a big data engineer is a programmer who is proficient in statistics, and a statistical dog who cannot program is not a good big data expert.

Statistics: multivariate statistical analysis, applied regression

Computers: R, Python, SQL, Data Analytics, Machine Learning

Matlab and Mathematica two software is also need to master, the former in the actual engineering application and simulation analysis has great advantages, the latter is in the calculation function and mathematical model analysis is very good, mutual support can complement each other.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.