Shulou(Shulou.com)06/03 Report--

Get to know big data

What is big data? Some people may say that the information of everyone in the office building is big data. NO! The data here can only be said to be relatively large, but it cannot be called big data. Baidu encyclopedia gave a very clear explanation: "big data (big data), refers to the data set that can not be captured, managed and processed with conventional software tools within a certain time range. It is a massive, high growth rate and diversified information asset that requires a new processing model to have stronger decision-making power, insight and process optimization ability."

Here I still want to recommend the big data Learning Exchange Group I built myself: 529867072, all of them are developed by big data. If you are studying big data, the editor welcomes you to join us. Everyone is a software development party. Irregularly share practical information (only related to big data software development), including the latest big data advanced materials and advanced development tutorials sorted out by myself. Welcome to join us if you want to go deep into big data.

Big data has five characteristics (proposed by IBM): Volume (mass), Velocity (high speed), Variety (diversity), Value (low value density) and Veracity (authenticity). Among them, Volume is generally believed that the data is large enough, so large data can not be said to be big data, in other words, large data is only one of the characteristics of big data.

The strategic significance of big data's technology is not to master huge data and information, but to deal with these meaningful data professionally. In other words, if big data is compared to an industry, the key to profitability of this industry lies in improving the "processing ability" of data and realizing the "value-added" of data through "processing".

Now we "from acquaintance to acquaintance", a clear understanding of what is big data, if we distinguish between big data and big data, is the first step for us to learn from big data.

How to start learning

After you have the "first brick", it is time for you to choose the division door. Knocking on the "second brick" of the mountain gate is the basis for learning big data, just like practicing internal skills in the school, which will help you walk the rivers and lakes. Needless to say, let's see what foundations will be involved.

1. JavaSE,EE (SSM)

Ninety percent of big data's frames are written by java.

Such as: MongoDB-- 's most popular, cross-platform, document-oriented database. Hadoop-- is an open source software framework written in Java for distributed storage and distributed processing of very large data sets.

Spark, the most active project in Apache Software Foundation, is an open source cluster computing framework.

Hbase-- open source, non-relational, distributed database, using Google BigTable modeling, written in Java, and run on HDFS.

2. It is the foundation and tools in big data.

In order to build a solid foundation for a high-rise building, we must lay a good foundation, master the necessary knowledge of Linux, be familiar with the use of python and the compilation of crawlers and build the foundation of Hadoop (CHD), so as to lay a good foundation for learning big data's skills.

Advanced technology

1. Big data's offline analysis.

Master big data's core basic components: HDFS,MapReduce and yarn. Master MapReduce programming ideas and general big data computing platform: "spark"

Through the actual combat project, you can be familiar with the background of user behavior analysis business, and master the process of offline data processing (user analysis project is a classic offline processing project), architecture and the application of related technologies.

2. Big data real-time calculation

Master the mainstream technology component of real-time processing: kafka,spark streaming,flink,storm,hbase

And then integrate your own learning through the real-time transaction monitoring project.

Conclusion: I hope that the friends who are interested in big data can play a heuristic role, and the learning of the method also needs to be studied assiduously and integrated on the basis of interest.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.