Shulou(Shulou.com)06/03 Report--

For a long time, I felt very confused about big data's study. I don't know exactly what to learn! This leads to a lot of knowledge points, and the content of their own can not form a good system, and then add points to their own workplace. Recently, I have been learning about the architecture of large numbers, and then specific to a manufacturer. On the contrary, you can learn very quickly. Sum up your previous study and review the old and know the new!

First of all, big data began to enter the public as a concept and landed in the actual business in 13 years. From the perspective of the development of a technology, this technology will form a good closed loop in 18 years. During this period, no matter whether you are big data's project or not, you can benefit as long as you wear the name big data in these five years.

Therefore, the first thing big data can do is to cheat! Whether you are big data's project or not, as long as you know a little bit about distribution, know a little bit about Hadoop,Spark. You can say that I can undertake related projects, which is related to adventure. Because when you can not have a comprehensive understanding of big data, but rely on continuous learning in the actual project and then guide and optimize the work. This is tantamount to getting half the result with twice the effort! So returning to the direction of the topic is how to enrich your knowledge from the solution of the technical architecture.

1, Stora

Big data, as the name implies. That is, you have a large amount of data, and the traditional data are EXCEL,TXT,WORD. At first, these can also be stored on the hard drive. But to a certain extent, it is bound to affect the speed of your boot, at this time, it is necessary to introduce the concept of database to store. And when you use the database to store data, you have to involve the dialogue language that wants you to talk to the database-SQL statement.

After the completion of single-point, structured data, your work efficiency has been greatly improved. When people in neighboring towns look so good, they begin to deploy accordingly. But the content of the database still needs to be linked to the headquarters. This forms what we often call distributed storage.

With the unbalanced development of economy, marketing activities are different in different regions. The information of the user to be stored is also different. This results in a decline in the efficiency of traditional structured storage, which is changed to the storage of unstructured data related to a single ID+.

All right, after saving the data, put it there, the one on the hard disk is called Hadoop, and the one in memory is called Spark.

2, calculation

Any data must be involved in the operation, and CPU must be called in the process of operation, while the traditional handsome and rich mainframe has an advantage in high-performance computing. But when big data poured in like the sea, these handsome and rich mainframes didn't work well.

Or the wisdom of our ancestors is superior, divide and rule. This is not only the idea of flood control, but also to solve the loss of a handsome and rich man who is at a loss in the face of many requests. Technically, 100 PC or minicomputers are used to divide and conquer the data. And then uniformly report the solution. This is what big data often says: a hundred poor strands is equivalent to a rich and handsome man! This is the first calculation scheme.

Then what else is it? Similar to Zhou Botong's fight between the left and the right, you think about what you can do with a PC, and I add a CPU, is it equivalent to simulating two sets? The advantage of this calculation is that the data can be classified and processed, which is technically achieved through virtualization.

3, visible

After talking about the previous basics, let's take a look at the data presentation, there are many kinds of data presentation, and this group of people need code people to have some artistic cells (what, no! That's fine with art germs.

After all, present the results of your calculations to the public. It is very important that the corresponding logic is clear on one point. In this regard, it is recommended to read more excellent works of others. More importantly, you should have some knowledge of psychology, data highlighting and strategic control. It can be said that the success or failure of the big data project basically depends on whether it can produce results in the visual area.

All right, that's all for this time. In the future learning process, and then slowly sum up!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.