Shulou(Shulou.com)06/03 Report--

In two years, big data grew tenaciously all over the prairie like weeds after the rain. Whether it is the Internet, financial, medical, automobile, it seems that big data is not mentioned, as if he is out of date.

Recently, the author's team has made a feasible plan for big data for a certain part of the research institute, and there are some things that I happen to share with you.

Big data, as the name implies, the amount of data is very large, according to the definition, the amount of data is so large that it can not be solved by traditional means. The so-called traditional means is the IOE noted in the industry, that is, high-performance servers + dedicated storage devices + database devices. In fact, although the word big data is popular in recent years, big data has always existed, but it is due to the rapid development of the Internet and mobile Internet that the growth rate is getting faster and faster in recent years. At the same time, the rapid development of the Internet also makes foreign Google, FaceBook, domestic BAT to mine the deep value of the data, and further promote its development.

Big data's rapid development in the Internet has made various countries step in one after another. after the United States released the government's public big data plan, the European Union and China also joined in this battle. In China, the government's big data public plan has just begun, and the research institutes under various ministries and commissions and the major central enterprises have also begun to say that they have, are, and will immediately carry out the big data project. However, for the big data project, the information personnel of the government, enterprises and institutions, are you ready? Here are some suggestions for you from several angles.

The data source is the foundation of everything.

In fact, no matter big data or small data, the effect lies in the value and the starting point lies in the data. If there is no suitable data source, or if the data source still uses the traditional paper document management, then big data is of no value to you. The first step you need to do is how to use information technology to quantify the enterprise's data. The informationization of data is the foundation of everything. Without a solid data source, everything is nonsense.

two。 The goal of the business is the direction of data value.

This has been mentioned in many articles, but many of them want you to decompose the business scenario according to your business goals, and then model the data, and so on. What I'm trying to say here is that business goals are important, but you can't expect business leaders or business people to tell you that you want roast chicken wings as well as how to make them. In many cases, after leading you in, what is left is what the data analyst needs to do, including the decomposition of business objectives, including the selection of data sources, including the determination of data collection methods, including the establishment of data models. What the user needs to do is to make sure that there is a big gap between the results you have done and his requirements.

3. The choice of technical solution is also very important.

For users, the technical solution is like a wordless book, it all looks high-end, but as a technician or data analysis provider, you need to consider the issues do not know high-end, as well as landing. The chances of success of the big data project in China's current state-owned enterprises are pitifully low. The reason is that the technical solution can not be landed, when you do the technical solution, do you scientifically estimate the amount of data? What will you do when the amount of calculation is inestimable? Do you choose virtualization as the management of clustered servers? How are physical servers managed? Is the calculation content more batch processing or more stream computing? What algorithms may be needed to support the business? When I was working on the big data project, one of the difficulties I encountered was whether to choose a cluster with virtualized architecture or a cluster with physical servers. What's the difference between the two.

4. The operation of data should be foreseen at the beginning of the project.

Very often, to do a big data project is not only to analyze the results, but also to do a data operation, because the user's data is more of the future data. How to make your big data project adapt to the future development of users should be foreseen at the beginning of big data's project, and it is also necessary to make enough designs for users in the planning process. If we do not consider how the data will come, how to collect, how to use, how to exert value, and how to support the user's big data project after the analysis, the user's big data can only be limited to one or two years. Although the real big data can not be like a perpetual motion machine, it should be a windmill anyway. As long as there is wind, it can go on.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.