Shulou(Shulou.com)06/03 Report--

I. Overview

1 brief introduction of big data

1.1 Origin

"big data" is one of the hottest words in recent years. Although the word big data came into being only about 10 years ago, the analysis of big data has been around for a long time. As early as in the early days of the Internet, many companies used computer technology to analyze and deal with a lot of things, such as various browsing engines. However, big data's real proposal came from a paper in the special issue of Nature on September 03, 2008.09.03, and then the industry continued to follow up. McKinsey released the McKinsey Global Research report in 2011.06, marking the real rise of big data in the industry. With the White House issuing the big data R & D Act, the government began to join big data's competition.

1.2 definition

Since big data is so hot, it is necessary for us to find out what big data is. We often use 4 V to define big data: capacity, diversity, throughput, value. That is, big data must be a large number (at least T, P level), many sources, most of them are unstructured, and the speed of going in and out of the analysis system is fast, and the purpose of obtaining value is to obtain data.

2 big data characteristics of mobile Internet

2.1web2.0 and Generalized Mobile Internet

Web2.0 refers to the information exchange and sharing Internet represented by moments and Weibo, while the broad mobile Internet connects the Internet, the Internet of things and social networks through wireless means.

2.2 data characteristics

The data of mobile Internet has the characteristics of mobility, complexity and sociality. First of all, the node is mobile, and it has the function of universal perception. Secondly, the network is complex, multiple perception can be carried out through the network, and finally the individual is social, so he also has the role of social perception.

2.3 data types

Mobile Internet produces two types of data: one is data transmitted by people (UGU), which originates from people's needs for self-expression. One is machine-generated data (MGC), which comes from the needs of science and technology, military and business.

2.4 IntelliSense & Service

At present, the mobile Internet has a missing chain-IntelliSense-service. We know that the Internet solves the problem of information exchange between people, while the Internet of things solves the problem of information exchange between things. The intelligent service needs the cross-perception between human, nature and society, and the mobile Internet and big data technology are its bridges.

2.5 Smart City

In my opinion, smart city = digital city + mobile Internet + Internet of things + cloud computing, and to achieve it requires mobile Internet to integrate the Internet, interpersonal network and Internet of things.

3opportunities and challenges

3.1 opportunity

Big data has brought us opportunities and challenges, whether we can benefit from it depends on how we deal with these opportunities and challenges. Big data's opportunity is obvious. The data collection and disclosure of various large platforms, the opening of data analysis platforms such as MapReduce, and the provision of data mining services in various fields make it easier for us to obtain data. And these rich data have brought many opportunities for innovation, and the data in any field may have a great impact on this field.

3.2 Challen

Of course, big data also brings us a lot of challenges. First, the contradiction between data sharing and data private ownership. Big data's value is sparse, and a large amount of data is often monopolized by large companies, so for ordinary people, data sharing becomes very important, and one of the solutions is to establish a shared data center. Second, the contradiction between the flood of data and the lag of technology. First of all, there is a mismatch between data storage capacity and processing, which can be solved by real-time data flow processing, nearest principle storage and processing of original data, purchase of data storage and analysis services and so on. In addition, the analysis means and performance requirements do not match, the main reason is that the traditional data warehouse is no longer used in big data analysis, for which we can use large-scale concurrency, Map-Reduce distributed computing, NoSQL management of concurrent access and other methods. The third is the contradiction between social demand and the shortage of talents. In this regard, it is urgent to train excellent big data talents. Fourth, the contradiction between opening up data and protecting privacy.

It includes the problem that user privacy becomes a victim and may endanger national security. Our solution is to develop privacy protection data mining methods and improve legislation.

(4) Future development trend

I think big data will be the oil of the future, and the mobile Internet will become the main way to surf the Internet, and Mobile big data will also flourish. Here make a few predictions: 1 Mobile big data analysis will gradually become the research focus of cloud computing and the Internet of things. (2) the deep integration of mobile Internet UGC and MGC data will give birth to new industries. (3) data analysis services focusing on local areas will become the mainstream of industrial innovation in the near future. 4Map-Reduce will remain active, and distributed stream data analysis will become a hot topic in machine learning theory and application research. (5) data sharing is the general trend, but we need to pay special attention to national information security, open data need legislative support, and information security needs independent technical guarantee.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.