Shulou(Shulou.com)06/02 Report--

As a natural product of the mobile Internet era, data intelligence is also the core of a long period of development in the future. This paper mainly describes the nature of data intelligence and technical system requirements.

From 1956 Dartmouth Conference first put forward the concept of "artificial intelligence", to now a new round of scientific and technological revolution and industrial change is in the ascendant, big data, cloud computing, 5G and so on are well known to the public. The era of big data is unstoppable, especially the big data application in the vertical segmentation industry shows great potential. Big data combines organically with the business scenes of various industries to promote the digital transformation and upgrading of the subdivision industry and promote the innovative development of social governance. At the same time, a large number of new models and new business type were born.

The development of China's mobile Internet has witnessed the vigorous development of China's big data industry. As a natural product of the mobile Internet era, data intelligence is also the core of a long period of development in the future.

This article is mainly based on many years of practice in the field of data intelligence, to explain the nature of data intelligence and technical system requirements, and then will share with you about the data platform, data system construction of the essence, ideas and methods. I hope to use popular language to enable technicians, products or operators to have a clear understanding of data intelligence and the technical system involved, so as to bring some new inspiration to everyone's daily work.

1. The Development of big data

What is data intelligence and where does this concept come from?

I remember that since 2010, with the rise of the mobile Internet, big data has also appeared in various media websites and industry forums, everyone will ask: "did you do big data?" In fact, no one knows exactly what big data should do, .

As for what big data can do, I found a picture that I agree with around 2013:

I call it big data maturity model. In essence, we understand that this process is also a process in which data changes from a tool to an asset, from an auxiliary thing to a means of production. Many people are trying to make a theoretical definition of the digital economy in order to distinguish the digital economy from the real economy conceptually. My suggestion is to define it from the point of view of whether numbers are the main means of production and whether they are core assets.

Judging from the actual development in recent years, big data is basically evolving according to the model shown above.

Around 2013, enterprises have begun to recognize the value of data, and various industries with big data production environment, such as telecom operators, government, public security and finance, have begun to build big data platform to collect and store data generated by enterprise business. At the same time, financial and other industries also began to buy a large number of external data, hoping to quickly mine the value of data through external data to make up for their own data shortage. Many companies engaged in data aggregation and related services have obtained development opportunities.

In 2015, big data entered the monitoring stage, through the data screen and other forms to achieve business monitoring, which is big data's earliest and first mature application direction. For the government, central enterprises and large state-owned enterprises, data display applications such as big data screen and leadership Kanban are the most direct ways to reflect big data's value.

In 2017, the construction of big data platform was basically perfect, and simple data presentation began to be difficult to meet the diversified needs of enterprises. Big data began to combine with business scenarios, showing a situation of letting a hundred flowers blossom based on big data's insight into business problems. respectively applied in the financial field of precision marketing and risk control anti-fraud, criminal investigation in the field of public security, fault prediction and early warning in the industrial field.

Simple mathematical statistics are not enough to meet the requirements of enterprises' insight into business scenarios. Therefore, data mining and data modeling technology arises at the historic moment. AI modeling platform, data science platform began to enter people's field of vision, there are some start-up companies that focus on the modeling platform, but more companies internalize the AI modeling platform into their own capabilities, based on the AI modeling platform to form solutions to help enterprise customers implement big data applications.

Around 2019, big data began to enter the business decision-making stage, that is to say, data reports or data reports are formed by the machine, and the business staff make decisions, so that the machine directly gives decision-making suggestions, so that the machine has the reasoning ability. For example, in takeout and travel scenes, Meituan and Didi's system directly forms the best scheduling mode, and the system automatically completes the decision-making process and assigns tasks to riders and drivers. This kind of consumer Internet relatively common scene, will gradually appear in the industrial Internet, enterprise business scene. In other words, big data began to move forward from the business digitization stage to the data intelligence stage.

two。 Characteristics and Definitions of data Intelligence

From the development of big data in the previous section, we can see that data intelligence now corresponds to the stages of decision-making, optimization and business reshaping. In other words, machines have the ability to reason, and these capabilities mean the gradual maturity of cognitive technologies such as natural language processing (NLP) and knowledge graph (Knowledge Graph), which is why NLP and knowledge graph become hot spots in the market in 2018.

Therefore, the new demand of data-driven decision-making and data-driven business development will inevitably lead to the rise of a number of data intelligence companies.

In the future, as the technology becomes more mature, big data will move from decision-making to the last link, that is, business reshaping. Many execution links can be realized by machines, but there are still many links that people need to participate in. Therefore, human-computer cooperation will usher in the rapid development, from artificial intelligence AI (Artificial Intelligence) to human intelligence enhanced IA (Intelligence Augmented).

At this point, we try to define data intelligence:

Data intelligence is to extract, discover and acquire knowledge from a large amount of data by combining large-scale data processing, data mining, machine learning, human-computer interaction, visualization and other technologies. Provide effective data intelligence support for people in decision-making, reduce or eliminate uncertainty.

3. Technical system of data Intelligence

First of all, data intelligence needs to be provided by data, and data plays the role of core assets and means of production, so data governance is particularly important.

What is data governance (Data Governance)?

We often hear the word corporate governance, which mainly solves several problems in economics:

How do ownership and management rights be separated? How do company owners authorize and supervise professional managers scientifically?

Accordingly, data governance has to solve several similar problems:

What are the data (assets)? How to separate the ownership of data from the right of use? How do data asset owners scientifically authorize and supervise data users?

All the means of data intelligence are actually solving the above problems.

I will describe in detail about data governance in a later article.

At the same time, we know that the difference between the rich and the poor lies in their attitude towards wealth. The rich are more likely to treat wealth from the perspective of asset appreciation, thinking about how to create more assets and let them continue to grow in value; the poor tend to look at wealth from the perspective of consumption, and most of the money earned is spent on consumption.

So in the era of data intelligence, if we want to be a "rich man", we need to consider how to make the data more valuable and how to find other partners to jointly create value, but the data is different from other assets. it has the nature of replicability and uncertainty, which requires us to solve the problem of data security, that is, the secure computing technology that is more concerned in the industry at present.

Another point we need to pay attention to is that because of its 4V characteristics, especially its large quantity and variety, big data sometimes makes us doubt its aggregation or results, although some can be judged by common sense or intuition. But there's always something unspeakable. This requires a quality assurance system to allow us to have a complete inspection process from the generation to the beginning of the data.

To sum up here, the technical system of data intelligence needs to include at least three aspects:

Data governance system data quality assurance system data security computing system conclusion

As an important and exciting stage in big data era, data intelligence has both opportunities and challenges.

As the beginning of this series, this article gives an overall overview of the topic, and the specific content will be carried out step by step. I hope it will be helpful to all of you.

Http://www.woshipm.com/ai/3702180.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.