Shulou(Shulou.com)06/03 Report--

In the era of big data, data has become a valuable asset of society and organizations, driving everything like oil and electricity in the industrial age. however, if there are too many impurities in oil and the voltage of the current is unstable, the value of the data will be greatly reduced. It is even impossible to use it at all. therefore, data governance is the inevitable choice for us to make good use of massive data in big data era.

But as we all know, data governance is a long-term and complicated work, which can be said to be the dirty work in the field of big data. Very often, data governance manufacturers have done a lot of work, but customers think that they have not seen any results. Most data governance consulting projects can hand in an answer that makes the customer satisfied enough, but when the consulting results are put into practice, it is likely to be a completely different landscape for a variety of reasons. How to avoid this situation is a problem worth pondering for every enterprise that does data governance.

Cdn.xitu.io/2019/7/24/16c2292f15f1b496?w=865&h=487&f=jpeg&s=54959 ">

The author has dabbled in the field of big data governance for more than 6 years and has been responsible for data governance projects in government, military, aviation, and large and medium-sized manufacturing enterprises. In practice, there have been successful experiences, and of course, I have also experienced many lessons of failure. In these processes, I have been thinking about what big data governance is really about. What reasonable goals should be achieved? How to avoid some detours in the middle? Today, I would like to take this opportunity to share with you the pit I have been in, hoping to give you some reference.

Misunderstanding 1: customer needs are not clear

Since customers have invited manufacturers to help them do data governance, they must have seen all kinds of problems in their own data. But what to do, how to do, how to do, what to do first and then what to do, what goals to achieve, and how to work together among business departments, technical departments, and manufacturers, many customers do not think clearly about the problems they really want to solve. It is difficult to find a starting point for data governance.

Myth 2: data governance is a matter for the technical department

The causes of data problems are often business > technology, such as: many data sources and unclear responsibilities, resulting in different expressions of the same data in different information systems; unclear business requirements, non-standard or missing data reporting, and so on. Many superficial technical problems, such as data processing errors caused by a code change in the ETL process, affecting the correctness of the data in the report, are actually non-standard in business management in essence.

When I communicated with many customers on data governance, I found that most of them did not understand the root cause of the data quality problems and only wanted to solve the data problems unilaterally from the technical dimension. this way of thinking leads to that when planning data governance, customers simply do not consider the establishment of a strong organizational structure and institutional processes that cover technical groups and business groups. As a result, the effect is greatly reduced.

Misunderstanding 3: large and comprehensive data governance

For the sake of return on investment, customers tend to do a large and comprehensive data governance project covering the whole business and technology domain. From the generation of data to processing, application and destruction, they hope to manage the whole life cycle of data. From business systems to data centers to data applications, they want every data in it to be included in the scope of data governance.

But do not realize that data governance in a broad sense is a big concept, including a lot of content, it is usually impossible to finish it in a project, but need to be implemented in stages and batches, so if manufacturers succumb to the idea of customers, it is easy to lead to the last one can not be done well, can not be used. So we need to guide our customers to follow the 2can8 principle-- 80% of problems arise from 20% of systems and data-- and start with data governance from the core systems, the most important data, and the places where problems are most likely to occur.

Myth 4: tools are omnipotent

Many customers think that data governance is to spend some money, buy some tools, and think that the tool is a filter. Once the filter is done, as soon as the data passes through the middle, there will be no problem. The result is: on the one hand, more and more functions are done, on the other hand, after the line, the function is complex, and users do not want to use it.

In fact, the above idea is a kind of simplified thinking, data governance itself contains a lot of content-organizational structure, institutional processes, mature tools, on-site implementation and operation and maintenance-these four are indispensable, tools are only part of the content. What is most easily ignored in data governance is the organizational structure and staffing, but in fact, all activity processes and institutional norms need people to implement, implement and promote. Without personnel arrangements, it is difficult to ensure the follow-up work. It is suggested that when doing data governance, we should put the organizational structure first. With the existence of an organization, some people will think about the work in this area, how to promote and continue to do things well, and people-centered data governance work. It's easier to promote landing.

Misunderstanding 5: it is difficult to land data standards.

As soon as many customers talk about data governance, they immediately say that we have a lot of data standards, but none of these standards have landed, so we have to do the landing of data standards first. If the data standard is really on the ground, the quality of the data will be better.

But this statement actually confuses data standards with data standardization. First of all, we must understand a truth: data standards must be done, but data standardization, that is, the landing of data standards, needs to be implemented on a case-by-case basis. There will be a special article to discuss data standards and standardization work.

Myth 6: the data quality problem has been found, and then what?

Work hard to set up the platform, business and technical personnel work together, configure the data quality check rules, but also find a lot of data quality problems, and then what? Six months later, a year later, the same data quality problems still exist.

The root cause of this problem is that there is no closed loop of data quality accountability. In order to achieve the accountability of data quality problems, we need to determine the responsibility of data quality problems first. The basic principle of determining responsibility is: who produces, who is responsible. Where the data comes from and who is responsible for dealing with data quality issues. Accountability is followed by accountability, followed by rectification and feedback, followed by a new round of evaluation of quality issues, until the formation of performance appraisal and ranking. Only by forming this kind of working closed loop can the data quality be really improved.

Myth 7: you don't seem to have done anything?

Many data governance projects are difficult to accept, and customers often ask: what on earth have you done in data governance? Seeing that you have reported that you have done a lot of things, why can't we see anything? The reason for this situation is that the customer demand mentioned in the previous misunderstanding is not clear, and the misunderstanding three mentioned has done large and comprehensive data governance and is difficult to finish, but another reason can not be ignored. that is, customers are not aware of the results of data governance, which can be visualized in the visual presentation of the results, as well as in the process of communication, training and knowledge transfer with customers. Exert an imperceptible influence on customers on the importance and value of data governance.

[summary]

In the fierce market competition, big data manufacturers put forward various concepts of data governance, some of which cover the whole life cycle of data, and some of which are user-centered self-service data governance. some propose automatic data governance based on artificial intelligence to reduce manual intervention and save cost. when facing these concepts, on the one hand, we should have a clear understanding of the current situation of data. There is a clear demand for the goal of data governance, on the other hand, we also need to know all kinds of common misunderstandings in data governance. Only by crossing these traps can we really implement the data governance work, achieve results in the project, and make the data more accurate. Data is easier to get, data is easier to use, and data is really used to improve business level.

Brief introduction of the author: Jiang Zhenbo, 6 years + big data governance experience, good at providing customers with scientific and reasonable data governance solutions. He has successively worked for Longtong, Ruotong Power, Puyuan Information and other companies, and has been responsible for data warehouse construction, BI, big data platform, data governance and other pre-sales consulting, with experience in government, power, manufacturing and other industries. At present, he works as a pre-sales consultant for big data platform in Qilan Technology.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.