Shulou(Shulou.com)06/03 Report--

With the gradual deepening of the integration of enterprises and digital technology, more and more enterprises feel unprecedented anxiety on the road of digital transformation. They believe that the organization already has economically valuable data resources and expect it to bring more future benefits to the organization like other assets.

Such as employee assets, like data, employees are an important part of corporate assets. However, compared with data asset management, it is still in the early stage in domestic enterprises. Most enterprises have formed a set of standardized processes and relatively mature management strategies in the selection and use of employees. Based on this, should we consider dealing with data in the same way as managing employee assets? At this time, the data strategy is particularly important.

I. definition of data strategy

Data policies are methods for obtaining, integrating, storing, protecting, managing, monitoring, analyzing, using, and manipulating data. A modern and comprehensive data strategy is not just about data, it is a roadmap that defines people, processes, and technologies, and illustrates how data will implement and inspire business policies.

These include: a practical guide for articulating the goal vision and achieving it, and clearly articulating success criteria and key performance indicators that can be used to evaluate and rationalize all follow-up data plans. It does not include detailed solutions to specific technical problems. With the development of enterprise goals, data strategy is not immutable, and needs to keep up with the technological innovation and operation mode of the enterprise.

Second, why do enterprises establish data strategies?

In view of the background described above, the data strategy, like the employee selection and retention strategy, is to manage corporate assets more scientifically. If there is no strategy, the organization will passively respond to the data requirements put forward by each business unit. From the foreground business and the market to the finance, supply chain and human resources of the middle and backstage, demand will be raised to a department, which may involve some kind of data analysis, master data management, business intelligence, data governance, data quality plan and so on.

As a result, the data department has to understand the connotation of business data needs every day, try its best to meet the schedule, and internally need the outdated tools and systems used by operation and maintenance to ensure its normal operation, which is overwhelmed every day. Once there are problems such as data quality and metadata, they will be challenged to pieces, and even upgrade to a high level of ability and trust. More importantly, if these plans are not supported in a timely and correct manner, it is easy to cause enterprises to make wrong business decisions.

Who will formulate and promote the data strategy

Please don't leave enterprise-wide data policies to the Chief Information Officer (CIO) alone. Why? Data is not only an IT asset, but also a kind of enterprise asset, and data strategy is a kind of enterprise strategy to some extent.

DataPipeline believes that CEO and the board need to have a deep understanding of the significance and risks of quickly landing a data strategy, and begin to build the following organizational structure to encourage culture and innovation.

Compared with other corporate roles, CEO should focus on both survival and development, and the data is difficult to achieve immediate results, so the test of balancing short-term benefits and long-term development is the wisdom of CEO. If CEO refers to data when publishing decisions and is familiar with data innovation within the enterprise, then the data strategy is already half successful, otherwise there is a good chance that other people's efforts will be wasted.

Led by CEO, CDO is directly responsible for the detailed path and overall rhythm of the data development strategy within the company's organization, and determines the compliance requirements, the value, speed, process, and the choice of automation and intelligent technology route according to the business model. Attention must be paid to the speed and quality of meeting business needs. Due to the great challenges of data requirements, too many CDO can not quickly achieve the desired results of CEO, board of directors and business departments within a certain period of time and within a certain business scope. Without a good starting point, the work of the chief data officer will lose the pace of progress and get stuck in the quagmire of long-term discussions and saws with business departments on the process of data collection and use, creating a vicious circle and turning this position into a high-risk position. According to DataPipeline observation, many companies are starting to create CDO jobs and try to generate business growth through data, which, objectively speaking, is a situation of both opportunities and challenges like other executive positions.

The data Compliance and Standards Committee, led by CEO and composed of the company's line of business leaders, legal leaders and chief data officers, details the boundaries, degrees of freedom and data quality standards for data use. Responsible for maintaining the highest frequency of discussion updates (usually once a week) as the business develops, while using automated tools to synchronize rules into the data system. If the changes in the business are not consistent at the compliance level, it will gradually become a bottleneck to limit the use of data. The challenge here is not to allow the discussion of the rules to be too large and comprehensive, to reach a consensus within a certain scope as soon as possible, and to gradually promote the rapid landing of some rules, otherwise the landing of the vision will lose the pace of progress.

The data department is headed by the Chief data Officer, including data engineers, analysts and data scientists. Data engineers are responsible for using self-developed or commercial tools that meet the challenges of the times to ensure that business users can use and manage data throughout the lifecycle on their own. At the same time, it is responsible for the automatic and efficient integration and fusion of data sources inside and outside the enterprise, quickly meeting the needs of business fetch and usage, and by ensuring that metadata, master data, and data consanguinity are consistent with the time of business development. let the business accurately understand the semantics of the data.

They not only want to ensure the load balance and stability of big data platform, but also can respond to the calculation and query requirements of the data model at any time. It is also necessary to follow the standards established by the Standards Committee and to ensure data quality and early warning as much as possible through manual rules and algorithms. Last but not least, there needs to be a "pricing system" when dealing with the needs of business units. Because there is a cost to explore the development of data support business, but at present, the business department has no perception of it, let alone accounting for ROI. In the face of cost, it is easy to screen out the real requirements, prioritize, and sort out the ROI in the follow-up service. This road is difficult, but it is imperative, otherwise the business value dilemma of the data department will always exist.

Sometimes the data department is led by CIO without a chief data officer, and there is an art of dividing responsibilities, and the situation is different for each enterprise, but the key responsibility of CDO is to lead data groups in the right enterprise to quickly generate business value with data. CIO has a broader range of responsibilities, but the area of expertise is not at this point.

The business unit should have analysts and scientists with in-depth understanding of the business, and use the tools provided by the data department by themselves. At this time, the threshold for use will be continuously reduced, and the difficulty and cycle of fetching the number will be greatly reduced. The skill requirement is generally at the SQL level. Therefore, business departments need to better understand the data, and conceive that the data can be applied to their own business development perspective, and then through the management of the whole life cycle of data use, constantly summarized in practice. The challenge is how to quickly and efficiently use data to bring business value, through decoupling to get rid of the current situation that development is restricted by the efficiency of the data department.

4. Data fusion strategy in data strategy

Why to formulate a data fusion strategy

Enterprises hope to use data to support data applications and data services in the future, but only if they can extract and use data quickly at any time. In this process, there are some barriers, such as technology, organization, culture and so on. If you do not think about these difficulties in advance, you will encounter a lot of resistance in the subsequent efforts towards the goal, and may even make the project abort.

Which questions need to be answered when formulating data fusion strategy

After figuring out why the data fusion strategy is in place, we will start from the following aspects:

Who: who in the organization will participate in data fusion? Are there IT experts with programming knowledge to solve all data fusion tasks? Or do you need to enable business unit employees to use data fusion tools on their own? Once implemented, what are the potential technical risks and implementation cycle? These issues will have a significant impact on the type of data fusion solution you choose to purchase.

What: what data might need to be integrated? From DataPipeline's point of view, we are opposed to large and comprehensive, and support enterprises to develop very flexible data fusion strategies on demand. Enterprises need to think in reverse from a business perspective and sort out from an overall point of view what business goals they want to be driven by data in the next one to three years or more, what data these data include, and where these data exist to avoid duplication of construction.

If there are only a few data silos, the most cost-effective strategy for an enterprise may be to choose a basic data exchange or ETL tool that meets specific needs. However, if you need to integrate many different islands (or different types of data), it is best to use a more fully functional data fusion platform.

When: when will the data be fused? Once the enterprise has formulated the data strategy, the data fusion strategy will be placed in a very high position. Because, without data fusion, the rest will be difficult, but in terms of time, we have to do a top-level design, and then proceed step by step.

If you are creating a data warehouse, data fusion may occur before the analysis. If you want to create a data lake based on Hadoop or similar technology to store data in the original unchanged form, some data fusion will be done before running the analysis workload. At present, many enterprises have data warehouses and data lakes, the choice of architecture will affect the types of technologies required for data fusion.

Where: where will data fusion take place? Cloud and local are not a contradiction, because there are many companies that have both local IDC (data center), cloud-based or even cloudy architecture. Enterprises need to choose a data fusion tool that can support local deployment, cloud deployment, Docker, and a multi-pronged approach on a platform with large containers. In addition, the degree of modernization and intelligence of the tool should also adapt to the very complex and changeable environment in the enterprise.

How: how to do data fusion? This issue is the most complex because it involves tools, culture, and processes. With regard to employees, as a strategic resource for enterprises and organizations, we do not want to train everyone to be a high-level, experienced and well-trained data engineer. Instead, we should pay attention to cultivating their awareness of data, knowing how to use data and how to give full play to the value of data.

At the same time, enterprises should adopt flexible, easy-to-use and stable data fusion tools. In the face of no matter how powerful the talent is, it will be very difficult to handle it in the work without matching the corresponding tools and methodology. In the future, enterprises need to think more about which tools, platforms and working methods can enable organizations to release data effectiveness as soon as possible.

What considerations need to be made in selecting a data fusion platform

Companies need to combine their own stage of development, the level of information construction and the literacy of personnel to choose their own solutions.

If there are more R & D personnel, the range of options will be relatively wide. In the face of open source and commercial data fusion tools, you need to choose according to the needs of the enterprise and the requirements of production. Open source is a good way to start here, but business tools may be needed for business continuity requirements and some production-level requirements.

In addition, for some technical considerations, you can read this article: when building a real-time data integration platform, please add a link description to the considerations in technology selection.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.