Shulou(Shulou.com)06/03 Report--

Data platform is not imported, but a concept created by Chinese Internet giants.

In fact, foreign users rarely mention it when all walks of life in China set off a so-called data fever. However, this does not mean that foreign markets do not value data. On the contrary, in highly digitized markets such as Europe and the United States, the DataOps concept has taken root. DataOps is not just a product, service or solution, but a new enterprise data management model in the AI era, aiming to quickly discover and use the value contained in all data in the entire user architecture.

Therefore, whether it is a data center or DataOps, there is quite a "different approach", just a different name. For traditional industry users, it is biased to put their own development into the environment of Internet companies. On the contrary, the views of some manufacturers with rich experience in traditional industry data value practice are worth learning and learning from.

Recently, Hitachi Vantara, which has deep accumulation, rich practical experience and perfect products and solutions in the field of DataOps, analyzed the hot topics such as the current domestic data boom and data analysis. Hitachi Vantara's views on data center are worth further consideration by domestic users.

Three typical challenges of data analysis Compared with Internet companies that are inherently digital, most enterprises in traditional industries have a steady evolution process of digitalization from scratch. This determines that the traditional enterprise data value road can not be achieved overnight. From the past data warehouse, data management, to the current big data platform, data platform, traditional enterprise data analysis and value mining has gone through several stages.

According to Yu Xiguo, director of Hitachi Vantara's data and analysis business in China, the traditional BI/DW analysis era in the 1990s was mainly to solve the processing requirements of structured data in a batch and centralized manner. With the rise of mobile Internet and social media after 2005, a large number of multi-source and heterogeneous data began to be generated. From 2010 to the stage of business big data, traditional centralized data analysis methods were not competent, followed by the emergence of distributed processing such as Hadoop, and a number of data analysis applications began to appear in the Internet, finance and other industries. Since 2016, data analysis has really entered the stage of ML/AI big data analysis. Sensing, real-time, professional and closed-loop are its remarkable characteristics, and they have begun to affect all aspects of society.

Yu Xiguo, Director of Hitachi Vantara's Data and Analytics Business in China

Because of this, traditional industries may face more complex situations on the road of data analysis and value mining. Traditional enterprises may face more complex and diverse data environments, technology environments, and demand characteristics than Internet companies. According to Yu Xiguo, many enterprises currently face three very obvious challenges in data analysis:

First of all, many users 'environments have the coexistence of traditional digital warehouse products and open source products, especially the popularity of open source. A large number of open source products are integrated into the user environment, but it is difficult for these different software products to work together, and it is easy to form isolated islands, which brings huge hidden dangers for subsequent data analysis.

Secondly, for traditional industry users, under the injection of multi-source, heterogeneous and massive data, it is difficult for different software to communicate with each other, and business invocation and performance improvement are very typical challenges.

Third, most of the traditional digital warehouse products that exist for many users are closed systems, and it is difficult to embed new technologies and products into applications, such as popular technologies and applications such as machine learning.

Because of this, the value of the data center can be highlighted. Yu Xiguo said that Hitachi Vantara called DataOps for data center platform, that is, establishing a good connection between data consumers and data managers, so that the foreground application service and background data play a good role in connection and support, and make data analysis more automated, civilian and business.

In fact, the role of the data center is to integrate data collection, fusion, governance, organization management, and intelligent analysis into one, and provide data to the business foreground in an efficient way to improve business operation efficiency and continuously promote business innovation. However, the construction of data center is often complex, which needs to be comprehensively considered according to the industry attributes and actual business conditions of industry users themselves. Therefore, data mid-office products and solutions need to be not only complex enough, but robust enough to scale and flexible enough to support the full range of possible future business resilience.

In Xi'an's view, the construction of data center needs a link, which can integrate various software and solutions of traditional enterprises in the past, just like a "data pipeline," which can realize real efficient flow and intelligent analysis of data. Hitachi Vantara Pentaho plays this versatile, intelligent, efficient and easy-to-use "data pipeline" role, which plays a key role in greatly improving data analysis efficiency and mining data value.

In fact, Pentaho serves as a platform that encompasses all data business processes from data access, integration, and exchange, to visual development, optimization, to data marts, embedded analytics, and machine learning. "It makes data analytics really a closed loop. "Yu Xiguo said. Pentaho can realize the rapid import of various data sources into the data lake, and also has a platform for data integration, data exchange and data migration. At the same time, it can be combined with the development of full-process visualization and big data computing capabilities. It can also provide various data analysis platforms for data scientists through self-service data marts, and support for artificial intelligence such as machine learning is also very comprehensive. Pentaho not only supports machine learning, deep learning, but also reinforcement learning. "

Pentaho, as an open source software product, has been continuously enhanced and upgraded at the functional level since it was acquired by Hitachi Vantara. Take Pentaho 8.3, the latest version on the market, for example, which adds a number of new features. For example, the upgraded data pipeline drag function and data visualization ability make data access, fusion and governance more convenient and fast; for example, under the trend of multi-cloud, this version has been optimized and improved for multi-cloud services: users can access data in multiple cloud platforms through Pentaho 8.3, including AWS, Google Cloud, Microsoft Azure, etc. "Pentaho has good embedded capabilities. If you lack a driver, you only need to package the driver to support cloud operation. "Yu Xiguo revealed.

Pentaho 9.0, which is scheduled to be available in February, will help users further improve the efficiency of data integration and analytics, providing new data flow services for sharing, managing and monitoring activities in the data pipeline, reducing congestion in the data pipeline, and providing advanced data discovery, access and integration capabilities from the edge to the cloud.

At present, Pentaho has precipitated ten typical application scenarios after rich practice: unified data exchange platform, host downshift and data warehouse migration, data cleaning conversion performance improvement, Internet crawler and machine learning, intelligent operation and maintenance of data center, IoT industrial data acquisition and analysis, data visualization and self-service analysis and reporting, unstructured data conversion and analysis, data desensitization and test development environment construction, data multidimensional model construction and analysis.

Data analysis platform needs to be open and ecological "Recently, we have a customer testing ETL data extraction tools. He has bought more than ten ETL tools. Among the 24 scenarios summarized by the customer, none of these ten tools can meet all of them. It was not until Pentaho entered that all 24 scenarios were met. "While this example demonstrates the power of Pentaho's products, it also reflects the diversity, complexity and rapid change of user needs.

Therefore, Dai Jianping, Hitachi Vantara's global vice president and general manager of China, believes that data analysis platforms often need to be more open."The limitations of closed platforms are too obvious. Many technologies are emerging very quickly, and data analytics platforms need to be more open and capable of integrating these technologies. Pentaho's advantages are, first, open source and good compatibility and openness, and can migrate with various manufacturers in the industry, including local manufacturers. Secondly, Pentaho is a seamless end-to-end open tool platform with efficient and close integration of multiple data source types, which can not only connect different islands and different data links into a whole, but also integrate into customers 'complex and diverse IT environments. "

Jianping Dai, Global Vice President and General Manager of Hitachi Vantara China

"We have seen that kind of customer, initially based on the early version of Pentaho, closed after not open source, only rely on their own patches, gradually out of touch with the times, and then can not keep up with the development trend. This customer's version is ten times less powerful than Pentaho's current version. "Yu Xiguo revealed.

In fact, Pentaho is the most popular open source business intelligence software in the world, and its openness and open source are its biggest advantages. After being acquired by Hitachi Vantara, Pentaho retained its open and open source edge. At present, Pentaho Community Edition has more than 7.5 million downloads worldwide, and China ranks first in the world in downloads. Pentaho has a large number of enterprise customers worldwide, with more than 2500 commercial users and more than 20000 product deployments.

In addition, Hitachi Vantara also emphasizes the concept of ecological Co-Creation in the Chinese market, and has set up special teams in the financial, education, medical, manufacturing and other industries to innovate together with partners and customers. "Every industry has precise requirements and forward-looking needs, which are often not what a company can do. Hitachi Vantara wants to work with partners and customers to meet their real needs. "Dai Jianping added.

In the future, Hitachi Vantara will focus on expanding Pentaho's market and ecosystem in two ways. First, for users of Pentaho Kettle open source version, continue to dig deep into the needs of cooperation; second, strengthen the contact with partners, including ISVs and other partners, jointly explore how Pentaho provides better platform tools and promotion methods in the industry, and provide customers with richer choices in the way of product integration.

"One of the largest e-commerce platforms in China, which is often mentioned in the data center, is Pentaho's largest download customer in the world. "Dai Jianping, Hitachi Vantara's global vice president and general manager for China, is confident about Pentaho's future.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.