Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to use data catalog to solve the problem of data spread in big data. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Big data becomes a big problem when the database is replicated for different tasks in the enterprise. The data catalog provides a powerful solution.

The enterprise's security analysis team needs a copy of its own production database so that they can find fraudulent accounts. The accounts payable department of the enterprise needs an analytical extract to improve the efficiency of the supply chain. The sales manager of an enterprise needs all its customer records. Enterprise database administrators are using two snapshots and two full backups to ensure that all data is secure.

Data sprawl occurs when data is duplicated unnecessarily

What enterprises are facing is a typical problem of data spread. This happens in Lianjiang when the organization (for whatever reason) creates multiple copies of production data. There is always a good reason to create each copy, but overall, they become a mess.

As business users increasingly want to analyze data themselves in the context of big data, data contagion is becoming a real problem. IDC estimates that as much as 60% of the storage capacity is currently used to replicate data, and the total cost of replicating data storage in 2018 will be as high as $50 billion. However, it is estimated that less than 20% of organizations have replication management standards. DaveRussell, an analyst at Gartner, a research firm, says many companies will retain 30 to 40 pieces of business data.

Data spread causes organizations to be out of sync

In addition to the obvious impact of data sprawl on infrastructure and performance, data integrity becomes a real problem. For example, a salesperson who updates customer records in a customer relationship management (CRM) system may be out of sync with the same records in the customer database. A database administrator who restores an incorrect backup may overwrite production data with old information.

Technology-based solutions that many enterprises are developing for replication contagion are costly, but for many customer organizations, the simplest and most cost-effective approach is good data governance based on data catalogs.

The enterprise data catalog maintains a single directory of all data owned by the company. This can include not only production data, but also backups, excerpts, and abstracts. Production data can be "fingerprinted" with unique signatures so that outdated copies do not inadvertently enter mission-critical applications. Similarly, copies and extracts can be marked according to their intended use. Directories can even improve data integrity by ensuring that data marked with certain meta tags is not overwritten.

Data catalog strengthening data governance strategy is the solution

The use of data catalogs should be combined with good governance practices. For example, employees need to know which data can be used for analytical purposes and which should not be touched, which is a copy or new relevant data. The database administrator needs clear parameters to explain how to restore the backed-up dataset. One way to make data governance both effective and enjoyable is to encourage business users to join the process by marking their own data through crowdsourced data quality programs.

The use of data catalogs reduces the infrastructure loss of data sprawl and reduces the occurrence of isolated data. It can also reduce the burden on database administrators and improve the response speed to business user requests. For example, sales managers who need customer records can use catalogs to find databases that already exist in other departments and to avoid adding the backlog of IT work orders.

Enterprises should not be affected by too much internal data. The solution is not to reject requests for agile viewing processes, but to better understand what data you have in order to be more useful. The right directory can provide management and governance, which is a path to address data sprawl and data-driven companies.

Big data on how to use the data directory to solve the problem of data spread is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.