Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what is the basic function of data warehouse". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

The basic functions of the data warehouse include: 1, ETL design, including data extraction synchronization, data cleaning, data conversion; 2, data layering, generally divided into ODS layer, CM layer, ML layer; 3, preliminary data modeling.

The operating environment of this tutorial: windows7 system, Dell G3 computer.

Data warehouse, English name is Data Warehouse, can be abbreviated as DW or DWH. A data warehouse is a strategic collection that provides all types of data support for the decision-making process at all levels of the enterprise. It is a single data store created for analytical reporting and decision support purposes. Provide guidance for business process improvement, monitoring time, cost, quality and control for enterprises that need business intelligence.

Basic functions of data Warehouse

ETL design: data extraction synchronization, data cleaning, data conversion. It involves relational database (mysql, mariadb, oracle, etc.) and document database (mongodb, elasticsearch, etc.).

Data layering: generally divided into ODS layer, CM layer, ML layer. The ODS layer represents unprocessed data. The CM layer represents the data that cleans the merge layer.

Preliminary modeling of data: corresponding to the hierarchical ML layer of data, the relational model (snowflake model) or star model is generally used to form a wide table to provide data support.

Related technologies: HDFS, HIVE, HBASE, MR, SPARK, YARN and so on.

Data warehouse architecture

The following figure shows the data architecture planned with reference to the data architecture of many companies, for reference only.

This is the end of the content of "what are the basic functions of the data warehouse". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.