Shulou(Shulou.com)06/01 Report--

Today, what the editor shares with you is the introduction of the logical architecture design of the data warehouse. I believe many people do not know much about it. In order to let you know more about the logical architecture design of the data warehouse, I have summarized the following contents for you. Let's move on. I'm sure you'll get something.

Offline data warehouses are usually built based on dimensional modeling theory, and offline data warehouses are usually logically layered, mainly for the following considerations:

1. Isolation: users should use the data carefully processed by the data team, rather than the original data from the business system. The first advantage is that users use carefully prepared, standardized, clean data from a business perspective. Very easy to understand and use. Second, if the upstream business system is changed or even reconstructed (such as table structure, fields, business meaning, etc.), the data team will be responsible for dealing with all these changes to minimize the impact on downstream users.

2. Performance and maintainability: professional people do professional things, and data layering makes the data processing basically in the data team, so that the same business logic does not have to be executed repeatedly, saving the corresponding storage and computing overhead. In addition, data layering also makes the maintenance of the data warehouse clear and convenient, each layer is only responsible for its own tasks, a layer of data processing problems, only need to modify this layer.

3, normative: for a company and organization, the caliber of data is very important, when we talk about an index, it must be based on a clear and recognized I caliber, this appearance, fields and indicators must be standardized.

4. ODS layer: the data table of the source system of the data warehouse is usually stored intact, which is called the ODS (Operation Data Store) layer, and the ODS layer is often called the preparation area (Staging area). They are the source of the processing data of the subsequent data warehouse layer (that is, the fact table and dimension surface generated by Kimball dimension modeling, and the summary layer data processed based on these fact tables and parts list). At the same time, the ODS layer also stores historical incremental data or full data.

5. DWD and DWS layer: data warehouse detail layer (Data Warehouse Detail, DWD) and data warehouse summary layer (Data Warehouse Summary, DWS) are the subject content of data warehouse. The data of DWD and DWS layer is generated by ETL cleaning, transformation and loading of ODS layer, and they are usually built based on Kimball's dimension modeling theory, and ensure the dimensional consistency of each sub-topic through consistent dimension and data bus.

6. Application layer (ADS): the application layer is mainly the data Mart (Data Mart,DM) established by each business or department based on DWD and DWS, and the data Mart DM is relative to the data warehouse (Data Warehouse, DW) of DWD and DWS. Generally speaking, the data in the application layer comes from the DW layer, but in principle, direct access to the ODS layer is not allowed. In addition, compared with the DW layer, the application layer only contains the detail layer and summary layer data that the department or the party is concerned about.

These are the details of the logical architecture design of the data warehouse. Is there anything to gain after reading it? If you want to know more about it, welcome to the industry information!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.