Shulou(Shulou.com)06/01 Report--

Recently, while I am not busy, I am planning to build a complete open source project. I will explain the reasons and the entire project framework later. Since it is necessary to build a complete project, data warehouse access is essential. In this article, I mainly introduce my simple thinking and implementation process of the warehouse in this new project (OSS.Core) (the current project is still in the construction stage), mainly focusing on the following aspects:

1. The demand of data warehouse reservoir

2. ORM framework selection

3. Design and implementation of OSS.Core warehouse reservoir.

4. Invocation example

The following implementation section may require you to have a basic understanding of .NET generics, delegates, extensions, expressions, etc. It is precisely because of these language features that it is convenient for us to extract and unify the commonness of operations.

one。 Data warehouse reservoir demand

Since it is a complete project, data access is the most basic part, at the same time, data access is also the most prone to bottlenecks in the whole project. In my division, its role is to be responsible for the input and output of the entire data, not only for a single database (sometimes even multiple databases), but also to complete the implementation of a first-level cache to provide the most basic data support for the logic layer.

The business is always changing, so the project should also have the ability to evolve rapidly, so I hope that the data layer can remain relatively simple, minimize complex coupled queries in structure, and minimize unnecessary consumption in performance, such as the heavy use of reflection. At the same time, the basic CRUD unified encapsulation implementation at the database level is completed for each business object. Cache updates can be added with minimal changes if necessary. (how to implement different cache storage strategies for different modules, such as Redis,Memcached, will be described in a later article)

At the same time, for a slightly larger project, the fastest way to solve database access is to achieve read-write separation, so I hope this framework can implement read-write separation support at the very beginning. To avoid a lot of changes to the business code at a later stage.

two。 ORM framework selection

Of course, for the sake of simplicity and performance, direct ADO.NET connection is theoretically more efficient, but it will result in a lot of repeated operation of logic code, but also cause code clutter and increase maintenance complexity. As a technician, I not only need to solve business problems to improve efficiency, but also need to improve my own efficiency, so I will choose an ORM framework to complete part of the basic work.

At present, under the .NET system, there are many open source ORM frameworks, such as Entityframework,NHibernate,iBATIS.NET,Dapper, etc., each with its own characteristics, based on what I said earlier, ensuring efficiency while taking into account simplicity can minimize the loss of performance, and provide support under the .net standard standard library. After the comparison here, I choose Dapper, a semi-automated ORM, as the basic framework of the warehouse. The reasons are as follows:

1. Its structure is simple, the whole package is mainly concentrated in the Dapper.cs file, and the volume is very small.

two。 The encapsulation function is simple and powerful, and the support for native SQL is very flexible.

This is almost better than other frameworks, without any extra settings, while basically you can call all the native ADO.NET functions, sql statements are completely under your own control, but do not care about command parameter assignments, and the resulting entity conversion and so on.

3. Efficiency in performance

A lot of ORM entity mapping is done through reflection, and Dapper once again shows its charm. In the key modules such as Commond parameter assignment and entity conversion, the Reflection.Emit function is used to indirectly realize the assignment at the MSIL compilation level. Indirectly, it is because its own code also needs the compiler to generate IL code. Dynamically create an assignment delegate method based on type properties at run time.

three。 Design and implementation of OSS.Core warehouse reservoir

Through Dapper, we can achieve a simple encapsulation in the database access part, but I still need to write a lot of sql statements manually and carry out parameterized processing, including data read-write separation and so on. Then I will complete the implementation of these functions in OSS.Core.RepDapper. In order to facilitate understanding, I will first post a simple encapsulated method call transfer process:

A simple method invocation process is shown in this diagram. Around the core parts of this diagram, I will introduce:

1. Interface design

Because I want this to be a complete example project, I want to be compatible with different databases later, so external warehouse access is based on interface calls. Of course, if your project has no need to switch databases at all, I suggest removing this link and implementing the singleton pattern directly in the base class and calling it directly in the business logic layer.

In the figure, we can see that the interface layer is independent of the implementation part. I put the specific business entity model and interface in the OSS.Core.DomainMos class library separately, on the one hand, to share the entity model in each module, and on the other hand, to decouple the dependency between the business logic layer (Services) and the warehouse layer (Reps).

At the same time, most of the database access codes in a project will be based on CRUD, so here I define a basic interface (IBaseRep), which mainly includes the following methods (the expression section is described later):

The specific business data interface can be inherited from the basic interface, in which the expression part is encapsulated by myself, which will be briefly introduced later.

two。 Warehouse base class implementation (BaseRep)

First of all, as shown in the figure, we have implemented two extensions of read-write separation, both of which will eventually go through the Excute method, so here is the specific implementation of the method:

You can see that this method provides a delegate for IDbConnection, provides the calling layer to use the Dapper method freely, and unifies the entry of the data access method to facilitate logging and troubleshooting.

Secondly, in many projects, users and orders are in different libraries. Because sublibraries are involved, the ability to modify connection strings in subclasses is required. Here, in the form of constructors, two nullable parameters are provided:

As you can see, if the subclass defines its own connection string, it is mainly defined by the subclass, otherwise the default connection information is taken.

Finally, we also implement the specific implementation of the basic interface method, as an example:

At the same time, in order to ensure that cache processing can be added to the subclass, the form of virtual method (virtual) is adopted to ensure that the subclass can be rewritten.

3. Connection-based extension

This place is mainly divided into two parts, a. Parsing of expressions and parameterized processing of b. Extend the Insert,Update... of Connection There is no way to extend Dapper:

a. Friends who are familiar with Expression expressions should know that the expression itself is a tree interface, and its subexpressions can be parsed continuously according to different types, until it is not possible to continue parsing. So this is simply recursive iteration, according to its different NodeType can assemble different sql elements, because the code is long, you can see the SqlExpressionVisitor.cs class under github, in which the parameter assignment part, does not use reflection, but uses reflection emission, the code is detailed in SqlParameterEmit.cs

b. After you have extended the expression, you can get the corresponding sql and parameters. You can extend the Connection method through this. For more information, please see ConnoctionExtention.cs.

four。 Invocation example

1. We define a simple UserInfoMo entity (including attributes such as mobile)

two。 Define the interface IUserInfoRep: IBaseRep

3. Define implementation classes UserInfoRep: BaseRep, IUserInfoRep

Without adding other code, we can complete the following call:

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.