Shulou(Shulou.com)05/31 Report--

This article mainly analyzes the relevant knowledge of how to understand Newbe.Claptrap, the content is detailed and easy to understand, the operation details are reasonable, and has a certain reference value. If you are interested, you might as well follow the editor to have a look, and follow the editor to learn more about "how to understand Newbe.Claptrap".

The wheel comes from demand #

With the vigorous development of Internet applications, related technical theories and implementation means are also constantly being created. A series of keywords such as "cloud native architecture", "micro service architecture", "DevOps" and so on appear more and more in the vision of engineers. To sum up, the emergence of these new theories and technologies is to solve some technical pain points in Internet applications:

Higher capacity scalability requirements. On the basis of commercial success, the number of users of Internet applications, system pressure and the number of hardware devices will increase significantly with the passage of time. This puts forward the requirements for the capacity scalability of the application province. This capacity scalability is often described as "applications need to support horizontal scaling".

Higher system stability requirements. The application can run continuously to ensure the continuous progress of business activities, which is what anyone associated with the application system would like to see. But it is usually very difficult to do so. Today's Internet applications in the face of many similar competitors, if not sound enough in this respect, then it is likely to lose the favor of some users.

Higher functional scalability requirements. "embracing change" is a word that people refer to when they mention something related to "agile project management". This word fully reflects how important it is for today's Internet applications to be successful in terms of functionality. It also reflects the changeable product demand under the current Internet environment from one aspect. As a system engineer, this should be considered at the beginning of application establishment.

Higher development ease of use requirements. The development ease of use here refers to the degree of difficulty in the development of the application system itself. The easier it is to develop, you need to make corresponding efforts in applying your own code structure, testability, and deployability.

Higher performance requirements. The performance requirements mentioned here refer specifically to the performance requirements when the system capacity increases. Avoid the single point performance problem of the system, so that the application system has the characteristics of horizontal scalability. Generally speaking, when there is a performance problem, it is usually the easiest way to solve the problem by adding physical equipment. Under different system capacity, the optimization scheme of system performance is usually different. Therefore, the selection of technical solutions combined with application scenarios has always been a problem that system engineers need to consider.

The Newbe.Claptrap project is a set of development framework summed up based on the above system functional requirements. This includes the relevant theoretical cornerstone, development class library and technical specification.

There is no silver bullet in the world. One framework can't solve all the problems. -- the moon set on condition of anonymity.

# starting from the demand #

When explaining distributed systems, the simple business scenario of "account transfer" is often used to describe it. Here is a description of this business scenario.

Suppose we need to build a business system with an account system. Every account has a balance. Now you need to perform a transfer operation to transfer 300 of the balance of account A to account B. In addition, based on the basic requirements of the previous section, we need to consider the following when implementing this scenario:

Need to cope with the surge in system capacity. There may be only 1000 initial users at the beginning of the application. Due to the good effect of application promotion and the influx of robot accounts, the number of users has risen by three orders of magnitude in one month, that is, to the level of millions.

The stability and recoverability of the system need to be considered. Minimize the average failure time of the system as a whole, and it should be as easy to recover as possible even if a system failure occurs. That is, to avoid a single point of failure.

The scalability of the business needs to be considered. Some business logic may need to be added later: limit the amount of daily transfer according to the level of the account, notify by SMS after the transfer is successful, support a certain amount of secret-free transfer, and achieve "Tunable 1" transfer for specific accounts.

You need to consider the testability of the code. The business code and system code of the system can be well separated, and the correctness and performance of the business code and system code can be preliminarily verified by means of unit testing.

# the theory of the wheel #

This section will introduce some theoretical contents that are closely integrated with this framework, so that readers can understand the working process of this framework in the follow-up process.

Actor mode #

Actor pattern is a concurrent programming model. Through the application of this programming model, some system concurrency problems can be well solved. The concurrency problem mentioned here refers to the problem that the data may be incorrect due to multiple simultaneous requests when the computer logically processes the same data. This problem is a problem that must be encountered in multithreaded programming. To take a simple example, suppose you use 100 threads to perform + + operations on an int variable in memory concurrently without a synchronization lock. In the end, the result of this variable is often less than 100. Here is how the Actor mode avoids this problem.

First of all, for ease of understanding, the reader can think of Actor as an object here. In object-oriented languages (Java, C #, etc.), Actor can be thought of as an object created by new keywords. However, this object has some special features:

Have a state of your own. Objects can have their own properties, which is a basic function of object-oriented languages. In Actor mode, these attributes are collectively referred to as Actor states (State). The state of the Actor is maintained by the Actor itself.

This emphasizes two points:

First, the state of Actor can only be changed by itself. If you want to change the state of Actor from the outside, you can only change it by calling Actor.

Second, the state of the Actor is maintained only within the Actor and is not shared with anything other than the current Actor. The term "non-sharing" here also emphasizes that it cannot change the internal state of Actor through the change of an external attribute. This is mainly to distinguish it from some programming languages that have the "object reference" language feature. For example: in the public property of the class of C #, if it is a reference type, then you can change the property in the class after getting the class externally. But this is not allowed in Actor mode.

Single thread. Actor usually accepts only one call at a time. The thread described here does not exactly refer to the thread in the computer, but is used to highlight the "feature that Actor can only handle one request at a time". If the current Actor is accepting a call, the remaining calls are blocked and the next request is not allowed to enter until the call ends. This is actually similar to a synchronous lock mechanism. Through this mechanism, the possibility of concurrency problems can be avoided when the internal state of Actor is modified. To be specific: if you use 100 threads to make concurrent calls to an Actor, let Actor do + + operations on an int variable in the state. The final value of this state must be 100.

However, single threading is not absolute, and concurrent processing is allowed in cases where there is no concurrency problem. For example, read the state in Actor, which usually does not have concurrency problems, so concurrency operations are allowed at this time.

When reading about the single-threaded feature of Actor, readers usually wonder whether this will cause performance problems due to the slow processing of Actor itself. With regard to this point, I hope that readers will continue to hold this question and read later to find the answer.

Event traceability mode #

Event traceability mode is a kind of software design idea. This design idea is usually different from the traditional system design idea based on addition, deletion, search and modification (CRUD). CRUD applications usually have some limitations:

Generally speaking, CRUD applications will directly manipulate the data storage. Such an implementation may lead to performance bottlenecks due to insufficient database optimization, and it will be more difficult to achieve application scaling.

In a specific area, there is usually some data that needs to pay attention to deal with the concurrency problem in order to prevent errors in data update. This usually requires the introduction of "locks", "transactions" and other related technologies to avoid such problems. However, this may lead to a loss of performance.

Unless additional audit techniques are added, the change history of the data is generally untraceable. Because the final state of the data is usually saved in the data store.

In contrast to the CRUD approach, event traceability is designed to avoid the limitations of the above description. Next, around the "money transfer" business scenario mentioned above, we briefly describe the basic work of event traceability.

The method of CRUD is used to realize "transfer".

Use the way of tracing the source of the event to realize the "transfer".

As shown in the above figure, the balance changes involved in the transfer business are stored in the way of events through the event traceability mode. It also implements the business itself, which brings some benefits:

Through the event, you can restore the balance of the account at any stage, which realizes the tracking of the balance of the account to a certain extent.

Because the events of the two accounts are handled independently. Therefore, the processing speed of the two accounts will not affect each other. For example, the transfer of account B may be slightly delayed due to the need for additional processing, but account A can still be transferred out.

You can do some business asynchronous processing by subscribing to events. For example: update the statistics in the database, send SMS notifications and other asynchronous operations.

Of course, some technical problems related to event traceability are introduced after the introduction of event traceability mode. For example, events may consume a lot of storage, have to apply final consistency, events are immutable, and may be difficult to reconstruct. These related issues will be explained in more detail in some articles. Readers can read the following extended reading content, and then understand and evaluate it.

Business complexity is not reduced by changes in system design, it just shifts from one place to another. Always talk about the moonset of your own food

# turn the wheel #

Based on the reader's general understanding of the theory of the previous section, this section will introduce the working principle of this framework combined with the "transfer" business scenario described above. First of all, the reader needs to know the two nouns of this framework.

Claptrap#

Claptrap is a special Actor defined by this framework. In addition to the two Actor features mentioned above, Claptrap is defined as having the following features:

The state is controlled by the event. The state of Actor is maintained within Actor. The same is true of Claptrap, but changing the state of a Claptrap is limited not only to changes within Actor, but also to events. This combines the event traceability pattern with the Actor pattern. The correctness and traceability of the Actor state are ensured by the event traceability mode. These events that change the state of the Claptrap are generated by the Claptrap itself. Events can be caused by external calls or by trigger-like mechanisms within Claptrap.

Minion#

Minion is a special Actor defined by this framework. It is an adjustment made on the basis of Claptrap. It has the following characteristics:

Read events from the corresponding Claptrap. Like Claptrap, the state of Minion is controlled by events. The difference is that Minion, like its literal meaning, always gets events from the corresponding Claptrap, thus changing its state. Therefore, it can asynchronously handle subsequent operations after Claptrap generates events.

Business implementation #

Next, with the previous basic introduction, let's take a look at how the Newbe.Claptrap framework implements the "transfer" scenario above.

In addition, there are some points to be pointed out:

In the figure, the call waiting for Client and Claptrap exists only in the first phase, that is, this allows Client to get a response more quickly without having to wait for the entire process to end.

After Claptrap A processes its own request and sends the event to Minion A, it can re-accept the request, thus improving the throughput of Claptrap A.

Minion does more than just handle calling proxies between Claptrap. In Minion, you can also do other operations according to business needs, such as sending text messages, updating database statistics, and so on.

Minion can also have its own state, keeping part of the data in its own state so that the outside world can query from itself, rather than from the corresponding Claptrap. For example: count the transfer changes of the account in the last 24 hours for quick inquiry.

Business capacity #

What the Newbe.Claptrap framework needs to build is a system architecture that can scale horizontally, only in this way can it cope with the continuous growth of business capacity. At this point, this framework uses the open source Dapr to implement the scaling of applications and physical devices at this stage.

Of course, when it comes to data storage, it is bound to involve a series of problems such as database cluster. These belong to the details of the technical application, not the content of the framework theory design. Therefore, it is only shown here that this framework can be scaled down based on the above open source architecture.

This is the end of the introduction on "how to understand Newbe.Claptrap". More related content can be searched for previous articles, hoping to help you answer questions and questions, please support the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.