Shulou(Shulou.com)06/02 Report--

The first part: the birth, evolution and application strategy of micro-service.

Reporter: in recent years, micro-service architecture design has been proposed and put into practice in more and more enterprises, but for people who are just beginning to come into contact with micro-services, they still do not know from which aspects to start to understand. Can you talk about the development and characteristics of micro-services according to the development history of software architecture?

Liang Xin: micro-service is essentially a style of architecture. If you want to understand micro-service, I think you need to understand the development of the whole architecture first.

Software architecture is always in the process of evolution. If we go back to 20 years ago, the enterprise-level R & D mainly advocated the C _ pact S mode, and the development software such as PB and Delphi were the mainstream of enterprise application development.

Over time, we found that the standardized client has some disadvantages, such as I have a thousand terminals, the upgrade version requires each terminal to upgrade, which is very troublesome. Then, enterprise application R & D begins to learn from the Internet and use the browser as a client to avoid this problem. As a result, the browser-based Bamp S architecture is becoming more and more popular.

It started with ASP, and then there was JSP, because the performance of Ja.va was greatly improved because of the precompiled mode of Ja.va, and then the J2EE architecture based on Ja.va language became more and more popular. At this point, the architecture has undergone a transformation from the traditional Cpact S mode to the BCMG S mode.

In the initial stage, the BPX S architecture is basically a single architecture, each system is relatively independent, they often do not need to interact with each other, even if there are some interactions, most of them are at the data level. At that stage, ETL tools developed rapidly to solve the problem of isolated data islands.

With more and more enterprise applications, the relationship between systems is getting closer and closer, and there are more and more requirements for real-time interactive access between systems. At this time, engineers found that no matter what language the software developed basically supports a language called XML, so they put forward a technical solution for real-time interaction: remote calls between enterprise application systems through the XML language. As a result, the concept of SOA was put forward, and WebService became popular.

When the second wave of the Internet comes, in order to adapt to the more flexible business development, many companies replace the original bulky WebService with the architecture style based on HTTP protocol and Restful, and the concise and clear JSON instead of XML. At the same time, service bus technology is often used in SOA architecture, which undoubtedly adds an extremely troublesome bottleneck to the system architecture. If we use the mechanism of registration and discovery to make direct calls between service processes, it is more suitable for the development of enterprise applications. This is the history of micro-service architecture from a technical point of view.

There are two basic principles to define micro-service in "micro-service design": loose coupling & high cohesion. That is, "gather things that change due to the same factors together, and separate things that change due to different factors." As for the size of micro-services, there is no unified standard, generally speaking, you think it is about the same size, while in line with the "loose coupling-high cohesion" principle.

Microservices have many benefits, some of which are listed in general.

Heterogeneity: microservices can help us easily adopt different technologies and understand the benefits of these new technologies. Trying a new technology usually comes with risk, but if you cut the service very small, there are always points that allow you to choose a service with the least risk to adopt the new technology and reduce the risk. Resilience: obviously, microservices can handle service unavailability and functional degradation very well, because it can form many nodes. Isolation: the micro-service architecture decomposes the system into independent operating units, which brings better isolation to the system. Independent micro-services are easier to locate and isolate problems when exceptions occur, and isolation is also the basis of service scalability. Expansion: huge single services can only be extended as a whole, even if only a small number of modules in the system have performance problems, the whole system needs to be extended. The micro-service architecture can scale different modules horizontally according to the performance needs. Deployment is simple: in a micro-service architecture, the deployment of each service is independent, so that specific parts of the code can be deployed more quickly. Problems with services are also easier to roll back quickly, while agile delivery and deployment lead to a better business requirements response experience. Flexibility: in the micro-service architecture, the system opens up many interfaces for external use, and when the situation changes, applications can be built in different ways. The purpose of reusable and composable can be achieved by decomposing monomer applications into multiple micro-services.

Reporter: it is reported that you published an article "Why does the company need to establish a unified development framework?" What problems do you think the company is trying to solve by establishing a unified development framework?

Liang Xin: this is a question of benevolence and wisdom. Everyone's starting point is different. Some people may advocate unity, while others may reject unity. Combined with my experience and practice, I think the company needs to establish a unified development framework.

In the past decade, the development of the Internet has subverted many traditional industries, and many new companies have sprung up like bamboo shoots after a spring rain. Their business is growing very fast, and their companies are getting bigger and bigger. This is due to the rapid growth of China's economy and the rapid development of the Internet. However, this rapid development process is accompanied by disadvantages that can not be ignored:

Drawback one: self-reproduction

In the process of the rapid development of the company, such a chain often appears. Add a new business-> recruit a senior technical staff-> build a technical team around this colleague-> this team is basically in charge of the business, and then a closed loop is formed. When it comes to interacting with other businesses, it is often the technical leaders who make their own decisions. This forms a state of self-reproduction.

The second drawback is to control barriers.

With the rapid development of the business scale, the team quickly formed a department, and the team decision makers usually consider their own interests and want to reduce the dependence of the external department as much as possible. whether it is technology selection, specification establishment, component selection, operating environment can be self-controlled.

Drawback 3: cliff effect

Once such a technical atmosphere is formed, the impact of individual employees on a single project will become very great. A product is often difficult to sustain because of the departure of one or two core employees, and finally has to redevelop a new product.

Drawback 4: waste of resources

When each team is trying to build its own complete R & D process. In the middle of technical research, product development, operation and maintenance management, there will be a lot of waste of resources.

Drawback 5: it is difficult to assess

How to measure which is better, a Sichuan chef or a Shandong chef? When each team is a closed loop, using different technology stacks, different technology components, different maintenance methods and norms, it is impossible to judge the performance of a team from the output efficiency, and it is very difficult to set up KPI indicators.

The establishment of a set of company-level unified development platform can effectively solve the above problems. From a technical point of view, if a company-level unified development platform can be formed, it will bring great benefits in the actual production process.

First of all, the repetitive technical research is avoided and the labor cost is saved. Build a basic development architecture platform under the project team, extract the technical common problems, give it to a special team to deal with, and let the project team devote its energy to the business. Secondly, the technical specifications are standardized and the quality of the product project is improved. It takes a thousand people to do a project, not a thousand people. After adopting a unified development platform, a standardized technology output mode can be formed in the technology stack, technology components, technology implementation scheme, and even in the code specification, and the effect of standardization is not only the rapid improvement of development efficiency, but also the substantial improvement of product quality. Thirdly, technical precipitation can be carried out to enhance the overall technical capability of the company and avoid falling into the ability of a person to determine a project. The progress of technology comes from the continuous accumulation and precipitation of technology, and the establishment of a unified development framework (platform) at the company level. The project team carries out its own project research and development based on this platform, and no longer needs to focus on the underlying technology implementation. You just need to focus on the business. Moreover, colleagues who focus on the platform make continuous improvements to the platform in order to better meet the technical needs of the project team, so as to achieve the goal of technology accumulation and precipitation. Finally, the input and output of R & D can be measured, and the R & D team can be managed and assessed effectively. When the standardized technical specifications based on the same development platform are established, the code implementation of business functions can be evaluated and considered relatively effectively, and all kinds of problems caused by technical implementation differences can be avoided. This is a great help to the formulation and assessment of KPI.

I put forward such an idea from the year before last. Through more than a year of efforts, I have achieved certain results in the company. Our unified development platform is located at the technical level, and its main purpose is to unify the technical framework and development tools used in the company's related product research and development and project implementation, effectively improve unified technical support, and form a continuous means of technology accumulation. improve the utilization rate of technicians and reduce dependence on personnel, and ultimately enhance the large-scale and pipelined production capacity of the software.

Reporter: recently, words such as "Spring Boot" and "Spring Cloud" are always mentioned. What are the advantages of these new framework collection solutions compared with the traditional micro-service framework? Based on your experience, what do you think is the possible future development trend of microservices?

Liang Xin: I am the person who studied the Spring Cloud technology stack earlier in the company, and I am also a member of the Spring Cloud Chinese community. Spring Cloud became the most popular micro-service development framework in 2017. However, there is a problem that needs to be viewed dialectically. "it is not that using the Spring Cloud framework to achieve the micro-service architecture, with the advantages of the micro-service architecture", the correct understanding should be "using the Spring Cloud framework to develop the micro-service architecture system, so that the system has the advantages of the micro-service architecture."

The reason why Spring Cloud stands out from other frameworks to become the most popular framework is due to the integrity of its own system. This can be seen intuitively through the comparison of Spring Cloud, Dubbo, and ServiceComb in the following figure.

In addition, Spring has a broad mass base in China, and I also admire the R & D idea that "agreement is greater than configuration" and does not need to rely entirely on standardized things.

I dare not talk about the future of the micro-service architecture. Based on the current situation, I think the current Spring Cloud+Docker containerization technology is a good choice for micro-service architecture. I agree with an interesting term "genetic architecture", which means that architecture is meant to change from the beginning, so the easier it is to change your architecture, the better. I think the future of the structure will develop in this way.

The construction of our unified development platform is based on Spring Cloud technology stack.

Reporter: the hot topic in the software research and development industry this year is "Zhongtai". At the architecture level, some people have also proposed to do micro-service middleman. what do you think of this?

Liang Xin: a variety show last year had a popular saying, "disk it". There is a sentence in the program, "dry, dodgy, not round at all, dish him!" . Later, whatever you talk about, you can hold it in your hand, no matter what it is. Anyway, the disk is. Does it sound particularly magical, and then there is the joke that "everything can be dished"? In fact, this is only a kind of ridicule, and it will not really be what anyone sees. The interesting thing is that you can think deeply about anything. Will you also make some seemingly ridiculous mistakes?

One of the most popular terms in the technology circle this year is "Zhongtai". When applied here, it becomes "everything can be Zhongtai". A noun is everywhere. I think many companies should avoid blindly following the trend and let "Zhongtai" become a noun trap.

In the face of a new technology or trend, we must first understand its origin and roots. The source of Zhongtai needs to be traced back to Ali. In 2015, Alibaba Group launched the China Taiwan strategy, with the goal of building an innovative and flexible mechanism of "large, medium and small foreground" in line with the era of big data on the Internet. in other words, as the front-line business of the foreground will be more agile and faster to apply to the rapidly changing market, while Zhongtai will integrate the operational data capacity and product technical capabilities of the whole group to form a strong support for each foreground business.

Then why does Ali Group want to build a structure of "large and medium-sized platform and small receptionist"? The book "the way of Enterprise IT Architecture Transformation-- Alibaba's Strategic thinking and Architecture practice between China and Taiwan" is introduced in detail. Starting from the development history of Ali's shared business division, Ali had only one Taobao business department at first, and then set up Tmall's business department, at which time the technical team of Taobao supported both divisions. At that time, the e-commerce systems of Taobao and Tmall, like those of many of our large enterprises, were divided into two sets of independent chimney systems, both of which included commodity, transaction, payment, evaluation, logistics and other functions. For the above reasons, Ali Group also set up the shared Business Division, whose members are mainly from the previous Taobao technical team, and combed and precipitated the two sets of e-commerce business at the same time. Precipitate the common and common business functions of the two platforms to the shared business department to avoid repeated construction and maintenance.

As a veteran with more than 10 years of programming experience, one of the problems I often think about is the law of system development. Through its shape and meaning, and reviewing the development of the architecture, I think one point can be summed up: "fast". Of course, there are prerequisites for this speed, such as accuracy and resource constraints. Speed should be achieved under the condition of stability and minimizing the consumption of resources.

"Fast" can be broken down again. From a development point of view, it is necessary to write code quickly, develop quickly, test functions quickly, deploy the environment quickly, and start and stop services quickly; from the perspective of production, programs should run faster, or faster under high concurrency, and so on.

The micro-service architecture is popular because the service can be highly reused without having to write and modify the code frequently, which saves a lot of time. Containerization technology is popular because containerization technology can make the production environment consistent with the test environment, save a lot of environment deployment time, reduce the possibility of errors, and increase container nodes at will to enhance business processing capability. ensure fast response under high concurrency. The same is true of distributed architecture. Micro-service architecture is actually an evolution of distributed architecture. All changes remain the same, all in pursuit of "fast".

Returning to the topic of "China-Taiwan", the goal of building China-Taiwan is to avoid repeated construction and maintenance and respond quickly to demand. The systems of the background and the platform are relatively stable and generally do not change easily, and from the point of view of stability, the number of updates of the background and platform systems should be reduced as much as possible, and the front-end system is constantly changing in order to meet the needs of users. in this way, there is a contradiction between the foreground and the background in docking, and we hope to establish an intermediate platform between the two. It is in line with the idea of "fast" to centralize the things that can be reused in the front end and backstage into this intermediate platform and re-encapsulate the combination to provide services.

This is the source and foundation of China Taiwan, and enterprises must first understand these before building China Taiwan to see whether the China Taiwan to be built is in line with the idea of "avoiding repeated construction and maintenance" and the principle of "speed."

The second part: the key problems to be solved in the application of micro-services in business.

Reporter: this year, Yixin opened up the micro-service task scheduling platform SIA-TASK, which has a wide range of applications within the Yixin technical team, and has been supported by many developers after open source. Can you introduce the design ideas and core functions of this platform? (what problems do the design and development of this platform want to solve)

Liang Xin: when talking about Zhongtai, in fact, I think "Zhongtai" is just a name. It can be called anything as long as it meets the principles of "avoiding repeated construction and maintenance" and "fast". For example, our micro-service scheduling platform SIA-TASK is a system very similar to CCTV.

Before introducing the design idea of SIA-TASK, let me introduce its background. Whether Internet applications or enterprise applications, there are a large number of batch processing tasks. Some task scheduling systems are often needed to help developers solve problems. With the gradual evolution of micro-service architecture, single architecture has gradually evolved into distributed and micro-service architecture. Many of the original task scheduling platforms can no longer meet the needs of business systems, so there are some distributed task scheduling platforms. These platforms have their own characteristics and shortcomings, such as do not support task scheduling, high coupling with business, do not support cross-platform and other problems, do not meet the needs of the new generation of micro-service architecture, so we developed a micro-service task scheduling platform (SIA-TASK).

SIA-TASK uses the SpringBoot system as the architecture selection, and carries out secondary development based on Quartz and Zookeeper to support the corresponding functions. The logical architecture diagram of SIA-TASK is shown below:

SIA-TASK is an one-stop solution for task scheduling platform, which accords with the current micro-service architecture model. It has the characteristics of cross-platform, orchestration, high availability, non-invasion, consistency, asynchronous parallel, dynamic expansion, real-time monitoring and so on.

To understand the task scheduling platform, you need to understand task scheduling first. Tasks are roughly divided into three categories: tasks that are executed at regular intervals; tasks that are executed at regular intervals but not concurrent; and tasks that are executed at regular intervals but can be concurrent. There may also be some order relationships between tasks, such as serial, parallel, branching, and so on.

When we schedule tasks, the following problems often occur.

Forgotten, a project has run-batch tasks, and the run-batch process continues after the project is offline, and it is not found until the generated log is full of disk space; at a single point, the run-batch process of a project has been a single point, and the process stops running after a machine failure; dependent, the run-batch process An and B of a project are dependent, so you can only set the time stagger of two tasks. If A delay occurs, you need to process the error data manually.

When we designed SIA-TASK, we separated the platform from the operation of the project team. SIA-TASK includes five modules, task executors, that is, businesses whose real business logic needs to run batches, belonging to the project team. When starting, the project team will expose the executed tasks as a service, which is registered with the task registry; the task registry takes a task and stores it in a persistent database, and choreographs it into dependent tasks at the same time. Finally, the task scheduling center gets the scheduled tasks that need to be executed in the task scheduling center, and makes remote calls.

In the whole process, task scheduling, scheduling and execution are separated from the project team, the project team only needs to care about the business logic code of task scheduling, and the rest is executed on the SIA-TASK platform, which is equivalent to atomizing each project task and turning it into a micro-service.

Only the business logic is left in the project team, scheduling, scheduling and execution are classified into the platform, and all the things that need to be handled by the code are realized through configuration, which is also in line with the idea of "maintenance to avoid repeated construction".

SIA-TASK is currently open source. Specific design ideas and features as well as deployment operations can be viewed at GitHub at   https://github.com/siaorg/sia-task.

Reporter: what scenarios does the micro-service task scheduling platform (SIA-TASK) apply to?

Liang Xin: SIA-TASK is based on HTTP protocol for remote scheduling. The support of high concurrency scheduling processing in actual business is definitely not ideal. If the business is highly concurrent and tens of millions of tasks need to be awakened per second, it is not suitable to use SIA-TASK. In addition, SIA-TASK can be used in almost all other scenarios.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.