Shulou(Shulou.com)06/03 Report--

Software Architecture: business Architecture, Application Architecture and Cloud Infrastructure

This section is excerpted from "Software Architecture Design"

Software development is to decompose a complex problem into a series of simple problems, and then combine a series of simple solutions into a complex solution. The biggest challenge in software development is that it can not only quickly and efficiently make changes in response to changes in the demand and environment, but also continue to provide stable and highly available services. The software architecture is the skeleton and framework of the software system.

The so-called architecture is of different opinions, and it is very difficult to have a clear or standard definition. However, architecture is not mirrored or bright. Where there is a system, architecture is needed, from aviation to aircraft to a functional component in an e-commerce system. Design and architecture are required. Abstractly speaking, architecture is the abstract description of the entities in the system and the relationship between them, and the distribution of the correspondence between the functions of objects / information and the formal elements. it is the definition of the relationship between elements and the relationship between elements and the surrounding environment. The architecture can divide the target system according to a certain principle, the principle is to facilitate different roles to work in parallel, well-structured creative activities are better than unstructured creative activities.

The core value of software architecture is to control the complexity of the system, to separate and decouple the core business logic and technical details. The software architecture is the sketch of the system, and the objects it describes are the abstract components that directly constitute the system, while the connections between the components clearly and relatively carefully describe the communication between the components. In the implementation phase, these abstract components are refined into actual components, such as a concrete class or object. In the object-oriented field, the connection between components is usually realized by interface. The responsibility of the architect is to train his own thinking, to use it to understand complex systems, to understand and analyze requirements through reasonable decomposition and abstraction, to create useful models, to confirm, refine and expand the model, and to manage the architecture; to be able to decompose the system to form an overall architecture, to make correct technical selection, to develop technical specifications and effectively promote implementation.

Software architecture classification

In the author's knowledge system, the architecture is actually divided into business architecture, application architecture and cloud infrastructure. Business architecture mainly focuses on controlling the complexity of the business. Infrastructure focuses on solving a series of problems in distributed systems. No matter what kind of architecture, we hope to achieve the variability of the system while ensuring the high availability of the business. On the other hand, according to the division of responsibilities in the enterprise, we can often divide software architecture and associated architects into the following categories:

Business architecture / solution architecture: the core is to solve the system complexity caused by the business, understand customer / business pain points, project definition, and existing environment; sort out high-level and non-functional requirements; carry out problem domain division and domain modeling; communicate, propose solutions, iterate many times, and deliver the overall architecture.

Application architecture: according to the needs of the business scenario, design the hierarchical structure of the application, develop application specifications, define interfaces and data exchange protocols, and so on. And try to control the complexity of the application to an acceptable level, so as to quickly support the business development, while ensuring the availability and maintainability of the system, ensure that the application meets the requirements of non-functional attributes (performance, security, stability, etc.).

Data architecture: focus on building a data center, unifying data definition norms, standardizing data expression, and forming effective and easy-to-maintain data assets. Create a unified big data processing platform, including data visualization operation platform, data sharing platform, data rights management platform and so on.

Middleware architecture: focus on the construction of middleware systems, need to solve the server load, distributed service registration and discovery, message system, cache system, distributed database and other problems, while the architect has to make a tradeoff between CAP.

Operation and maintenance architecture: responsible for the planning, selection and deployment of the operation and maintenance system, and establish a standardized operation and maintenance system.

Physical architecture: physical architecture focuses on how software components are placed on hardware, focusing on infrastructure, certain software and hardware systems, and even cloud platforms, including computer room construction, network topology, network shunt, proxy server, Web server, application server, report server, integration server, storage server and host. Architectural pattern and architectural style

A core issue of software architecture design is whether repetitive architecture patterns can be used, that is, whether software reuse at the architecture level can be achieved. That is to say, whether the same architecture can be used in different software systems. When we discuss software architecture, we often mention software architecture pattern (Architectural Pattern) and software architecture style (Architectural Style).

Software architecture patterns are often used to specifically solve a specific repetitive architectural problem, while the architectural style is the naming of a specific architectural design. Software architecture style is an idiomatic pattern that describes the organization of systems in a particular application domain; it reflects the structural and semantic characteristics shared by many systems in the domain, and guides how to effectively organize each module and subsystem into a complete system.

In the author's series of articles, CRUD, hierarchical architecture, hexagonal architecture, onion architecture, REST and DDD are all architectural styles, while CQRS, EDA, UDLA, microservices and so on are classified into architectural patterns.

The Source and Countermeasures of system complexity

In software development, programmers are often able to break away from the shackles of reality and create a wild world, which is also one of the most creative activities. The only thing that programming requires is creative thinking and the ability to organize thinking, which means that the biggest limitation in the software development process is to understand the objects we are creating. With the evolution of the software and the addition of more function points, the system becomes more and more complex: there are all kinds of subtle dependencies among the Module. The complexity of the system accumulates over time, and it becomes more and more difficult for programmers to consider all the relevant factors when modifying the system. This will slow down the progress of software opening and introduce Bug, which will further slow down the development progress and increase the development cost. In the life cycle of any system, complexity will inevitably increase; the larger the system, the more people are needed to develop, and the more difficult it is to manage the complexity of the system.

In his book Domain-Driven Design, Eric Evans complains about the so-called spaghetti architecture, that is, the code does do something useful, but it is difficult to explain how it is executed; he believes that the main reason for this dilemma is that the complexity of domain problems is mixed with the complexity of technical details, resulting in an exponential increase in overall complexity.

Complexity does not come out of thin air, and most of the time it is not intentional, which means that the increase in complexity is often not transferred by our subjective will. Like the elephant in the room, we can't escape or turn a blind eye to it. The source of complexity may be:

Accretion and continuous iteration: incremental design means that software design never ends, design continues to occur in the life cycle of the system, programmers should always consider design problems. Incremental development also means continuous refactoring. The initial design of a system is almost never the best solution. With the increase of experience, it is inevitable to find a better design.

Interactive and non-scalable design: when the large-scale system caused by accretion effect, combined with the feature of interaction, it will make the technical system more complex. A technical system not only acts on itself, but also interacts with a large number of other systems. For example, if you place an order to buy a commodity, then the order system, commodity system, payment system, logistics system, card voucher system will cooperate with each other. The complexity of such accretion will increase geometrically due to the emergence of interactive characteristics.

Unreasonable business encapsulation: unreasonable business encapsulation is a relatively broad concept, such as process-oriented rather than object-oriented, unreasonable layering and so on.

Lack of a unified language: in a typical agile development structure, the roles on the assembly line tend to focus on their own responsibilities, and the fine division of labor limits the overall perspective of each role; although we often promote the so-called sense of ownership, but it's hard to push it when you land.

Lack of constraints and specifications: in the context of team collaborative development, the lack of specifications and constraints will seriously damage architectural consistency (Consistency), and the maintainability of code will decline sharply. It is possible that the specification at the implementation level is naming, subcontracting and other minor problems that do not affect the operation of the code, but the dike of thousands of miles is broken by the ant nest, and it is the inattention of these micro particles that leads to the avalanche of overall complexity.

The response to complexity will never be once and for all, we need to constantly push through the old and bring forth the new, and it is a continuous iteration to dynamically and gradually reshape our understanding of the software system, constantly understand problems and find better solutions. The first way to control complexity is through simple code and Obvious. For example, reducing the handling of special scenarios, or variable naming consistency can reduce system complexity. Another way is to abstract complex problems and divide and conquer them.

Domain-driven design

This section is excerpted from "Domain-driven Design"

DDD domain-driven design originated from his most influential book "Domain-Driven Design-Tackling Complexity in the Heart of Software" published by Eric Evans, a famous modeling expert in 2004. Eric Evans only provides a set of original theories, but not a set of methodology, so there are different opinions on DDD for many years. Earlier, MartinFowler put forward the concept of anemia model and hyperemia model. He thought that most of our systems take POJO as the model, only ordinary getter and setter methods, and there is no real behavior, like people who lack blood. In Evans's view, models in DDD exist in the form of hyperemia, that is to say, in DDD, our model contains not only business attributes, but also methods that can describe actions. Some concepts in the domain can not be used in model objects, such as warehousing, factories, services, etc., such as imposed on the model, will destroy the definition of the model.

The strategic core of domain-driven design is to separate the problem domain from the application architecture, manifest the business semantics, and transform the previously obscure business algorithm logic into a clear explicit expression of domain concepts through domain objects (Domain Object) and unified language (Ubiquitous Language).

Unified language, software developers / users all use the same language, that is, the cognition of a concept and nouns is unified, a clear business model is established, and a unified business semantics is formed. Use the model as the pillar of the language. Make sure the team uses this language in all internal communication, in code, in drawing and writing, especially when speaking. Such as account, transfer, overdraft strategy, these are very important domain concepts, if these names are consistent with our daily discussion and the description in PRD, it will greatly improve the readability of the code and reduce the cognitive cost. For example, no one will repeatedly confirm the meaning of "work order", "audit order" and "form" in meetings, and the modeling of DDD will not be kidnapped by DB.

Domain-oriented, business semantics are explicit, thinking about problems in terms of domain, not modules. Extract the implicit business logic from a push if-else, name it, write the code, extend it in a common language, and turn it into a display concept; many important business concepts, according to the way transaction scripts are written, are completely submerged in code logic without prominence.

The division of responsibilities, according to the reasonable division of the model according to the actual business, the dependency structure and boundary between the models are clearer, which avoids the chaotic dependency relationship, and then increases the readability and maintainability; for a single responsibility, the model only pays attention to its own work, and avoids the chaotic calling relationship caused by "ultra vires". Through modeling, we can better express the complex business in the real world, and with the development of time, we will continue to increase the precipitation of the system to the actual business, and will better describe the business logic through clear code. The cohesion of the model increases the high degree of modularization of the system and improves the reusability of the code. Compared with the traditional three-tier model, it is very likely that a large number of repeated functions are scattered in each Service. Micro Services and Cloud Native Architecture

This section is excerpted from "Micro Services and Cloud Native"

Single hierarchical architecture

In the early days of Web application development, most projects packaged all server-side functional modules into a single Monolith application, for example, many enterprise Java applications were packaged into war packages, resulting in the following architecture:

Stonehenge application is easy to build development environment, easy to test, easy to deploy; its defects are also very obvious, unable to make local changes and deployment, compilation time is too long, regression testing cycle is too long, development efficiency is reduced. The centralized architecture is divided into three standard layers: the data access layer, the service layer and the Web layer.

At the beginning of the Web2.0 era, there is no essential difference between Internet applications and enterprise applications. The centralized architecture is divided into three standard layers: data access layer, service layer and Web layer.

The data access layer is used to define the data access interface to achieve access to the real database; the service layer is used to deal with the application business logic; and the Web layer is used to handle exceptions, logic jump control, page rendering template and so on. SOA Service oriented Architecture

SOA (Service-Oriented Architecture) service-oriented architecture is a relatively broad concept involving modular development and distributed expansion deployment under the background of the rapid growth of Internet applications and the centralized architecture has been unable to improve the throughput of the system indefinitely.

SOA is a component model that connects different functional units of an application (called services) through well-defined interfaces and contracts between these services. The interfaces in SOA are defined in a neutral way, independent of the hardware platform, operating system and programming language that implements the service. This allows services built in a variety of systems to interact in a unified and common way. Service-oriented architecture, which can distribute, combine and use loosely coupled coarse-grained application components through the network according to the requirements. The service layer is the foundation of SOA, which can be directly called by the application, so as to effectively control the human dependence of interaction with the software agent in the system.

The key goal of implementing SOA is to maximize the IT assets of enterprises. To achieve this goal, it is necessary to keep the following characteristics in mind in the process of implementing SOA: accessible from outside the enterprise, readily available, coarse-grained service interface classification, loose coupling, reusable services, service interface design management, standardized service interfaces, support for various message patterns, and well-defined service contracts.

A service consumer (Service Consumer) can invoke a service by sending messages that are converted by a service bus (Service Bus) and sent to the appropriate service implementation. This service architecture can provide a business rule reference (Business Rules Engine), which allows business rules to be merged into one service or multiple services. This architecture also provides a service management infrastructure (Service Management Infrastructure) for managing services, such as auditing, billing, logging, and so on. In addition, the architecture provides enterprises with flexible business processes to better handle control requests (Regulatory Requirement), such as Sarbanes Oxley (SOX), and to change a service without affecting other services.

Because the distributed system is very complex, a large number of distributed middleware and distributed databases have been produced to simplify the development of distributed systems, and the concept of service-oriented architecture design has been recognized by more and more companies. Here is an official Dubbo document that publishes a picture of the evolution of the SOA system:

MSA micro-service architecture

Microservices Architecture Pattern, proposed by Martin Fowler in 2014, aims to transform a single application into an aggregation of services or applications that can be run, developed, deployed and maintained independently, so as to meet the needs of rapid business changes and distributed multi-team parallel development. As stated in Conway's Law Law, when any organization designs a system (generalized concept), the design solution delivered is consistent with the communication structure of the organization. Micro-service and micro-front-end are not only the changes of technical architecture, but also include the changes of organization and communication.

For micro-services, people from different backgrounds also have different views. For developers who are familiar with SOA, micro-services can also be regarded as an implementation scheme of SOA without ESB; ESB is the central bus in SOA architecture, and the design graphics should be star-shaped, while micro-services are decentralized distributed software architecture. SOA places more emphasis on reuse, while microservices tend to rewrite. SOA tends to horizontal service, micro-service to vertical service, SOA to top-down design, and micro-service to bottom-up design.

Micro-service is the same as the principles of micro-front-end and software engineering and object-oriented design, which follow the basic principles of single responsibility (Single Responsibility), separation of concerns (Separation of Concerns), Modularity (Modularity) and Divide & Conquer. The evolution from Stonehenge applications to microservices did not happen overnight, and the following figure also illustrates a simple gradual substitution process:

Cloud Native Cloud Native Architecture

Cloud natives use automation and architecture to manage system complexity and liberate productivity by building teams, cultures, and technologies.

-Joe Beda,Heotio CTO, co-founder

Pivotal is the proponent of cloud native applications, and launched the Pivotal Cloud Foundry cloud native application platform and Spring open source Java development framework, becoming a pioneer and pathfinder in cloud native application architecture. Back in 2015, Pivotal's Matt Stine wrote a booklet called migrating to Cloud Native Application Architecture, which discussed several main features of cloud native application architecture: compliance with 12 Factors applications, micro-service-oriented architecture, self-service agile architecture, API-based collaboration, and anti-vulnerability. In 2015, Google led the establishment of the Cloud native computing foundation (CNCF). At first, CNCF's definition of cloud native (Cloud Native) includes the following three aspects: application containerization, micro-service-oriented architecture, and application support container scheduling.

Cloud native applications are simply defined as building applications for cloud computing architectures from scratch; this means that if we design applications that are expected to be deployed on a distributed, scalable infrastructure, our applications are cloud native. As the public cloud will carry more and more computing power, and cloud computing will be the mainstream way to deliver IT capabilities in the future, CNCF has also redefined cloud nativeness: cloud native technology enables organizations to build and run scalable applications in new dynamic environments such as public cloud, private cloud and hybrid cloud; the representative technologies of cloud natively include containers, service grids, micro services, immutable infrastructure and declarative API.

Codeless corresponds to service development and implements source code hosting. You only need to focus on your code implementation, not where your code is, because you will not feel the existence of code bases and code branches throughout the development process.

Applicationless corresponds to service publishing. Under the service-oriented framework, your service publishing no longer needs to apply for an application, nor does it need to pay attention to where your application is.

Serverless corresponds to service operation and maintenance. With Serverless capability, you no longer need to pay attention to your machine resources. Servlerless will help you to flexibly expand and scale machine resources.

The combination of these technologies can build a loosely coupled system that is fault-tolerant, easy to manage and easy to observe; combined with reliable automation, cloud native technology enables engineers to easily make frequent and predictable major changes to the system. It can be seen that cloud native is an effective way to ensure the flexibility of system capabilities, and cloud native technology helps organizations to build and run flexible and scalable applications in new dynamic environments such as public cloud, private cloud and hybrid cloud. Micro-service architecture is very suitable for cloud native applications; however, cloud native applications also have certain limitations. If your cloud native applications are deployed on public clouds such as AWS, cloud native API is not cross-cloud platforms.

Key attributes of cloud native applications include: lightweight container packaging, development using the most appropriate language and framework, loosely coupled micro-service design, API-centric interaction and collaboration, stateless and stateless services with clear architectural boundaries, independent of underlying operating systems and servers, deployment on self-service, resilient cloud infrastructure, agile DevOps process management, automation capabilities, Resource allocation driven by definition and policy. Cloud native is an important development path of distributed applications, and its final state should be Distributionless. All distributed-related problems are solved by cloud platform. The development of distributed applications will be as convenient or even more convenient as the development of traditional applications.

Cloud Infrastructur

This section is excerpted from "Virtualization and orchestration of distributed Infrastructure"

Virtual machine

The virtual machine consists of some specific hardware and kernel virtualization that runs the guest operating system. Software called hypervisors create virtualized hardware, which can include virtual disks, virtual network interfaces, virtual CPU, and so on. The virtual machine also includes a guest kernel that can communicate with this virtual hardware. The hypervisor can be managed, which means that it is software that runs on the host operating system (MacOS), as shown in the example. It can also be bare metal, running directly on the machine hardware (replacing your operating system). Either way, the hypervisor approach is considered heavyweight because it needs to virtualize multiple parts (if not all hardware and kernels).

VM requires hardware virtualization to achieve machine-level isolation, while containers only need to be isolated within the same operating system. With the increase in the number of isolation spaces, the cost difference becomes very obvious.

Container

In the past few years, the cloud platform has developed rapidly, but what bothers operation and maintenance engineers most is the need to install the corresponding runtime environment for a variety of different development languages. Although automated operation and maintenance tools can reduce the complexity of environment construction, it still can not fundamentally solve the problem of environment.

The emergence of Docker has become a new watershed in the software development industry, and the maturity of container technology also marks the beginning of a new era of technology. Docker provides the ability for development engineers to encapsulate applications and dependencies in a portable container, a move that makes Docker have a tendency to sweep the entire software industry and change the rules of the game, much like what happened when smartphones first appeared-- changing the rules of the game in the entire mobile phone industry. Through container packaging, Docker enables development engineers and operation engineers to publish applications in the standardized way of mirror distribution provided by Docker, so that heterogeneous languages are no longer the shackles of the team.

Containers are software packages that contain application code, configuration, and dependencies to provide operational efficiency and productivity. Containers provide us with predictable, repeatable and immutable operational expectations, and the rise of containers is a huge driver of DevOps as a service, which can overcome the biggest security obstacles faced today. Containerization makes applications portable by virtualizing at the operating system level, creating a kernel-based isolated encapsulation system. Containerized applications can be placed anywhere without the need for dependencies to run or require the entire VM, thus eliminating dependencies.

As a stand-alone unit, the container can run on any host operating system, CentOS,Ubuntu,MacOS, or even non-UNIX systems like Windows. Containers also act as standardized work or computing units. A common example is that each container runs a single Web server, a single shard of a database, or a single Spark worker, etc., and you can easily expand the application by expanding the number of containers. Each container has a fixed resource configuration (CPU,RAM, number of threads, etc.), and the extension application needs to extend only the number of containers rather than a single resource primitive. This provides an easier abstraction for engineers when the application needs to zoom in or out proportionally. Containers are also a good tool for implementing a micro-service architecture, and each micro-service is just a set of collaborative containers. For example, Redis microservices can be implemented using a single master container and multiple slave containers.

Kubernetes and choreography

With the maturity of virtualization technology and the popularity of distributed architecture, cloud platforms used to deploy, manage and run applications are more and more mentioned. IaaS, PaaS and SaaS are the three basic service types of cloud computing, which respectively focus on infrastructure-as-a-service of hardware infrastructure, platform-as-service of software and middleware platforms, and software-as-a-service of business applications. With the emergence of containers, the original virtual machine-based cloud application has been completely transformed into a more flexible and lightweight container and scheduling cloud platform application.

However, the increasing fragmentation of container units leads to a gradual increase in management costs, and the demand for container orchestration tools is unprecedented. Kubernetes, Mesos, Swarm and so on provide powerful orchestration and scheduling capabilities for cloud native applications, which are distributed operating systems on cloud platforms. Container orchestration is a process in which multiple containers can usually be deployed to automate the implementation of an application. Container management and container orchestration engines such as Kubernetes and Docker Swarm enable users to guide container deployment and automate update, health monitoring, and failover processes.

Kubernetes is currently the most watched open source project in the world. It is an excellent container orchestration system used to provide one-stop service. Kubernetes, which comes from the Internet giant Google, draws on the idea of a Borg system that has been built by hundreds of engineers for more than a decade. It is extremely easy to install and flexible to connect with the network layer. The outstanding performance of Kubernetes and Mesos has brought about disruptive changes in the working mode of all kinds of engineers in the industry. They no longer have to pay attention to every server, and when there is a problem with the server, just replace it. Business development engineers no longer need to focus too much on non-functional requirements, just focus on their own business areas. On the other hand, middleware developers need to develop robust cloud native middleware to connect business applications with cloud platforms.

Kubernetes, Service Mesh and Serverless work together to interpret different levels of encapsulation and shield the details below. Kubernetes introduces different design patterns to achieve new, effective and elegant abstraction and management patterns for various cloud resources, which makes cluster management and application release become quite easy and error-prone. The widely used micro-service software architecture migrates the complexity of distributed applications to services. How to govern through global consistency, systematization, standardization and non-intrusive means has become a vital content under the micro-service software architecture. Kubernetes refines the decomposition granularity of applications and simplifies application development by taking service discovery, configuration management, load balancing and health check as infrastructure functions. The declarative configuration of Kubernetes is particularly suitable for the CI/CD process, and now there are open source tools such as Helm, Draft, Spinnaker, Skaffold and other open source tools that can help us release Kuberentes applications.

Service Mesh does this easily by stripping off the common and environment-related content of each service into the Sidecar process deployed on the edge of each service. This divestiture makes the service and platform can be fully decoupled and convenient for their evolution and development, but also makes the service lighter and helps to improve the timeliness of service start and stop. Because Service Mesh strips the logic related to service governance into Sidecar and acts as an independent process, the functions implemented by Sidecar naturally support multiple languages, creating more favorable conditions for the above services to adopt multilingual development. Through the technical closure of the service traffic of the whole network through Service Mesh, the system engineering involving traffic scheduling such as multi-activity in different places can be realized more elegantly, succinctly and effectively, and the grayscale and rollback of service version upgrade can be realized more conveniently so as to improve the quality of safe production. Due to the technical closure, it has created a new development space for difficult problems such as the governance and evolution of service traffic, troubleshooting, the economy of log collection and so on.

Extended reading

You can read the author's series of articles in Gitbook through the following navigation, covering technical data induction, programming language and theory, Web and large front-end, server development and infrastructure, cloud computing and big data, data science and artificial intelligence, product design and other areas:

Knowledge system: "Awesome Lists | CS Collection", "Awesome CheatSheets | Quick Learning Quick check Manual", "Awesome Interviews | essential for Job interview", "Awesome RoadMaps | programmer's Advanced Guide", "Awesome MindMaps | brain Map of knowledge context", "Awesome-CS-Books | Collection of Open Source Books (.pdf)"

Programming languages: "programming language theory", "Java practice", "JavaScript practice", "Go practice", "Python practice", "Rust practice"

Software Engineering, patterns and Architecture: "programming Paradigm and Design pattern", "data structure and algorithm", "Software Architecture Design", "cleanliness and Reconstruction", "R & D methods and tools"

Web and large Front end: "Modern Web Development Foundation and Engineering practice", "data Visualization", "iOS", "Android", "mixed Development and Cross-end applications"

Server-side development practice and engineering architecture: "server-side foundation", "micro-service and cloud native", "testing and high availability assurance", "DevOps", "Node", "Spring", "information security and * * testing"

Distributed Infrastructure: distributed system, distributed Computing, Database, Network, Virtualization and orchestration, Cloud Computing and big data, Linux and operating system

Data science, artificial intelligence and deep learning: "mathematical statistics", "data analysis", "machine learning", "deep learning", "natural language processing", "tools and engineering", "industry applications"

Product design and user experience: "product design", "interactive experience", "project management"

Industry applications: "industry myth", "functional domain", "e-commerce", "intelligent manufacturing"

In addition, you can go to xCompass to interactively retrieve and find the articles / links / books / courses you need, or view more detailed directory navigation information such as articles and project source code in the MATRIX article and code index matrix. Finally, you can also follow the official Wechat account: "the Technology Road of a Bear" for the latest information.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.