Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the method of micro-service design". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought. Let's study and learn "what is the method of microservice design"!

I. the theory of micro-service architecture

1. Hexagonal structure

1) hexagonal architecture (Hexagonal Architecture), also known as port and adapter architecture style; uses adapters to interact with the outside world, and the outside world interacts with the interior through the application layer API.

2) the classical hierarchical architecture pays more attention to the separation of abstractions, and the responsibilities of each layer are clearly defined. In the hexagonal architecture, the form of "componentization" is used to avoid coupling, and each business unit is minimized as much as possible, which is summed up in one word, that is, "flattening".

3) the classical hierarchical architecture is divided into four layers, while for hexagonal architecture, it is generally divided into three layers: domain layer, port layer and adapter layer.

two。 The size of the service

1) in some scenarios, it is considered that the development workload of each microservice should not be more than 2 weeks.

2) according to the Golden Rule, the modification and deployment of a micro-service will not affect other services.

3. Advantages of micro-services

1) technological heterogeneity

2) High fault tolerance, service can be degraded (bulkhead). A single service failure does not cause an entire system failure.

3) already expanding, you only need to expand the applications with high load.

4) it is easier to simplify deployment than to deploy a single large application.

5) match the organizational structure and make it easy to assign work.

6) combination and reusability.

7) alternative optimization can avoid the pain of being unable to transform a large single application.

4.OSGI

OSGI (Open Services Gateway Initiative) supports module hot deployment and facilitates module management. But in the end, it is based on jvm, and the failure of a service will still cause the whole jvm crush.osgi to be suitable for building internal applications with a single service node. for micro-service architecture, most scenarios are a bit impractical; after java9 launched JPMS, the application scenario of OSGI is even more squeezed.

2. Microservice architect

1. Principle

1) Evolutionary architects should look at problems from an evolutionary perspective, instead of paying attention to specific technical details and pursuing perfection.

2) Partition architects should not pay too much attention to the problems within the service, but should pay more attention to the problems between regions.

3) it is not the responsibility of the architect to set strategic goals

4) Code governance control example

5) assess technical debt

6) exception management

7) centralized governance and team building

two。 Design standard

1) Monitoring health monitoring should be standardized and monitoring services should not be changed for specific services.

2) Interface

A.API should be decoupled from the consumer, should choose a technology-independent implementation, and easy to upgrade, such as you can add fields at will without affecting existing consumers.

B.REST API design should be based on "resources", stateless, with no verbs in URL, only nouns, and manipulating resources such as GET, POST, DELETE, etc.

GET: query, POST: add, PUT: update, DELETE: delete

C.API should have the concept of version.

d. Use Token for authentication instead of cookie.

In the e.RFC standard, the case in URL is sensitive, although it is not sensitive in practical use, you should use lowercase uniformly and use-instead of _ as the string separator. Search engines think that the character "-" is multiple keywords, and IE browsers do not support "_" well.

f. Speed limit

If you do not control the number of visits, you are likely to be attacked by DDos. For this reason, RFC6585 introduces HTTP status code 429 (too many requests).

Three related headers used by Github API:

X-RateLimit-Limit: the maximum number of requests that users are allowed to send per hour

X-RateLimit-Remaining: the number of available requests left in the current time window

X-RateLimit-Rest: when the time window is reset, the number of requests available at this point in time becomes the value of X-RateLimit-Limit

g. Using HTTP headers to handle caching and concurrency

h. Large result sets should be paginated

i. Avoid situations where multiple services share a database

3) if you are concerned about architecture security, you should properly handle error requests, and circuit breakers can be used.

Third, micro-service modeling

1. Principle of high cohesion and loose coupling

two。 Use domain engineering and boundary context to divide services

3. Avoid prematurely dividing services

4. Priority should be given to geographical location and organizational structure, and the division according to technical seams is not necessarily correct.

IV. Integration

1. Choose reasonable interface technology

two。 Reasonable selection of arrangement and coordination

In general, coordination can better reduce the degree of coupling.

3. To avoid catastrophic failover such as messages that will cause consumers to crash, set the maximum number of retries and move them to the dead letter queue in a timely manner.

4. Access to service resources by reference, such as when invoking the mail sending module, is often delayed, and the sending content may have been modified. The correct way is to send only the reference and query the details when sending the email.

5. The service should be read with fault tolerance.

Postel law, robustness principle that each module should be broad in and strict out, strict with what they send, and fault tolerant to what they receive.

Decompose the monolithic system

1. Through the boundary context, find out the system seams, and split the service and database through the seams. The best practice for refactoring a database is to detach the database before splitting the application.

VI. Deployment

1.CI (Continuous Integration, continuous integration)

CD (Continuous Delivery, continuous delivery)

a. Build the pipeline, decompose the build into multiple phases, and fail quickly.

b. In the initial phase of the project, all services can be placed in a single build, and when the services are stable, each microservice should have its own build.

c. Customize the image to speed up deployment; pre-install the micro-service itself in the image, using the image as a build.

d. Unified configuration file management, the development of dedicated deployment system.

VII. Testing

1. Consumer driven contract testing (Consumer-Driven Contract,CDC)

CDC can effectively avoid changing consumers who damage new services. CDC defines consumer expectations, which become test code for producers.

two。 Test after deployment

It is not possible to reduce the defect rate to 0 by relying solely on pre-deployment testing. Deployment and launch are different stages. Two systems can be deployed through blue and green deployment, but only one system accepts the real request. Canary releases can also be used to direct a small portion of real traffic to a new version for verification.

3. Resident fault repair time MTTR is better than average failure interval time MTBF

In 4.Scrum agile software development, automated testing is divided into three layers: unit testing, service testing and user interface testing.

VIII. Monitoring

1. Using semantic monitoring, we can run some test cases in the production environment, and if these cases do not reach our expected value, we will think that there is something wrong with a certain environment in the production environment, such as opening the home page regularly.

two。 The calling chain is marked by the associated identity, for example, the identity is appended to the header of the HTTP.

3. Establish a standardized log, record the log in a standard format, and unify the service indicator name (response time, error rate, application indicator, etc.).

IX. Safety

1. Authentication and authorization methods

1) SSO

2) SAML Security assertion markup language is based on XML/SOAP 's open source standard data format

3) OpenID Connect protocol 4) the single sign-on gateway uses the gateway to verify identity and authorization; after the authentication is passed, the gateway puts the principal information on the HTTP header.

Defense in depth should be carried out and should not rely entirely on single sign-on gateways. HTTPS can also be terminated on the gateway and intrusion detection can be carried out.

The single sign-on gateway should only be responsible for coarse-grained authorization, while fine-grained authorization should be implemented in the service.

two。 Inter-service authentication and authorization

1) everything is allowed within the boundary.

2) use HTTPS basic authentication

3) use SAML or OpenID Connect

4) use client certificate

5) use HAMC over HTTP

HMAC can be used when using HTTPS consumes a lot of performance and cannot be cached.

HMAC (Hash-based Message Authentication Code, HASH-based message code), which uses a symmetric key to HASH the requested subject together with the key.

The server also hash to know whether the data has been tampered with, but can not prevent data leakage. This method is used in Amazon S3.

6) API key

7) Traffic on SSL cannot be cached by reverse proxy server. The CDN price of HTTPS is higher.

8) confusing the agent problem

Successfully authenticated users may disguise themselves as other users and access other people's data.

3. Data security

Use the password hash with salt

Sensitive data should not be written to the log

4. Defense in depth

Firewall / log / intrusion detection / network isolation / operating system

5.OWASP Security Framework Security threat Top10

The relationship between organization and system design

1. Conway's law

The design delivered by any organization is structurally consistent with the communication structure of the organization.

Reverse Conway's law

Poorly designed systems sometimes force organizations to modify their structure to suit the system.

two。 Tissue coupling degree

In general, loosely coupled organizations develop products with better modularization and lower coupling.

Typical tightly coupled organizations such as commercial product companies and loosely coupled organizations such as open source communities.

3. The relationship between Organization and Software quality

In the development of windows vista, it is found that the indicators associated with organizational structure have the highest correlation with software quality. Not common metrics such as code complexity.

4. The organization is responsible for the entire lifecycle of the service

Organizations should have ownership of services, rather than multiple organizations sharing ownership of services.

In projects involving multiple organizations, not sharing service ownership can lead to communication problems. A more effective solution is to open source internally, open code and documentation to each other, and contribute and merge code like open source projects.

Ensure code quality and continuous maintainability.

5. The organizational structure should be consistent with the boundary context.

6. Services between different lines of business should be batch-processed asynchronously so that they can be stopped for maintenance at any time.

11. Large-scale micro services

1. Deployment noun

1) Blue and green deployment

Two sets of redundant systems are deployed at the same time to achieve non-downtime upgrade. It involves in-transit business and data migration.

2) ABG B test

Some users use the old system / some use the new system to verify the reliability of the new system.

3) Grayscale release / canary release

While keeping the old version available, deploy a new system, and if there is a problem with the new system, immediately switch back to the old system.

4) Rolling release

There is only one set of clusters without redundancy, and some of them are taken out at a time, such as 20% of the server upgrades, until all the upgrades are completed.

two。 Architectural security measures

In the event of a failure, it should be able to properly downgrade the function, rather than take the entire system offline.

It is necessary to prevent the collapse of the Cascade University due to the failure of a certain service.

Architectural security measures:

The harm caused by timeout / delay is often fatal, so we should pay attention to the use of timeout mechanism.

Circuit breaker, when there is a problem downstream, disconnect in time and fail quickly.

Bulkhead, isolate the influence.

Isolation, design allows downstream offline, and services are isolated from each other.

3. Idempotent

4. Expansion

Vertically scale better hosts

Split the load into different services

Disperse risk and deploy

The load balancer SSL often terminates in front of the load balancer. In order to prevent information leakage, you can try to put the load balancer and the backend instance in the same LAN.

In addition to load balancing, worker-based systems can also reduce vulnerability and load through message queues.

5. Database expansion

Read-write separation extended caching is often more efficient than read-write separation.

Write extended data sharding, better host, sharding expansion is difficult, can not solve HA.

Avoid sharing a database infrastructure and building multiple schema with one physical database.

Try to use CQRS (Command query Duty Separation Mode) to divide the application into two parts: the command side and the query side. The command-side processor creates, updates, and deletes requests and issues events when data changes. The query side is responsible for the query.

6. Caching

1) client cache / reverse proxy and CDN cache / server cache (primary cache, secondary cache)

2) HTTP caching

A.cache-control TTL (Time To Live) specifies how many seconds to cache

b. Set the Expires header to specify a specific time for caching, such as the launch time of the new version

c. Set Etag to add random strings after URL

3) increase the write cache if necessary, write the cache first, and then write to the database.

4) when a failure occurs downstream, using elastic caching is better than stopping the service.

5) Hidden source services should avoid avalanches in the event of cache failures or mass failures. The source service can be protected through quick failures, database local-level caching, and so on.

6) Multi-level caching should be avoided, do not use too long cache expiration time, caching in ISP and user browsers is uncontrolled.

7) the Guardian uses crawler technology to crawl its own web page and has a static web page that can be used in the event of a system failure.

7.CAP theorem

Cannot satisfy consistency Consistency, availability Availability, and partition fault tolerance Partition at the same time

Because distributed systems must be partitioned, there is often a tradeoff between AP or CP.

You can use AP in some functions and CP in others.

XII. Principle of micro-service

1. Micro-service concept

Small services with core autonomy

Modeling around business concepts

The culture of automation

Hide the internal implementation details

Everything is decentralized.

Independent deployment

Isolation failed

Highly observable

two。 problem

Be familiar with business areas and divide services.

Cluster deployment / monitoring and other issues.

It is not recommended to develop micro-services from 0 and give priority to single services.

Data integration should not be used.

Micro-services should be business-oriented, not technology-oriented; technology-oriented should be avoided.

Thank you for your reading. the above is the content of "what is the method of micro-service design". After the study of this article, I believe you have a deeper understanding of what the method of micro-service design is. Specific use also needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.