Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to understand the micro-service gateway". In the daily operation, I believe that many people have doubts about how to understand the micro-service gateway. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "how to understand the micro-service gateway"! Next, please follow the editor to study!

What is a gateway?

Gateway, as a traffic portal, is actually a process of extending HTTP requests to back-end services. The gateway encapsulates back-end services and provides some more advanced features, such as authentication, monitoring, load balancing, caching, multi-protocol support, current limit, circuit breaker, and so on.

Why do you need gateway unified traffic ingress?

The client communicates directly with the micro-service. If a large number of micro-services are involved, the client code will be very complex.

Some services use protocols that are not Web friendly. One service may use Thrift binary RPC, while another service may use the AMQP messaging protocol, neither of which is browser-friendly or firewall-friendly. Outside the firewall, the application should use protocols such as HTTP and WebSocket, that is, the protocol needs to be adapted.

Another drawback of this approach is that it makes microservices difficult to restructure. Over time, we may want to change the way the system is divided into services. For example, we might merge two services or split a service into two or more services. However, if the client communicates directly with the microservice, it is very difficult to perform this kind of refactoring.

Unified maintenance of common functions

Interface authentication, current limitation, circuit breaker degradation, isolation, log monitoring and other general functions are implemented in the gateway without sinking to business logic to reduce the cost and risk of separate maintenance.

Unified access to the function of gateway

The traffic from the front end (including app or other sources) can be accessed at the unified gateway layer, which requires high-performance transparent transmission and high concurrent access.

Flow control

In the face of massive traffic, how do we protect the gateway from being washed down by large traffic through some anti-brushing technology; and how to protect the gateway in all directions through some quadrant current, degradation, circuit breaker and other technologies.

Protocol adaptation

Because in the internal development we all use the RPC protocol (thrift or dubbo) to develop and expose to the internal services, when the external services need to use this interface, we often need to convert the RPC protocol into the HTTP protocol.

Safety protection

Prevent malicious traffic, blacklist mechanism, limit illegal IP and other means to reject malicious traffic.

Key Point access in Gateway Design

The access layer needs to be able to carry a large amount of concurrent traffic. Nginx is naturally the asynchronous way of NIO, which can well support the business requirements of large concurrency. So we can put some core things, such as demotion, flow control, etc., on this layer (with the help of lua's application logic), so that it can prevent the traffic for us at the front end.

Async

For our unified gateway layer, how to use a small number of machines to access more services, which requires our asynchronous, used to improve more throughput. There are generally two strategies for asynchronization:

Tomcat/Jetty+NIO+servlet3

This strategy is commonly used. JD.com, Youzun and Zuul all choose this strategy, which is more suitable for HTTP. Asynchrony can be turned on in Servlet3.

Netty+NIO

Netty is born for high concurrency. At present, VIPSHOP's gateway uses this strategy. In VIPSHOP's technical article, under the same circumstances, Netty has a throughput of 30w + per second and Tomcat is 13wp. we can see that there is a certain gap, but Netty needs to deal with HTTP protocol on its own, which is troublesome.

Servlet can be used in cases where there are many HTTP request scenarios for gateways. After all, there is a more mature HTTP protocol. If you pay more attention to throughput, then you can use Netty.

Full-link async

For incoming requests to be asynchronous, in order to achieve full-link async, we need to process outgoing requests asynchronously. for outgoing requests, we can use the asynchronous support of rpc to make asynchronous requests, or we can consider using RxJava to adopt observer mode for asynchronous requests.

Flow control

Current limiting, downgrade, fuse demotion (after fuse is equivalent to demotion)

Separation (isolation) Separation of request parsing and business processing

Figure-tomcat separation

The first is to separate the request parsing thread from the later processing business thread through NIO. Requests are handled by a single thread of Tomcat, and a large number of link cases can be handled with a very small number of threads in NIO mode. Business logic processing and generating responses are handled by another Tomcat thread pool, thus isolated from the request thread. The business thread pool here can be further isolated and different thread pools can be set up for different businesses.

You can also use responsive programming, with the help of Reactor mode, to improve throughput and avoid blocking with few fixed number of threads and less memory. Spring-WebFlux is a responsive programming framework.

Business thread pool separation

The second is business thread pool separation, which isolates different interfaces or different types of interfaces through a thread isolation technology.

For example, 20 separate threads are used to deal with the interface related to the order, and 10 separate threads are used to deal with the interface related to the commodity, so that different interfaces do not affect each other. If there is a problem with this part of the order, it consumes itself at most and will not affect the invocation of threads of other interfaces.

Specific thread isolation can specify the number of threads based on the business, which are prepared for fixed interfaces.

When there is a problem with this interface, it uses up its own number of threads and does not occupy threads from other interfaces, which acts as thread isolation so that when a single API goes wrong, it will not affect others.

Semaphore isolation

Semaphore isolation only limits the total number of concurrency, and the service or the main thread makes synchronous calls. This isolation will still affect the main thread if the remote call times out, which will affect other businesses. Therefore, if you just want to limit the total concurrent calls to a service or if the invoked service does not involve remote invocation, you can use lightweight semaphores. Because there is no custom filter in the like gateway, semaphore isolation is selected.

Thread pool isolation

The simplest thing is that different businesses are isolated through different thread pools. Even if there is a problem with the business interface, because the thread pool has been isolated, it will not affect other businesses. In JD.com 's gateway implementation, thread pool isolation is adopted, and more important businesses such as goods or orders are processed separately through the thread pool. However, because it is a unified gateway platform, if there are many lines of business, everyone feels that their business is more important and needs separate thread pool isolation. If you are using Java language development, then threads are relatively heavy and resources in Java are relatively limited. If too many thread pools need to be isolated, it is not very suitable. If you use other languages such as Golang to develop gateways, threads are lighter resources, so it is more appropriate to use thread pool isolation.

Cluster isolation

What if some businesses need to use isolation but Unified Gateway does not have thread pool isolation? Then you can use cluster isolation, and if some of your businesses are really important, you can apply for a single cluster or multiple clusters for this series of businesses, which can be isolated between machines.

Fast failure-timeout in the link

Timeout setting is very important for a distributed system. If no timeout is set, slow response to requests may accumulate and lead to chain reactions or even avalanches, including timeouts for Mysql, Redis, RPC services, HTTP services and so on. Quick failure is a very important practice, not only for gateway systems, but also for other systems, which need to focus on timeout settings throughout the chain.

Chain processing

In the design pattern, there is a pattern called the chain of responsibility pattern, whose role is to prevent the request sender from being coupled with the receiver, make it possible for multiple objects to receive the request, connect these objects into a chain, and pass the request along the chain. until an object processes it. This pattern decouples the sender of the request from the processor of the request. There are implementations of this pattern in all of our frameworks, such as Interceptor in filter,springmvc in servlet. Zuul adopts this mode, which is divided into pre-filter, routing filter, post-filter and error filter. A request first passes through all the pre-filters in sequence, then forwards it to the back-end application in the route filter, gets the response through all the post-filters, and finally responds to the client. If an exception occurs in the whole process, it will jump to the error filter.

At this point, the study on "how to understand the micro-service gateway" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.