In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-19 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Development >
Share
Shulou(Shulou.com)06/03 Report--
This article mainly introduces "what is the principle of distributed system service registration and discovery". In daily operation, I believe many people have doubts about the principle of distributed system service registration and discovery. I have consulted all kinds of materials and sorted out simple and useful operation methods. I hope it will be helpful to answer the doubts about "what is the principle of distributed system service registration and discovery". Next, please follow the editor to study!
Reasons for introducing service registration and discovery components
Let's look at a question first. if we are going to do a mall project now, as an architect, how should you design the architecture of the system? You must be thinking: it's not easy to copy Taobao's structure directly. However, in the real entrepreneurial environment, a project may be a close call, if you invest huge human and financial resources at the beginning, once the project fails, it will lose a lot of money.
As an experienced architect needs to combine the company's financial resources, human input budget and other current situation to choose the most suitable architecture is the king. Large websites are developed from small websites, and so is the architecture.
The architecture of any large website is not the same from the beginning, but the result of iterative evolution with the continuous increase of the number of users and data.
In the process of continuous iterative evolution of the architecture, we will encounter a lot of problems. The essence of technological development is to constantly find problems and then solve problems, solve problems and find problems.
Monolithic architecture
At the beginning of the establishment of the system, there may not be many users, all the business will be packed into an application package to run in the tomcat container, sharing a server with the database, this architecture is generally called a single architecture.
Single architecture-joint deployment of applications and databases
In the initial stage, this architecture is very efficient and can be iterated and launched quickly according to the feedback of users. However, with the increase in the number of users, the memory and CPU of a service are tight, which can easily cause bottlenecks. How to solve new problems?
Separation of application and data
As the number of user requests increases, the memory and CPU of a server continue to soar, and the response time of user requests becomes slower. At this time, you can consider taking apart the application and the database and each using a server. You see, the problem has been solved again.
Single architecture-separation of application and database
Suddenly one day the floor sweeping aunt accidentally touched the wire, one of the servers lost power, and all the users' requests were wrong, followed by a series of complaint calls.
Cluster deployment
A single instance can easily cause a single point of problem, such as server failure or service capability bottleneck, what should I do? You must have thought of the smart one, using clusters.
Application cluster deployment
Cluster deployment means that applications are deployed on multiple servers or virtual machines, and users randomly access one of the instances through service balance, so that the traffic of multiple instances is balanced. If an instance fails, it can be taken offline. Other instances can still provide services without being affected.
With the rapid increase in the number of users, the boss decided to increase investment to expand the size of the team. The efficiency of the development team has not improved significantly since the growth of the development team. In the past, small teams could iterate online once a week, but now it takes at least two to three weeks.
The business logic is becoming more and more complex, the coupling between the code is very serious, and modifying a line of code may introduce several online problems. The architect realized the need for architectural refactoring.
Micro-service architecture
When the single architecture evolves to a certain stage, the complexity of development and testing will increase, and the expansion of team size will also make their work coupling more serious.
When the single architecture encounters a bottleneck, the micro-service architecture comes out of nowhere. Micro-service is to split the previous single service according to the business dimension, the split granularity can be large or small, and the split time can be carried out at a rhythm. The best practice is to first separate some independent functions from the monomer into one or more micro-services, which can ensure the continuity and stability of the business.
Micro-service architecture
As shown in the figure above, a commercial application is split into six separate micro-services. Six microservices can be deployed with multiple instances using Docker containerization.
The evolution of the architecture has encountered a difficult problem. If you want to query all the orders of the user, the user service may rely on the order service. How does the user service interact with the order service? There are multiple instances of the order service. Which one should I access?
There are usually several solutions:
(1) Service address is hard-coded
The address of the service is written in the database or configuration file, and the address is routed by accessing the DNS domain name.
Service metadata hard coding
The address of service B is hard-coded in the database or configuration file. Service A first needs to get the address of service B, then obtains the real address of one of the instances through DNS server parsing, and finally initiates a request to service B.
If there is a need to expand the capacity of the service instance, and the service instance needs to be offline after the promotion, operation and maintenance personnel have to do a lot of manual operations, which is very easy to misoperate.
(2) dynamic registration and discovery of services
There is also a fatal problem with hard coding of service addresses. If an instance fails, the OPS staff may not be aware of it in time, resulting in abnormal requests from some users.
The introduction of service registration and discovery components can solve the problems encountered above and avoid too many manual operations.
Summary of Architecture Evolution
In a single architecture, an application is a service package, and the modules in the package call each other through function methods. The model is simple enough that there is no service registration and discovery at all.
In the micro-service architecture, an application will be divided into multiple micro-services, which will be deployed in different servers, different containers, or even multiple data centers, and the micro-services will call each other. Service registration and discovery have become an indispensable component.
Basic principles of Service Registration and Discovery
Service registration and discovery are two key steps of registration and discovery.
Service registration: the service process registers its own metadata information with the registry. It usually includes host and port number, and sometimes authentication information, protocol, version number, and running environment information.
Service discovery: the client service process initiates a query to the registry to obtain the information of the service. An important role of service discovery is to provide the client with a list of available services.
Service registration
Service registration has two forms: client registration and agent registration.
Client registration
Client registration is the work that the service itself is responsible for registration and logout. The registration thread registers with the registry when the service starts and logs itself out when the service is offline.
Client registration
The disadvantage of this approach is that the registration logout logic is coupled with the business logic of the service, and if the service is developed in a different language, multiple sets of service registration logic need to be adapted.
Agent registration
Agent registration is registered and unregistered by a separate agent service. Notify the agent service in some way when the service provider starts, and then the agent service is responsible for initiating registration with the registry.
Agent registration
The disadvantage of this approach is that more than one proxy service is referenced, and the proxy service should be kept in a highly available state.
Service discovery
Service discovery is also divided into client discovery and agent discovery.
Client discovery
Client discovery means that the client is responsible for querying the available service addresses from the registry, and after obtaining the list of available instance addresses, the client selects an instance to initiate a request call according to the load balancing algorithm.
Client discovery
This approach is very straightforward, and the client can control the load balancing algorithm. But the disadvantage is also obvious. The logic of obtaining instance address and load balancing is coupled with the business logic of the service. If the service discovery or load balancing changes, then all services have to be modified and re-online.
Agent discovery
Proxy discovery means that a new routing service is responsible for service discovery to obtain the list of available instances. If service consumers need to invoke an instance of service A, they can send the request directly to the routing service. According to the configured load balancing algorithm, the routing service can select an instance from the list of available instances and forward the request. If the instance is found to be unavailable, the routing service can retry on its own. Service consumers don't have to feel it at all.
Agent routing service registration
Heartbeat mechanism
If there are multiple instances of the service, and one of the instances is down, the registry can sense it in real time and remove the instance information from the list, also known as off-hook.
How to get off-hook? The most commonly used way in the industry is heartbeat detection. Heartbeat detection has two ways: active and passive.
Passive detection means that the service actively sends heartbeat messages to the registry. The time interval can be defined. For example, if the service is configured to send heartbeats every 5 seconds, if the registry does not receive the heartbeat message of the instance within three cycles, say 15 seconds, the instance will be removed from the list.
Heartbeat mechanism-passive detection
In the figure above, instance 2 of service A has been down and cannot actively send a heartbeat message to the registry. After 15 seconds, the registration will remove instance 2.
Active detection is initiated by the registry. Heartbeat detection messages are sent to all service instances in the list every few seconds, and the instance will be actively removed if it is not sent successfully or received a reply within multiple cycles.
Heartbeat mechanism-active detection
Comparison of service registration and discovery components commonly used in the industry
After understanding the basic principles of service registration and discovery, if you want to use service registration and discovery components in a project, how to select technology in the face of a large number of open source components?
In Internet companies, large companies with R & D strength generally choose self-research or secondary development based on open source components, but for small and medium-sized companies to directly choose an open source software will be a good choice.
Common registration and discovery components such as eureka,zookeeper,consul,etcd are no longer recommended because eureka abandoned maintenance in 2018.
Industry open source components
Let's compare the components with each dimension.
Component advantages and disadvantages interface type consistency algorithm zookeeper1. Powerful, not just service discovery
two。 Provide watcher mechanism to obtain the status of service providers in real time
3. Widely used, dubbo and other micro-service frameworks have been supported; 1. No health check-up
two。 Sdk needs to be introduced into the service, and the integration complexity is high.
3. Does not support multiple data centers; sdkPaxosconsul1. Out of the box, easy to integrate
two。 Take a health check
3. Support for multiple data centers
4. Provide web management interface; can not get real-time service change notification restful/dnsRaftetcd1. Out of the box, easy to integrate
two。 Strong configurability 1. No health check-up
two。 Need to cooperate with tripartite tools to complete the service discovery function.
3. Does not support multiple data centers; restfulRaft
On the whole, the function of consul is more complete and balanced. Let's take consul as an example to introduce it in detail.
Consul-- recommended Service Registration and Discovery of Open Source components
Get to know Consul briefly.
Consul is an open source worker developed by HashiCorp Company, which is developed in GE language and can be deployed easily out of the box. Consul is distributed, highly available and scalable for service discovery and configuration in distributed systems.
What are the advantages of Consul?
Service registration discovery: Consul provides a way to register services and discovery services through DNS or restful interfaces. The service can be chosen according to the actual situation.
Health check: Consul's Client can provide any number of health checks, either associated with a given service or with a local node.
Multiple data centers: Consul supports multiple data centers, which means that users do not have to worry about the high availability of Consul itself and the extended access brought by multiple data centers.
Architecture diagram of Consul
Consul architecture
The Consul implementation of multiple data centers depends on the gossip protocol protocol. The purpose of this is:
You do not need to use the address of the server to configure the client; service discovery is done automatically.
The work of health check for failures is not placed on the server, but distributed.
Usage scenarios of Consul
The application scenarios of Consul include service registration discovery, service isolation, service configuration and so on.
In the service registration discovery scenario, consul is used as the registry center. After the service address is registered in consul, you can use the dns and http APIs provided by consul to query. Consul supports health check.
In the service isolation scenario, consul supports setting access policies on a service-by-service basis, supports both classic and emerging platforms, and supports tls certificate distribution and service-to-service encryption.
In the service configuration scenario, consul provides key-value data storage function, and can notify changes quickly. Configuration sharing can be achieved with Consul. Services that need to read configuration can read accurate configuration information from Consul.
At this point, the study on "what is the principle of distributed system service registration and discovery" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 277
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.