Shulou(Shulou.com)06/02 Report--

This article mainly introduces why the configuration center is needed in the Internet architecture, which has a certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article. Let's take a look at it.

The configuration center is a very important part of the Internet architecture, but why there is a configuration center, whether there is a configuration center at the beginning, and what problems it solves are the issues to be discussed today.

With the increasing complexity of Internet services and the increasing number of users and traffic, "service layering" is the only way of architecture evolution.

As shown in the figure above, the site application invokes the service, the upstream service invokes the underlying service, and the dependency becomes very complex.

For the same service:

(1) it often has multiple upstream calls

(2) in order to ensure high availability, it is often a cluster of several nodes to provide services.

As shown in the figure above, the user-centric service user-service has three nodes. Ip1/ip2/ip3 provides services to the upstream, and any node downtime does not affect the availability of the service.

So here's the problem:

How does the caller maintain the downstream service cluster configuration?

Does the caller perceive when adding or decreasing nodes in the service cluster?

Initial stage: "configure private possession" architecture

"configuration hiding" is the most initial stage of configuration, upstream calls downstream, each upstream has a dedicated private configuration file, recording the configuration information of each node downstream of the call.

As shown above:

User center user-service has three nodes of ip1/ip2/ip3

Service1 calls user Center, which has a dedicated configuration file s1.conf, in which the cluster configured with us is ip1/ip2/ip3

Service2 also calls user Center. Similarly, there is a configuration file s2.conf, which records that the us cluster is ip1/ip2/ip3.

Web2 also calls user Center, which is the same as w2.conf. Us cluster is configured as ip1/ip2/ip3.

Voiceover: is it familiar? This is how most companies play in the early days.

What are the disadvantages of the "configuration hiding" architecture?

Let's take a look at a demand for capacity change:

The operation and maintenance staff detected a decline in the hard disk performance of the ip1 node and informed R & D that the ip1 node would be offline in the future.

Due to the large-scale operation promotion activities on May 8, traffic will surge in the future, and R & D is going to add two nodes, ip4 and ip5.

What should I do at this time?

The person in charge of the user center is required to notify all upstream callers to modify the "hidden" configuration and restart the upstream to connect to the new cluster. After there is no traffic on the ip1, notify the OPS to take the ip1 node offline to complete the whole process of capacity reduction and expansion.

What's wrong with this scheme?

When the business complexity is high, the number of R & D is large, and the service dependence is more complex, it is not so simple.

Question 1: the caller is in pain, and you are the one who changes the capacity. Why am I the one who changed the configuration and restarted it? This is a typical "reverse dependency" architecture design, upstream and downstream are coupled through configuration, which is unreasonable.

Problem 2: the server is in pain. Ta does not know how many upstream calls it has, and often can only locate the upstream in the following ways:

Roar in the group

Send an email to inquire

Find the ip through the connection, ask the operator through ip, find the person in charge of the machine, and then find the corresponding calling service through the person in charge of the machine.

Voiceover: is it deja vu?

Either way, it is likely to be omitted, resulting in ip1 traffic has been difficult to offline, ip4/ip5 traffic is difficult to evenly migrate. How to optimize it?

Medium term: "Global configuration" architecture

The upgrade of the architecture is not achieved in one step, let's first solve the above problem of "modifying configuration restart" with the lowest cost.

"Global configuration" architecture: for general-purpose services, establish a global configuration file and eliminate configuration privacy:

Make specifications at the operation and maintenance level and create a new global configuration file, such as / opt/global.conf; voiceover: if there are too many configurations, pay attention to the vertical split of the configuration.

For the server, if it is a general service, the cluster information is configured in global.conf.

For callers, the caller forbids configuration concealment and must read the general downstream configuration from global.conf

What are the benefits of global configuration?

If the downstream capacity changes, only one configuration global.conf needs to be modified, not each upstream modification.

The next time the caller restarts, it automatically migrates to the expanded cluster.

The cost of modification is very small, the directory of reading configuration files has changed.

What are the shortcomings of global configuration?

If the caller does not restart all the time, there is no way to migrate the traffic to the new cluster.

Is there any way to achieve automatic traffic migration?

The answer is yes, you only need to introduce two uncomplicated components to automatically migrate the caller's traffic:

File monitoring component FileMonitor: the function is to monitor the changes of the file. A timer can be used to monitor the ModifyTime or md5 of the file on a regular basis, and callback can be implemented when the file changes.

Dynamic connection pooling component DynamicConnectionPool: the connection pooling component is a subcomponent of RPC-client that maintains connections to multiple RPC-server nodes. The so-called "dynamic connection pool" means that the connections in the connection pool can be dynamically increased and decreased.

Voiceover: mutual exclusion with locks is easy to achieve.

With the introduction of these two components:

Once the global configuration file changes, the file monitoring component implements a callback

If the dynamic connection pool component finds that some nodes are reduced in the configuration, the corresponding connection is destroyed dynamically. If some nodes are added, the connection is established dynamically, and the capacity increase and reduction of downstream nodes are completed automatically.

Final Edition: configuration Center Architectur

The "global configuration" architecture is a quick solution to the "modify configuration restart" problem, but it still does not solve the problem that the service provider "does not know how many upstream calls they have".

If you don't know how many upstream calls you have:

"limit the flow by the caller"

"draw a global schema dependency graph"

What should we do when such needs are difficult to achieve?

The "configuration center" architecture can be solved perfectly.

Comparing the architecture diagrams of Global configuration and configuration Center, you will find that the configuration has been upgraded from static files to dynamic services:

The whole configuration center subsystem is composed of zk, conf-center service, DB configuration storage and conf-web configuration background.

The configuration of all downstream services is set in the configuration center through the background

All upstream needs to pull configuration, need to register with configuration center, and pull downstream service configuration information (ip1/ip2/ip3)

When downstream services need to be expanded and reduced:

Set conf-web configuration backend to add ip4/ip5 and reduce ip1

The conf-center service pushes the changed configuration to callers who have registered to care about the relevant configuration

Combined with dynamic connection pool components, automatic capacity expansion and reduction are completed.

What are the benefits of the "configuration center" architecture?

The caller does not need to restart again

The server is well aware of the upstream dependencies from the configuration center, thus limiting the flow according to the caller.

It is easy to get global schema dependencies from the configuration center

Pain point 1 and pain point 2 are solved at the same time.

What are the shortcomings of the "configuration center" architecture?

First, the complexity of the system is relatively high.

Second, the reliability of the configuration center is required to be high, and there is a global connection in one place.

Thank you for reading this article carefully. I hope the article "Why do you need a configuration Center in the Internet Architecture" shared by the editor will be helpful to everyone? at the same time, I also hope that you will support and pay attention to the industry information channel. more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.