Shulou(Shulou.com)06/03 Report--

This article mainly explains "how to understand the open source configuration center Apollo of Ctrip Architecture Department". The explanation in the article is simple and clear and easy to learn and understand. please follow the editor's train of thought to study and learn "how to understand Ctrip Architecture's open source configuration center Apollo".

Why use configuration Center?

With the increasing complexity of program functions, the configuration of programs is increasing: the switches of various functions, the configuration of parameters, the address of the server.

The expectation of program configuration is getting higher and higher: real-time effective after configuration modification, grayscale release, sub-environment, sub-cluster management configuration, perfect authority, audit mechanism …...

In such a large environment, the traditional configuration files, databases and other ways have been increasingly unable to meet the needs of developers for configuration management.

We must all know that our projects are accompanied by a variety of configuration files, and there is always a set of local configuration files, one set of test server configuration, one set of formal service configuration, and sometimes accidentally correct mistakes, being scolded is a small thing, deducting performance can be a big deal.

And whenever the project is released, the configuration file will be packaged, that is, the configuration file will be released with the project. Then every time there is a problem that requires us to modify the configuration file, we always have to modify it locally and then redistribute it for the new configuration to take effect.

When the request pressure is increasing, your project will change from 1 node to multiple nodes. If the configuration needs to change, the corresponding modification operation is the same. You only need to modify it once in the project. However, the workload for publishing operation is much greater than before, because multiple nodes have to be released.

Modify these configurations, the increased release workload reduces the overall work efficiency, in order to improve work efficiency, the configuration center arises at the historic moment, we can store the configuration in the configuration center for management.

In general, before the introduction of the configuration center, we all face the following problems:

The configuration of the scattered format is not standard, some use properties format, some use xml format, and some save DB, the team tends to build their own wheels, do a variety of practices.

Mainly uses the local static configuration, the configuration modification is troublesome, the configuration modification generally needs to go through a long test release cycle. In the distributed micro-service environment, when there are many service instances, it is time-consuming and laborious to modify the configuration.

It is easy to cause production accidents. This is my personal experience. In an Internet company, a team brought the configuration of the test environment to production at the time of release, causing millions of capital loss accidents.

Configuration lack of security audit and version control functions, who changed the configuration? What did you change? When did you change it? There is no way to trace it back, and if something goes wrong, it will not be rolled back in time.

It increases the workload of the operation and maintenance brother, and greatly damages the basic feelings of the operation and maintenance brother and the development brother.

What exactly is a configuration center?

The configuration center is to put all kinds of configurations, parameters and switches in the project into a centralized place for unified management, and provide a set of standard interfaces. When each service needs to obtain the configuration, configure the interface pull of the center. When the various parameters in the configuration center are updated, each service can also be notified to synchronize the latest information in real time to make it dynamically updated.

Introduction to Apollo

Apollo (Apollo) is an open source configuration management center developed by Ctrip Framework Department, which can centrally manage the configuration of different environments and clusters, and can be pushed to the application side in real time after configuration modification, and has standardized permissions, process governance and other characteristics.

Apollo supports 4 dimensions to manage the configuration in Key-Value format (more on this below):

1. Application (application)

2. Environment (environment)

3. Cluster (cluster)

4. Namespace (namespace)

What is configuration?

Since Apollo is located in the configuration center, it is necessary to give a brief introduction to what configuration is.

As we understand it, the configuration has the following properties:

Configuration is a read-only variable independent of the program

Configuration is first of all independent of the program, and the same program will behave differently under different configurations.

Second, the configuration is read-only to the program, and the program changes its behavior by reading the configuration, but the program should not change the configuration.

Common configurations are: DB Connection Str, Thread Pool Size, Buffer Size, Request Timeout, Feature Switch, Server Urls and so on.

Configure the entire lifecycle that accompanies the application

Configuration runs through the entire life cycle of the application. The application initializes by reading the configuration at startup and adjusts the behavior according to the configuration at run time.

Configurations can be loaded in many ways

Configuration can also be loaded in many ways, such as internal hard code, configuration files, environment variables, startup parameters, database-based, etc.

Configuration needs governance

There is also a special type of configuration-framework-like component configuration, such as the configuration of CAT clients.

Although this kind of framework class components are developed and maintained by other teams, the runtime is within the actual business application, so it can be considered that the framework class components are also part of the application.

The configuration corresponding to this kind of components also needs to be managed in a relatively perfect way.

The same program often needs different configurations in different environments (development, testing, production) and different clusters (such as different data centers), so it needs to have perfect environment and cluster configuration management.

Because the configuration can change the behavior of the program, incorrect configuration can even cause disaster, so the modification of the configuration must have relatively perfect permission control.

Authority control

Configuration management of different environments and clusters

Configuration management of framework class components

Why use Apollo,Apollo? what are the characteristics?

Based on the particularity of configuration, Apollo is determined to become a configuration publishing platform with governance capabilities from the beginning of its design. Currently, it provides the following features:

Uniformly manage the configuration of different environments and clusters

Apollo provides a unified interface for centralized management of configurations for different environments (environment), different clusters (cluster), and different namespaces (namespace).

The same code is deployed in different clusters and can have different configurations, such as the address of zookeeper.

Namespaces (namespace) can easily support multiple different applications to share the same configuration, while allowing applications to overwrite the shared configuration

Configuration changes take effect in real time (hot release)

After the user modifies the configuration and publishes it in Apollo, the client can receive the latest configuration in real time (1 second) and notify the application.

Version release management

All configuration releases have a version concept, so that configuration rollback can be easily supported.

Grayscale release

Supported grayscale publishing of configuration, for example, after clicking release, it only takes effect on some application instances, and then pushes it to all application instances after observing it for a period of time.

Rights management, release audit, operation audit

The management of application and configuration has a perfect authority management mechanism, and the management of configuration is divided into two links: editing and release, so as to reduce human errors.

All operations have audit logs, which can easily track problems.

Client configuration information monitoring

You can easily see which instances the configuration is being used on the interface.

Provide Java and .net native clients

Provides native clients of Java and .net to facilitate application integration

ConfigurationProperties that supports Spring Placeholder, Annotation and Spring Boot for ease of use (requires Spring 3.1.1 +)

At the same time, it provides Http interface, which can be easily used by non-Java and .net applications.

Provide an open platform API

Apollo itself provides a relatively complete unified configuration management interface, supporting multi-environment, multi-data center configuration management, permissions, process governance and other features. However, for the sake of generality, Apollo will not make too many restrictions on the modification of the configuration. It can be saved as long as it conforms to the basic format, and will not be checked specifically for different configuration values, such as database user name, password, Redis service address, etc.

For this kind of application configuration, Apollo supports the application side to modify and publish the configuration in Apollo through the open platform API, and has perfect authorization and permission control.

Simple deployment

The configuration center, as a basic service, requires very high availability, which requires Apollo to rely on as little external dependence as possible.

At present, the only external dependency is MySQL, so the deployment is very simple. As long as you install Java and MySQL, you can make Apollo run.

Apollo also provides packaging scripts that generate all the required installation packages with one click and support custom runtime parameters.

The overall architecture of Apollo

First, let's take a look at the basic workflow of Applo as shown in the following figure

1. The user modifies and publishes the configuration in the configuration center

two。 The configuration center notifies the Apollo client that there are configuration updates

The 3.Apollo client pulls the latest configuration from the configuration center, updates the local configuration, and notifies the application

Next, let's take a look at the overall architecture of Apollo.

The figure above briefly describes the overall design of Apollo, which we can look at from the bottom up:

Config Service provides configured read, push and other functions, and the service object is the Apollo client

Admin Service provides functions such as configuration modification and release, and the service object is Apollo Portal (management interface).

Both Config Service and Admin Service are multi-instance, stateless deployments, so you need to register yourself with Eureka and keep your heart beating

We set up a layer of Meta Server service discovery interface on top of Eureka to encapsulate Eureka.

Client obtains the list of Config Service services (IP+Port) by accessing Meta Server through the domain name, and then accesses the service directly through IP+Port. At the same time, load balance will be done on the client side and the error will be retried.

Portal obtains the list of Admin Service services (IP+Port) by accessing Meta Server through the domain name, and then accesses the service directly through IP+Port. At the same time, load balance will be done on the Portal side and error retry will be made.

To simplify deployment, we will actually deploy the three logical roles Config Service, Eureka and Meta Server in the same JVM process

Why Eureka

Why do we use Eureka as the service registry instead of traditional zk and etcd? I have roughly summed up the following reasons:

It provides a complete implementation of Service Registry and Service Discovery

First of all, it provides a complete implementation, and it also withstands the test of Netflix's own production environment, which is relatively easy to use.

Seamless integration with Spring Cloud

The project itself uses Spring Cloud and Spring Boot, and Spring Cloud also has a very complete set of open source code to integrate Eureka, so it is very easy to use.

In addition, Eureka also supports launching in our application's own container, which means that after our application is launched, it acts as both an Eureka and a service provider. This greatly improves the availability of the service.

This is the main reason why we choose Eureka instead of zk, etcd, etc. In order to improve the availability of the configuration center and reduce the complexity of deployment, we need to reduce external dependencies as much as possible.

Open Source

The last point is open source, because the code is open source, so it is very easy for us to understand its implementation principle and troubleshooting problems.

Each module provides an overview of Config Service

Provide configuration acquisition interface

Provides configuration update push interface (based on Http long polling)

The server uses Spring DeferredResult to achieve asynchronization, thus greatly increasing the number of long-standing connections

Currently, the default configuration of tomcat embed is a maximum of 10000 connections (which can be adjusted). Virtual machines using 4C8G are tested-they can support 10000 connections, so it meets the demand (only one persistent connection will be initiated by an application instance). The interface service object is the Apollo client.

Admin Service provides configuration management interface

Provide interfaces for configuration modification, release, etc.

The service object of the API is Portal

Meta Server

Portal gets the list of Admin Service services (IP+Port) by accessing Meta Server through the domain name

Client gets the list of Config Service services (IP+Port) by accessing Meta Server through the domain name

Meta Server obtains the service information of Config Service and Admin Service from Eureka, which is equivalent to an Eureka Client.

The main purpose of adding a Meta Server role is to encapsulate the details of service discovery, and for Portal and Client, always through a

The Http interface acquires the service information of Admin Service and Config Service, regardless of the actual service registration and discovery components behind it

Meta Server is just a logical role, and Config Service and IP are in the same JVM process at deployment time, so IP, port and Config Service are consistent.

Eureka

Provide service registration and discovery based on Eureka and Spring Cloud Netflix

Config Service and Admin Service register for services with Eureka and keep their heartbeats

For simplicity, Eureka is currently deployed in the same JVM process as Config Service (via Spring Cloud Netflix)

Portal

Provide Web interface for users to manage and configure

Get the list of Admin Service services (IP+Port) through Meta Server and access the services through IP+Port

Do load balance on the Portal side and try again in error

Client

The client program provided by Apollo provides functions such as configuration acquisition and real-time update for applications.

Get the list of Config Service services (IP+Port) through Meta Server and access the services through IP+Port

Do load balance on the Client side and try again in error

Introduction to the core concepts of Apollo

1 、 application

1. The Apollo client needs to know who the current application is at run time, so that the configuration of the corresponding application can be obtained according to different applications.

2. Each application needs to have a unique identity. You can configure the app.id parameter in the code to identify the current application, and Apollo will identify the current application according to this.

2 、 environment

In actual development, our applications are often deployed in different environments, which are generally divided into different environments, such as development, testing, production, and so on. The configuration in different environments is also different, which is provided by default in Apollo.

Four environments:

FAT: functional test environment

UAT: integrated test environment

DEV: development environment

PRO: production environment

If you want to specify which environment to use in the program, you can configure the value of the variable env to the corresponding environment name.

3 、 cluster

1. Grouping different instances under an application, for example, according to the data center, the application instance of the computer room in Shanghai is divided into one cluster, and the application instance of the computer room in Beijing is divided into another cluster.

2. For different clusters, the same configuration can have different values. For example, two clusters are set in the two computer rooms in Beijing and Shanghai mentioned above, all of which have mysql configuration parameters, in which the address configured in the parameters is different.

4 、 namespace

The grouping of different configurations in an application can simply compare namespace to different configuration files, and different types of configurations are stored in different files, such as database configuration files, RPC configuration files and so on.

Anyone familiar with SpringBoot knows that every SpringBoot project has a default configuration file, application.yml. If you still want to use multiple configurations, you can create multiple configuration files to store different configuration information through the

Specify the spring.profiles.active parameter to specify that a different profile is applied. The concept of namespace here is similar, putting different configurations into different configuration namespace.

Namespace is divided into two permissions, which are:

Public (public): the Namespace of public permission, which can be obtained by any application.

Private (proprietary): can only be obtained by the application to which it belongs. The Namespace,Apollo of an application that tries to get another application private will report a "404" exception.

Apollo real-time release configuration 1. Real-time push design after configuration release

One of the most important features of the configuration center is real-time push, and because of this feature, we can rely on the configuration center to do a lot of things. As shown in the figure.

Figure 1 briefly describes the general process of configuration publishing.

Users edit and publish configurations in Portal.

Portal will call the interface provided by Admin Service for publishing operation.

After the Admin Service receives the request, it sends a ReleaseMessage to each Config Service to notify the Config Service that the configuration has changed.

After receiving the ReleaseMessage, Config Service notifies the corresponding client, which is implemented based on Http persistent connection.

two。 The implementation of sending ReleaseMessage

ReleaseMessage messages implement a simple message queue through Mysql. The reason for not adopting message middleware is to make Apollo deployment as simple as possible and reduce external dependencies as much as possible, as shown in the figure.

The figure above briefly describes the process of sending ReleaseMessage:

Admin Service inserts a message record into the ReleaseMessage table after the configuration is published.

Config Service starts a thread that periodically scans the ReleaseMessage table to see if there are any new message records.

When Config Service finds a new message record, it notifies all message listeners.

When the message listener gets the information about the configuration publication, it notifies the corresponding client.

3. The implementation of Config Service notification client

Notification is implemented based on Http persistent connection, which is mainly divided into the following steps:

The client initiates a Http request to the notifications/v2 interface of the Config Service.

The notifications/v2 interface suspends the request through Spring DeferredResult and does not return immediately.

If no configuration concerned by the client is released within 60s, the Http status code 304 will be returned to the client.

If a change in the configuration is found, the setResult method of DeferredResult is called, passing in the namespace information with the configuration change, and the request is returned immediately.

After the client obtains the namespace of the configuration change from the returned result, it immediately requests Config Service to get the latest configuration of the namespace.

4. Real-time push Design for Source Code Analysis

Apollo push involves a lot of code, this tutorial will not do a detailed analysis, the author pushed here under the code slightly simplified to explain to you, so that it will be easier to understand.

Of course, the code is relatively simple, many details will not be considered, just to make people understand the core principles of Apollo push.

The logic of sending ReleaseMessage is that we write a simple interface, store it in queue, and call this interface to simulate configuration updates and send ReleaseMessage messages when testing. The specific code is shown below.

@ RestControllerpublic class NotificationControllerV2 implements ReleaseMessageListener {/ / simulate configuration updates, insert data into it to indicate that there is an update public static Queue queue = new LinkedBlockingDeque (); @ GetMapping ("/ addMsg") public String addMsg () {queue.add ("xxx"); return "success";}}

After the message is sent, according to the Config Service mentioned earlier, a thread will start to scan the ReleaseMessage table regularly to see if there is a new message record, and then notify the client. Here, we will also start a thread to scan, as shown below.

@ Componentpublic class ReleaseMessageScanner implements InitializingBean {@ Autowired private NotificationControllerV2 configController; @ Override public void afterPropertiesSet () throws Exception {/ / scheduled task scans the database for new configuration releases new Thread (()-> {for (;;) {String result = NotificationControllerV2.queue.poll (); if (result! = null) {ReleaseMessage message = new ReleaseMessage (); message.setMessage (result); configController.handleMessage (message)) }) .start ();}}

Read the queue in NotificationControllerV2 in a loop, construct a Release-Message object if there is a message, and then call the handleMessage () method in NotificationControllerV2 to process the message.

ReleaseMessage simulates the content of the message in one field, as shown below.

Public class ReleaseMessage {private String message; public void setMessage (String message) {this.message = message;} public String getMessage () {return message;}}

Next, let's take a look at what handleMessage has done.

NotificationControllerV2 implements the ReleaseMessageListener interface, and the handleMessage () method is defined in ReleaseMessageListener, as shown below.

Public interface ReleaseMessageListener {void handleMessage (ReleaseMessage message);}

HandleMessage is the message listener that sends notifications when the configuration changes. After receiving the information of the configuration release, the message listener will notify the corresponding client, as shown in the following code.

RestControllerpublic class NotificationControllerV2 implements ReleaseMessageListener {private final Multimap deferredResults = Multimaps.synchronizedSetMultimap (HashMultimap.create ()); @ Override public void handleMessage (ReleaseMessage message) {System.err.println ("handleMessage:" + message); List results = Lists.newArrayList (deferredResults.get ("xxxx")); for (DeferredResultWrapper deferredResultWrapper: results) {List list = newArrayList (); list.add (new ApolloConfigNotification ("application", 1)); deferredResultWrapper.setResult (list);}

The real-time push of Apollo is based on Spring DeferredResult. In the handleMessage () method, you can see that the Multimap,Key in the first line of DeferredResult,deferredResults obtained through deferredResults is actually the message content. Value is the business wrapper class DeferredResultWrapper of DeferredResult. Let's take a look at the DeferredResultWrapper code, as shown below.

Public class DeferredResultWrapper {private static final long TIMEOUT = 60 * 1000bot / 60 seconds private static final ResponseEntity NOT_MODIFIED_RESPONSE_LIST = new ResponseEntity (HttpStatus.NOT_MODIFIED); private DeferredResult result; public DeferredResultWrapper () {result = new DeferredResult (TIMEOUT, NOT_MODIFIED_RESPONSE_LIST);} public void onTimeout (Runnable timeoutCallback) {result.onTimeout (timeoutCallback);} public void onCompletion (Runnable completionCallback) {result.onCompletion (completionCallback) } public void setResult (ApolloConfigNotification notification) {setResult (Lists.newArrayList (notification));} public void setResult (List notifications) {result.setResult (new ResponseEntity (notifications, HttpStatus.OK));} public DeferredResult getResult () {return result;}}

Set the result to be returned to the client through the setResult () method. This is the principle that when the configuration changes and then notifies the client through the message listener, when is the client connected? The specific code is as follows.

@ RestControllerpublic class NotificationControllerV2 implements ReleaseMessageListener {/ / simulate configuration updates and insert data into them to indicate that there are updates public static Queue queue = new LinkedBlockingDeque (); private final Multimap deferredResults = Multimaps.synchronizedSetMultimap (HashMultimap.create ()); @ GetMapping ("/ getConfig") public DeferredResult getConfig () {DeferredResultWrapper deferredResultWrapper = new DeferredResultWrapper (); List newNotifications = getApolloConfigNotifications (); if (! CollectionUtils.isEmpty (newNotifications)) {deferredResultWrapper.setResult (newNotifications) } else {deferredResultWrapper.onTimeout (()-> {System.err.println ("onTimeout");}); deferredResultWrapper.onCompletion (()-> {System.err.println ("onCompletion");}); deferredResults.put ("xxxx", deferredResultWrapper);} return deferredResultWrapper.getResult ();} private List getApolloConfigNotifications () {List list = new ArrayList (); String result = queue.poll () If (result! = null) {list.add (new ApolloConfigNotification ("application", 1));} return list;}}

A / getConfig interface is provided in NotificationControllerV2, which will be called by the client when starting. At this time, the getApolloConfigNotifications () method will be executed to obtain the change information of whether the configuration has been configured. If it is proved that the configuration has been modified, the result will be returned directly to the client through deferredResultWrapper.setResult (newNotifications). After receiving the result, the client will pull the configuration information again to overwrite the local configuration.

If the getApolloConfigNotifications () method does not return information about the configuration modification, which proves that the configuration has not changed, then add the DeferredResultWrapper object to the deferredResults and wait for the message listener to notify you when the subsequent configuration changes.

At the same time, the request will be suspended and will not be returned immediately, which is achieved through the following code in DeferredResultWrapper, as shown below.

Private static final long TIMEOUT = 60 * 1000; / / 60 secondsprivate static final ResponseEntity NOT_MODIFIED_RESPONSE_LIST = new ResponseEntity (HttpStatus.NOT_MODIFIED); private DeferredResult result;public DeferredResultWrapper () {result = new DeferredResult (TIMEOUT, NOT_MODIFIED_RESPONSE_LIST);}

The timeout time and the response code returned after the timeout are specified when the DeferredResult object is created. If there is no message listener to notify within 60 seconds, the request will time out. After the timeout, the response code received by the client is 304.

The whole Config Service process is over, and then let's take a look at how the client is implemented. We simply write a test class to simulate the client registration, as shown below.

Public class ClientTest {public static void main (String [] args) {reg ();} private static void reg () {System.err.println ("registration"); String result = request ("http://localhost:8081/getConfig"); if (result! = null) {/ / configuration update, pull configuration / / again. } / / re-register reg ();} private static String request (String url) {HttpURLConnection connection = null; BufferedReader reader = null; try {URL getUrl = new URL (url); connection = (HttpURLConnection) getUrl.openConnection (); connection.setReadTimeout (90000); connection.setConnectTimeout (3000) Connection.setRequestMethod ("GET"); connection.setRequestProperty ("Accept-Charset", "utf-8"); connection.setRequestProperty ("Content-Type", "application/json"); connection.setRequestProperty ("Charset", "UTF-8"); System.out.println (connection.getResponseCode ()) If (200 = = connection.getResponseCode ()) {reader = new BufferedReader (new InputStreamReader (connection.getInputStream (), "UTF-8"); StringBuilder result = new StringBuilder (); String line = null; while ((line = reader.readLine ())! = null) {result.append (line) } System.out.println ("result" + result); return result.toString ();} catch (IOException e) {e.printStackTrace ();} finally {if (connection! = null) {connection.disconnect () }} return null;}}

First start the service where the / getConfig interface is located, and then start the client, and then the client will initiate a registration request. If any modification directly obtains the result, the configuration update operation will be carried out. If there is no modification, the request will be suspended. Here, the read timeout set by the client is 90s, which is greater than the server's 60s timeout.

Every time after receiving the result, no matter with or without modification, it must be re-registered. In this way, the real-time push can be configured.

We can call the / addMsg interface written earlier to simulate a change in the configuration, and the client can get the result immediately after the call.

Apollo client design

The figure above briefly describes the implementation principle of the Apollo client:

The client and the server maintain a long connection so that they can get the push of the configuration update the first time. (implemented through Http Long Polling)

The client also periodically pulls the latest configuration of the application from the Apollo configuration center server.

This is a fallback mechanism to prevent the configuration from being updated due to the failure of the push mechanism

Regular pull by the client will report the local version, so in general, the server will return 304-Not Modified for the scheduled pull operation.

The timing frequency defaults to pulling every 5 minutes, and the client can also override it by specifying System Property: apollo.refreshInterval at run time, in minutes.

After the client obtains the latest configuration of the application from the Apollo configuration center server, it will save it in memory.

The client will cache a copy of the configuration obtained from the server in the local file system (it can still restore the configuration locally if the service is not available or the network is not available)

Applications can get the latest configuration and subscribe to configuration update notifications from the Apollo client

Apollo client usage

Apollo supports API and Spring integration. How do you choose which one to use?

The API mode is flexible, the function is complete, the configuration value is updated in real time (hot release), and supports all Java environments.

Spring is easy to access, and there are N cool ways to play with Spring, such as

Directly used in the code, such as @ Value ("${someKeyFromApollo:someDefaultValue}")

Replace placeholder is used in the configuration file, such as: spring.datasource.url: ${someKeyFromApollo:someDefaultValue}

Directly host the configuration of spring, such as configuring spring.datasource.url=jdbc:mysql://localhost:3306/somedb?characterEncoding=utf8 directly in apollo

Placeholder mode:

@ ConfigurationProperties mode of Spring boot

Versions starting with v0.10.0 support automatic updates of placeholder at run time, as described in PR # 972. (versions prior to v0.10.0 will not be re-injected after configuration changes, and will not be updated until rebooted. If you need to update the configuration values in real time, please refer to the instructions for the following 3.2.2 Spring Placeholder)

Spring can also be used in combination with API. For example, if you inject a Config object into Apollo, you can obtain the configuration through API as usual:

@ ApolloConfigprivate Config config; / / inject config for namespace application

For more interesting practical usage scenarios and sample code, please refer to apollo-use-cases

1. How to use API

The API approach is the simplest and most efficient way to use Apollo configuration and can be used without relying on the Spring framework.

Get the default namespace configuration (application)

Config config = ConfigService.getAppConfig (); / / config instance is singleton for each namespace and is never nullString someKey = "someKeyFromDefaultNamespace"; String someDefaultValue = "someDefaultValueForTheKey"; String value = config.getProperty (someKey, someDefaultValue)

The latest real-time configuration values corresponding to someKey can be obtained through the above config.getProperty.

In addition, the configuration value is obtained from memory, so there is no need for the application to do its own cache.

Listen for configuration change events

Listen for configuration change events only when the application really cares about the configuration change and needs to be notified of the configuration change, for example, the connection needs to be rebuilt after the database connection string changes.

If you just want to get the latest configuration every time, just call config.getProperty as shown in the above example.

Config config = ConfigService.getAppConfig (); / / config instance is singleton for each namespace and is never nullconfig.addChangeListener (new ConfigChangeListener () {@ Override public void onChange (ConfigChangeEvent changeEvent) {System.out.println ("Changes for namespace" + changeEvent.getNamespace ()); for (String key: changeEvent.changedKeys ()) {ConfigChange change = changeEvent.getChange (key) System.out.println (String.format ("Found change-key:% s, oldValue:% s, newValue:% s, changeType:% s", change.getPropertyName (), change.getOldValue (), change.getNewValue (), change.getChangeType ());}})

Get the configuration of the public Namespace

String somePublicNamespace = "CAT"; Config config = ConfigService.getConfig (somePublicNamespace); / / config instance is singleton for each namespace and is never nullString someKey = "someKeyFromPublicNamespace"; String someDefaultValue = "someDefaultValueForTheKey"; String value = config.getProperty (someKey, someDefaultValue)

Get the configuration of namespace in non-properties format

Namespace in 1.yaml/yml format

Apollo-client version 1.3.0 has begun to provide better support for yaml/yml, which is consistent with the properties format.

Config config = ConfigService.getConfig ("application.yml"); String someKey = "someKeyFromYmlNamespace"; String someDefaultValue = "someDefaultValueForTheKey"; String value = config.getProperty (someKey, someDefaultValue); 2. Namespace in non-yaml/yml format

You need to use the ConfigService.getConfigFile interface and specify a Format, such as ConfigFileFormat.XML, when getting it.

String someNamespace = "test"; ConfigFile configFile = ConfigService.getConfigFile ("test", ConfigFileFormat.XML); String content = configFile.getContent (); Spring integration mode

Configuration

Apollo also supports integration with Spring (Spring 3.1.1 +), and you only need to do some simple configuration.

Apollo currently supports both traditional XML-based configurations and popular Java-based configurations.

For Spring Boot environment, it is recommended to refer to 3.2.1.3 Spring Boot integration mode (recommended) configuration.

It is important to note that if you have previously used org.springframework.beans.factory.config.PropertyPlaceholderConfigurer, replace it with org.springframework.context.support.PropertySourcesPlaceholderConfigurer. PropertyPlaceholderConfigurer is not recommended after Spring 3.1. instead, PropertySourcesPlaceholderConfigurer should be used.

If you have used it before, please note that the spring-context.xsd version introduced in xml needs to be more than 3.1 (usually it will be upgraded automatically as long as there is no specified version). It is recommended to introduce it without version number, such as http://www.springframework.org/schema/context/spring-context.xsd.

Note namespace in 1:yaml/yml format supports integration with Spring since version 1.3.0. A full name with a suffix, such as application.yml, is required for injection.

Note 2: namespace in non-properties and non-yaml/yml formats (such as xml,json, etc.) does not support integration with Spring.

XML-based configuration

Note: apollo-related xml namespace needs to be added to the configuration file header, otherwise xml syntax errors will be reported.

1. Inject the default namespace configuration into Spring

two。 Inject multiple namespace configurations into Spring

3. Inject multiple namespace and specify the order

The configuration of Spring is ordered, and if multiple property source have the same key, then the configuration that comes first takes effect in the end.

If apollo:config does not specify order, the default is the lowest priority.

Java-based configuration (recommended)

Compared with XML-based configuration, Java-based configuration is more popular at present.

Note that @ EnableApolloConfig should be used with @ Configuration, otherwise it will not take effect.

1. Inject the default namespace configuration into Spring

/ / this is the simplest configuration form, which is fine for general applications. It is used to instruct the configuration of Apollo injection application namespace into the Spring environment @ Configuration@EnableApolloConfigpublic class AppConfig {@ Bean public TestJavaConfigBean javaConfigBean () {return new TestJavaConfigBean ();}}.

two。 Inject multiple namespace configurations into Spring

@ Configuration@EnableApolloConfigpublic class SomeAppConfig {@ Bean public TestJavaConfigBean javaConfigBean () {return new TestJavaConfigBean ()}} / / this is a slightly more complex configuration, instructing Apollo to inject the configuration of FX.apollo and application.yml namespace into the Spring environment @ Configuration@EnableApolloConfig ({"FX.apollo", "application.yml"}) public class AnotherAppConfig {}

3. Inject multiple namespace and specify the order

/ / this is the most complex configuration form, instructing Apollo to inject the configuration of FX.apollo and application.yml namespace into the Spring environment in the order @ Configuration@EnableApolloConfig (order = 2) public class SomeAppConfig {@ Bean public TestJavaConfigBean javaConfigBean () {return new TestJavaConfigBean ();}} @ Configuration@EnableApolloConfig (value = {"FX.apollo", "application.yml"}, order = 1) public class AnotherAppConfig {} Spring Boot integration (recommended)

In addition to the above two integration methods, Spring Boot also supports configuration through application.properties/bootstrap.properties, which enables configuration injection at an earlier stage, such as the scenario where @ ConditionalOnProperty is used or some spring-boot-starter needs to read the configuration and do something (such as dubbo-spring-boot-project) during the startup phase. Therefore, it is recommended to access Apollo in the Spring Boot environment by the following methods (version 0.10.0 or above is required).

It is easy to use, as long as you configure it in application.properties/bootstrap.properties as shown in the following example.

Configuration example of injecting default application namespace

# will inject 'application' namespace in bootstrap phase apollo.bootstrap.enabled = true

Example of a configuration that injects non-default application namespace or multiple namespace

Apollo.bootstrap.enabled = true # will inject 'application',' FX.apollo' and 'application.yml' namespaces in bootstrap phase apollo.bootstrap.namespaces = application,FX.apollo,application.yml

Load the Apollo configuration before initializing the logging system (1.2.0 +)

Starting with version 1.2.0, if you want to put log-related configurations (such as parameters in logging.level.root=info or logback-spring.xml) in Apollo management, you can configure additional apollo.bootstrap.eagerLoad.enabled=true to put the load order of Apollo before the log system loads, but this will cause the Apollo startup process not to be output through logging (because the log system is not ready at all when Apollo loading is performed! So there is nothing to use the log output of Slf4j in Apollo code. For more information, please refer to PR 1614. Reference configuration examples are as follows:

# will inject 'application' namespace in bootstrap phase apollo.bootstrap.enabled = use of true # put apollo initialization before logging system initialization apollo.bootstrap.eagerLoad.enabled=trueSpring Placeholder

Spring applications typically use Placeholder to inject configurations in a format such as ${someKey:someDefaultValue}, such as ${timeout:100}. Before the colon is key, and after the colon is the default value.

It is recommended to give default values as much as possible in actual use to avoid run-time errors due to the lack of definition of key.

Versions starting with v0.10.0 support automatic updates of placeholder at run time, as described in PR # 972.

If you need to turn off the automatic update feature of placeholder at run time, you can turn it off in two ways:

1. By setting System Property apollo.autoUpdateInjectedSpringProperties, such as passing in-Dapollo.autoUpdateInjectedSpringProperties=false at startup

two。 By setting the apollo.autoUpdateInjectedSpringProperties property in META-INF/app.properties, such as

How app.id=SampleAppapollo.autoUpdateInjectedSpringProperties=falseXML is used

Suppose I have a TestXmlBean that has two configuration items to inject:

Public class TestXmlBean {private int timeout; private int batch; public void setTimeout (int timeout) {this.timeout = timeout;} public void setBatch (int batch) {this.batch = batch;} public int getTimeout () {return timeout;} public int getBatch () {return batch;}}

Then, I will define it in XML as follows (assuming that there are configuration items for timeout and batch in the application default application namespace):

How Java Config is used

Suppose I have a TestJavaConfigBean, and I can also use @ Value to inject it through JavaConfig:

Public class TestJavaConfigBean {@ Value ("${timeout:100}") private int timeout; private int batch; @ Value ("${batch:200}") public void setBatch (int batch) {this.batch = batch;} public int getTimeout () {return timeout;} public int getBatch () {return batch;}}

Used in the Configuration class as follows (assuming there are configuration items for timeout and batch in the application default application namespace):

@ Configuration@EnableApolloConfigpublic class AppConfig {@ Bean public TestJavaConfigBean javaConfigBean () {return new TestJavaConfigBean ();}} ConfigurationProperties usage

Spring Boot provides @ ConfigurationProperties to inject configuration into the bean object.

Apollo also supports this approach, and the following example injects redis.cache.expireSeconds and redis.cache.commandTimeout into the expireSeconds and commandTimeout fields of SampleRedisConfig, respectively.

@ ConfigurationProperties (prefix = "redis.cache") public class SampleRedisConfig {private int expireSeconds; private int commandTimeout; public void setExpireSeconds (int expireSeconds) {this.expireSeconds = expireSeconds;} public void setCommandTimeout (int commandTimeout) {this.commandTimeout = commandTimeout;}}

Used in the Configuration class as follows (assuming there are configuration items for redis.cache.expireSeconds and redis.cache.commandTimeout in the application default application namespace):

@ Configuration@EnableApolloConfigpublic class AppConfig {@ Bean public SampleRedisConfig sampleRedisConfig () {return new SampleRedisConfig ();}}

It is important to note that @ ConfigurationProperties needs to be used in conjunction with EnvironmentChangeEvent or RefreshScope if you need to automatically update the injected value when the Apollo configuration changes. For related code implementation, you can refer to ZuulPropertiesRefresher.java in apollo-use-cases project and SampleRedisConfig.java and SpringBootApolloRefreshConfig.java in apollo-demo project.

Spring Annotation support

Apollo also adds several new Annotation to simplify its use in Spring environments.

@ ApolloConfig

Used to automatically inject Config objects

@ ApolloConfigChangeListener

Used to automatically register ConfigChangeListener

@ ApolloJsonValue

Used to automatically inject the configured json string as an object

Examples of use are as follows:

Public class TestApolloAnnotationBean {@ ApolloConfig private Config config; / / inject config for namespace application @ ApolloConfig ("application") private Config anotherConfig; / / inject config for namespace application @ ApolloConfig ("FX.apollo") private Config yetAnotherConfig; / / inject config for namespace FX.apollo @ ApolloConfig ("application.yml") private Config ymlConfig / / inject config for namespace application.yml / * ApolloJsonValue annotated on fields example, the default value is specified as empty list-[] * * jsonBeanProperty= [{"someString": "hello", "someInt": 100}, {"someString": "world!", "someInt": 200}] * / @ ApolloJsonValue ("${jsonBeanProperty: []}") private List anotherJsonBeans; @ Value ("${batch:100}") private int batch / config change listener for namespace application @ ApolloConfigChangeListener private void someOnChange (ConfigChangeEvent changeEvent) {/ / update injected value of batch if it is changed in Apollo if (changeEvent.isChanged ("batch")) {batch = config.getIntProperty ("batch", 100) }} / / config change listener for namespace application @ ApolloConfigChangeListener ("application") private void anotherOnChange (ConfigChangeEvent changeEvent) {/ / do something} / / config change listener for namespaces application, FX.apollo and application.yml @ ApolloConfigChangeListener ({"application", "FX.apollo" "application.yml"}) private void yetAnotherOnChange (ConfigChangeEvent changeEvent) {/ / do something} / / example of getting config from Apollo directly / / this will always return the latest value of timeout public int getTimeout () {return config.getIntProperty ("timeout") } / / example of getting config from injected value / / the program needs to update the injected value when batch is changed in Apollo using @ ApolloConfigChangeListener shown above public int getBatch () {return this.batch;} private static class JsonBean {private String someString; private int someInt;}}

Used in the Configuration class as follows:

@ Configuration@EnableApolloConfigpublic class AppConfig {@ Bean public TestApolloAnnotationBean testApolloAnnotationBean () {return new TestApolloAnnotationBean ();}} already has configuration migration

In many cases, applications may already have a lot of configurations, such as Spring Boot applications, there will be bootstrap.properties/yml, application.properties/yml and other configurations.

After the application is connected to Apollo, these configurations can be easily migrated to Apollo. The specific steps are as follows:

Create a new project for the application in Apollo

Configure META-INF/app.properties in the application

It is recommended to convert the original configuration to properties format, and then paste it to the application namespace of the application through the text editing mode provided by Apollo, and publish the configuration.

If the original format is yml, you can use YamlPropertiesFactoryBean.getObject to convert to properties format

If you used to be yml and want to continue to use yml to edit the configuration, you can create a private application.yml namespace, paste all the original configuration into it, and publish the configuration

Apollo-client is required for version 1.3.0 or above

Delete previous configuration files such as bootstrap.properties/yml and application.properties/yml from the project

If you need to keep the local configuration file, you should note that some configurations such as server.port must ensure that the local file has deleted the configuration item.

Such as:

High availability design for spring.application.name = reservation-serviceserver.port = 8080logging.level = ERROReureka.client.serviceUrl.defaultZone = http://127.0.0.1:8761/eureka/eureka.client.healthcheck.enabled = trueeureka.client.registerWithEureka = trueeureka.client.fetchRegistry = trueeureka.client.eurekaServiceUrlPollIntervalSeconds = 60eureka.instance.preferIpAddress = trueApollo

High availability is one of the factors that must be considered in the design of distributed system architecture. it usually refers to reducing the time when the system can not provide services through design.

Apollo has put a lot of effort into high-availability design, so let's take a brief analysis:

1) some Config Service is offline.

No impact, Config Service can deploy multiple nodes.

2) all Config Service offline

All Config Service offline will affect the use of the client and cannot read the latest configuration. You can make the transition by reading a locally cached configuration file.

3) some Admin Service is offline.

No impact, Admin Service can deploy multiple nodes.

4) all Admin Service offline

Admin Service serves Portal. After all Admin Service goes offline, it will only affect the operation of Portal, not the client, which relies on Config Service.

5) some Portal is offline.

Multiple Portal can be deployed, and the load can be done through Nginx. The usage of a certain server will not be affected after it goes offline.

6) all Portal offline

There is no effect on the client reading the configuration, but it cannot be viewed or modified through Portal.

7) Database downtime

When the configured database goes down, it has no impact on the client, but it will cause the configuration to not be updated in Portal. When the client restarts, if you need to pull the configuration again, it will have an impact. You can use the option of enabling configuration cache to avoid the impact of database downtime.

Thank you for your reading, the above is the content of "how to understand the open source configuration center Apollo of Ctrip Architecture Department". After the study of this article, I believe you have a deeper understanding of how to understand the open source configuration center Apollo of Ctrip Architecture Department, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.