Shulou(Shulou.com)06/03 Report--

Author: Ah Fei, Senior Research and Development engineer of Infrastructure of Getui Application platform

In the micro-service architecture, due to the large number of micro-services and the mutual invocation relationship between services, a general distributed configuration management is essential. Generally speaking, configuration management needs to solve some problems, such as centralized configuration management, dynamic configuration during system operation, automatic refresh after configuration modification and so on.

In most microservice systems, there is a functional module called configuration file to provide unified distributed configuration management. It is of great significance to build a configuration center and uniformly manage each micro-service in the application.

Why Consul is suitable for configuration Management

Consul, as a lightweight distributed Kamp V storage system, is easy to build, high availability, and supports multiple data centers. Web UI is provided for KCompV management. In addition, Consul can also be combined with Consul-Template or introduce Consul Client-related dependencies into the code to create Watcher to real-time Watch Kamp V changes, which is the best choice for configuration management.

The following picture shows the overall design of a micro-push service system based on Consul configuration management. Among them, CCenter is the configuration center for secondary development on the basis of Consul.

Classification and organization of configuration under micro-service system

In practice, the configuration of different product lines will be placed under different paths of Consul to achieve isolation between different product line configurations.

According to the purpose of the configuration, configurations under the same product line can be divided into three categories:

Configuration related to 1.API Gateway

two。 Configuration related to service registration and discovery

3. Apply the relevant configuration.

Where each type of configuration corresponds to a different directory on the Consul.

According to the changing characteristics of the configuration, the configuration can be divided into two categories:

1. Environment-related global configuration

For example, MySQL and other external configurations depend on related configurations and other environment-related configurations. These configurations are different in the development and test production environment, and different values need to be configured for different environments.

two。 Configuration of the application itself

Generally speaking, it is not often changed, can be dynamically adjusted, and the configuration of the switch. This type of configuration is stable and will be changed only when needed after initialization, and default values are usually set. These two types of configurations are placed in different subdirectories on Consul. In this way, QA and OPS only need to pay attention to the differences in the environment.

Based on the above classification of configuration, the final format of Key on Consul is as follows:

/ ProductLine_Prefix/Usage_Prefix/Environmental_Correlation_Prefix/Config_Item_Path

Among them

ProductLine_Prefix: used to isolate the configuration of different product lines

Usage_Prefix: used to distinguish the purpose of the configuration

Environmental_Correlation_Prefix: used to separate environment-related configurations

Config_Item_Path: specific configuration items.

There are two forms of organization configured on Consul:

1. Organized in the form of a configuration file, a KUnip V on Consul corresponds to a configuration file, such as the configuration file of nginx.

two。 Organized in the form of configuration items, the configuration files are templated and divided into configuration items one by one, each configuration item corresponds to a K Consul V, and multiple configuration items correspond to a configuration file. Most of the configuration files themselves are organized in the form of Khand V and are suitable for templating. After templating, they can be divided into different categories on the Consul according to the characteristics of the configuration items.

How to implement configuration updates

How do you generate loadable applications, or usable configurations, for KUnip V on Consul?

1. The configuration update of micro-service implemented by Node and Lua is realized by Consul-Template.

two。 The configuration update of micro-service implemented with Java is realized through two ways: the Consul-Template tool (which requires restarting the application) and the dependency creation Watcher (hot update) that introduces Consul Client into the code.

How do I use Consul-Template?

Consul-Template is a background process, which can update any number of templates and generate corresponding files according to the changes of KUnip V on Watch Consul, and then run any command. To use Consul-Template, you generally need to define two files:

1. Template file

Template files are generally written in Go Template format. Examples are as follows:

Config-tree.ctmpl:

{{tree / consul/path/to/configFiles | explode | toJSONPretty}}

When the configuration of the template changes under the / consul/path/to/configFiles path, it will render a string in Json format, which contains all the K consul/path/to/configFiles V.

Config-kv.ctmpl:

Return {host=' {{printf "% s/mysql/host" (env "CONSUL_CONFIG_PREFIX") | key}', port= {{keyOrDefault (printf "% s/mysql/port" (env "CON-SUL_CONFIG_PREFIX")) "3306"}}, user=' {{printf "% s/mysql/user" (env "CONSUL_CONFIG_PREFIX") | key}}' Password=' {{printf "% s/mysql/password" (env "CON-SUL_CONFIG_PREFIX") | key}'}

The template is rendered according to the configuration item, in which Consul-Template is used to define two methods, key and keyOrDefault. Among them, key will create the corresponding K _ Consul V on the Consul and then render the template, while keyOrDefault will use the default value instead when there is no corresponding K _ pump V on the Consul.

The environment variable "CONSUL_CONFIG_PREFIX" is also used in the template, so that different product lines can use the same template file, only need to change the value of the environment variable "CONSUL_CONFIG_PREFIX".

two。 Configuration file

The configuration file is written according to HashiCorp Configuration Language (HCL), and the example is as follows:

Template {source = "config-tree.ctmpl", destination = "config-tree.json", command = "sh updateAndReload.sh config-tree.json"} template {source = "config-kv.ctmpl", destination = "config-kv.lua", command = "sh updateAndReload.sh config-kv.lua"}

The function of the configuration file is to use the two template files specified by "source" to render, save the rendered results in the file specified by "destination" respectively, and run the command specified by "command" respectively to update and load the configuration file.

How to update the configuration

In a push micro-service system, there are two ways to update the configuration:

1. Replace profile, reload service

two。 Call the service interface to update the configuration in memory directly

In the micro-service implemented by Java, the hot update configuration usually introduces the dependency of Consul Client into the code. When the application is started, a Watcher will be initialized to listen for the change of KUnip V in the corresponding directory on the Consul. When the relevant KUnip V changes, Watcher will pull it down, and then call the relevant code to update the configuration.

Secondary Development based on Consul-CCenter

Configuration Center CCenter provides a more friendly WEB UI on Consul and adds version control. Each update of the configuration will generate a version. After the version is applied, the configuration will really take effect. It is more convenient to compare the differences between configuration versions and apply any version of the configuration.

Summary

The above is a set of configuration management scheme based on Consul in micro-service practice. As a lightweight distributed KWeiV storage system, Consul is very suitable for configuration management and can help developers build configuration centers conveniently and quickly. Combined with Consul-Template, real-time configuration updates can be easily realized, secondary development on the basis of Consul, and effective control of configuration version. It plays a good auxiliary role in the configuration management of micro-services.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.