Shulou(Shulou.com)05/31 Report--

This article mainly introduces from the Nacos client perspective to see what the configuration center implementation principle is, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand.

Today, let's take a look at the principle of configuration center implementation from the perspective of Nacos client. When collating this article, we also refer to the blogs of some big names. Thank you here.

Before I begin to read the article, I would like to explain some ideas according to my understanding, so as to make it easier for everyone to sort out their ideas more quickly. Please criticize and correct what is wrong.

The Nacos client will cache the server configuration files locally to prevent the server from collapsing, causing the service to become unavailable.

The embodiment of the local cache class in the code is the CacheData mentioned below. We know that a configuration of the corresponding server can certainly be used by multiple clients at the same time. When this configuration changes, how to notify each client?

After the client starts, go back to register the monitor, and the monitor will eventually be saved to CopyOnWriteArrayList in the CacheData class

Listeners field, then, conversely, when the monitor callback method is executed, all clients can be found.

Long polling is mainly about refreshing the configuration, keeping the server configuration consistent with the local cache configuration.

First of all, let's take a look at the Nacos map given on the official website of Nacos. We can clearly see that dynamic configuration service is one of the three major functions of Nacos.

Here to borrow the description of the official website, let's take a look at what cool techs Nacos brings to us.

Dynamic configuration services allow you to manage application configuration and service configuration for all environments in a centralized, externalized, and dynamic manner. Dynamic configuration eliminates the need to redeploy applications and services when configuration changes, making configuration management more efficient and agile. Configuration-centric management makes it easier to implement stateless services and makes it easier for services to scale flexibly on demand.

So, with Nacos, maybe a series of problems such as the wrong configuration file and the need to restart the service before we went online will be significantly improved.

One dynamic configuration

In the future, I will take a look at the dynamic configuration capabilities of Nacos and see how Nacos manages configuration in a simple, elegant, and efficient way to achieve dynamic configuration changes.

Let's use a simple example to understand the dynamic configuration capabilities of Nacos.

1. Environmental preparation

First of all, we need to build a Nacos server, which has been interpreted in detail by the quick-start on the official website, so we won't repeat it here.

Https://nacos.io/zh-cn/docs/quick-start.html

After the installation is complete, we can access the Nacos console, as shown in the following figure:

The Nacos console does simple permission control, and the default account and password are nacos.

After logging in, it goes like this:

two。 New configuration

Next, let's create a simple configuration item on the console, as shown in the following figure:

3. Import configuration

Nacos supports importing configurations, and you can import configuration files directly. Here we take the micro-service project open source by everyone as an example.

4. Configure the client

Let me take the sub-service built by myself as an example to take a look at the use of the Nacos configuration center.

First of all, we need to configure it. You only need to pay attention to the configuration of the config node, and the discovery node can ignore it.

Cloud: nacos: discovery: metadata: management: context-path: ${server.servlet.context-path} / actuator server-addr: ${nacos-host:nacos-host}: ${nacos-port:8848} # nacos namespace ID The default is public namespace: ${nacos-namespace:} service: ets-web config: server-addr: ${spring.cloud.nacos.discovery.server-addr} namespace: ${spring.cloud.nacos.discovery.namespace} group: RENREN_CLOUD_GROUP file-extension: yaml # specify the shared configuration And support dynamic refresh extension-configs:-data-id: datasource.yaml group: ${spring.cloud.nacos.config.group} refresh: true-data-id: common.yaml group: ${spring.cloud.nacos.config.group} refresh: true

In fact, the configuration information of the extension-configs node corresponds to the following classes

Next, let's start the service and take a look at the console log

5. Modify configuration information

Next, we change our configuration information on the console of Nacos as shown in the following figure:

After modifying the configuration, click the "publish" button, and the client will receive the latest data, as shown in the following figure:

At this point, we have finished talking about a simple dynamic configuration management function, and the operation of deleting the configuration is similar to updating the configuration. I will not repeat it here.

6. Summary

Through the small case above, we have a general idea of how to use the service dynamically configured by Nacos. The Nacos server saves the configuration information to the database configured by its configuration file. After the client connects to the server, it can get the specific configuration information according to dataID,Group. When the configuration of the server changes, the client will be notified. When the client gets the latest configuration information after the change, it can do its own processing, which is very useful, and all scenarios that need to use configuration can be managed through Nacos.

The principle of configuration center (push or pull)

Now we understand the capabilities of Nacos's dynamic configuration service, but one problem we need to figure out is how the Nacos client gets the latest data from the Nacos server in real time.

In fact, the data interaction between the client and the server is nothing more than two situations:

The server pushes data to the client

Client pulls data from server

Is it a push or a pull? judging from the practice that the Nacos client receives the latest data through Listener, it feels like the data pushed by the server, but it cannot be taken for granted. The quickest and most accurate way to know the answer is to find it from the source code.

Official sample code try {/ / pass configuration String serverAddr = "{serverAddr}"; String dataId = "{dataId}"; String group = "{group}"; Properties properties = new Properties (); properties.put ("serverAddr", serverAddr); / / New configService ConfigService configService = NacosFactory.createConfigService (properties); String content = configService.getConfig (dataId, group, 5000); System.out.println (content) / / register listeners configService.addListener (dataId, group, new Listener () {@ Override public void receiveConfigInfo (String configInfo) {System.out.println ("recieve1:" + configInfo);} @ Override public Executor getExecutor () {return null;}});} catch (NacosException e) {/ / TODO-generated catch block e.printStackTrace ();} 1. Instantiate ConfigService

When we finish booting the package, we will find the following three packages about Nacos

As I understand it, the api package invokes the ability of the client package to interact with the Nacos server. Then when we interact with each other, we will mainly use the NacosConfigService class that implements the ConfigService interface that we analyze next.

Now let's take a look at the construction of NacosConfigService and see how ConfigService is instantiated, as shown in the following figure:

Public class NacosConfigService implements ConfigService {

Private static final Logger LOGGER = LogUtils.logger (NacosConfigService.class)

Private static final long POST_TIMEOUT = 3000L

/ * *

* http agent.

, /

Private final HttpAgent agent

/ * *

* long polling. This is long polling.

, /

Private final ClientWorker worker

Private String namespace

Private final String encode

/ / other codes are omitted

/ / Construction method ic NacosConfigService (Properties properties) throws NacosException {ValidatorUtils.checkInitParam (properties); String encodeTmp = properties.getProperty (PropertyKeyConst.ENCODE); if (StringUtils.isBlank (encodeTmp)) {this.encode = Constants.ENCODE;} else {this.encode = encodeTmp.trim ();} initNamespace (properties); / / object 1 this.agent = new MetricsHttpAgent (new ServerHttpAgent (properties); this.agent.start () / / object 2 this.worker = new ClientWorker (this.agent, this.configFilterChainManager, properties);}

When instantiating, two objects are initialized. They are:

HttpAgent

ClientWorker

HttpAgent

Among them, agent is implemented through the decorator pattern, ServerHttpAgent is the actual working class, and MetricsHttpAgent also calls the method of ServerHttpAgent internally, plus some statistical operations, so we only need to care about the function of ServerHttpAgent.

For students who are not familiar with it, you can read the interpretation of the decorator pattern in the rookie tutorial.

Agent actually plays its role in ClientWorker, and ClientWorker is also a real worker, so let's take a look at the ClientWorker class.

ClientWorker

The following is how ClientWorker is constructed, as shown in the following figure:

Public ClientWorker (final HttpAgent agent, final ConfigFilterChainManager configFilterChainManager, final Properties properties) {this.agent = agent; this.configFilterChainManager = configFilterChainManager; / / Initialize the timeout parameter init (properties); / / created a thread pool for scheduled tasks this.executor = Executors.newScheduledThreadPool (1, new ThreadFactory () {@ Override public Thread newThread (Runnable r) {Thread t = newThread (r) T.setName ("com.alibaba.nacos.client.Worker." + agent.getName ()); t.setDaemon (true); return t;}}) / / create a thread pool this.executorService = Executors .newScheduledThreadPool (Runtime.getRuntime () .ThreadPool (), new ThreadFactory () {@ Override public Thread newThread (Runnable r) {Thread t = newThread (r)) T.setName ("com.alibaba.nacos.client.Worker.longPolling." + agent.getName ()); t.setDaemon (true); return t;}}) / / create a thread pool of deferred tasks to check configuration information every 10ms this.executor.scheduleWithFixedDelay (new Runnable () {@ Override public void run () {try {checkConfigInfo ()) } catch (Throwable e) {LOGGER.error ("[" + agent.getName () + "] [sub-check] rotate check error", e);}, 1L, 10L, TimeUnit.MILLISECONDS);}

You can see that in addition to keeping HttpAgent internally, ClientWorker has created two thread pools:

Final ScheduledExecutorService executor; final ScheduledExecutorService executorService

The first thread pool is responsible for data interaction with the configuration center, and delays 1ms after startup, and then checks the configuration information regularly every other 10ms.

The second thread pool is responsible for maintaining a long polling link.

Next, let's take a look at exactly what the method executed by executor per 10ms does, as shown in the following figure:

/ * *

* groupKey-> cacheData.

, /

Private final AtomicReference cacheMap = new AtomicReference (

New HashMap ()

/ * Check config info. Check configuration information * / public void checkConfigInfo () {/ / split task (solve the problem of large data transmission) int listenerSize = cacheMap.get () .size (); / / round up to batches and check in batches / / ParamUtil.getPerTaskConfigSize () = 3000 int longingTaskCount = (int) Math.ceil (listenerSize / ParamUtil.getPerTaskConfigSize ()) If (longingTaskCount > currentLongingTaskCount) {for (int I = (int) currentLongingTaskCount; I)

< longingTaskCount; i++) { // 要判断任务是否在执行 这块需要好好想想。 任务列表现在是无序的。变化过程可能有问题 executorService.execute(new LongPollingRunnable(i)); } currentLongingTaskCount = longingTaskCount; } } 这里主要是先去拿缓存中 Map 现在我们来看看 LongPollingRunnable 做了什么，主要分为两部分， 第一部分是检查本地的配置信息， 第二部分是获取服务端的配置信息然后更新到本地。 1.本地检查 首先取出与该 taskId 相关的 CacheData，然后对 CacheData 进行检查，包括本地配置检查和缓存数据的 md5 检查，本地检查主要是做一个故障容错，当服务端挂掉后，Nacos 客户端可以从本地的文件系统中获取相关的配置信息，如下图所示： public void run() { List cacheDatas = new ArrayList(); List inInitializingCacheList = new ArrayList(); try { // for (CacheData cacheData : cacheMap.get().values()) { if (cacheData.getTaskId() == taskId) { cacheDatas.add(cacheData); try { //执行检查本地配置 checkLocalConfig(cacheData); if (cacheData.isUseLocalConfigInfo()) { //缓存数据的md5的检查 cacheData.checkListenerMd5(); } } catch (Exception e) { LOGGER.error("get local config info error", e); } } } } //检查本地配置 private void checkLocalConfig(CacheData cacheData) { final String dataId = cacheData.dataId; final String group = cacheData.group; final String tenant = cacheData.tenant; //本地缓存文件 File path = LocalConfigInfoProcessor.getFailoverFile(agent.getName(), dataId, group, tenant); //不使用本地配置，但是持久化文件存在，需要读取文件加载至内存 if (!cacheData.isUseLocalConfigInfo() && path.exists()) { String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant); final String md5 = MD5Utils.md5Hex(content, Constants.ENCODE); cacheData.setUseLocalConfigInfo(true); cacheData.setLocalConfigInfoVersion(path.lastModified()); cacheData.setContent(content); LOGGER.warn( "[{}] [failover-change] failover file created. dataId={}, group={}, tenant={}, md5={}, content={}", agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content)); return; } // 有 ->

No. Do not notify the business listener, but notify after getting the configuration from server. / / use local configuration, but there is no if (cacheData.isUseLocalConfigInfo () & &! path.exists ()) {cacheData.setUseLocalConfigInfo (false); LOGGER.warn ("[{}] [failover-change] failover file deleted. DataId= {}, group= {}, tenant= {} ", agent.getName (), dataId, group, tenant); return } / / change / / use local configuration, persistent file exists, cache is inconsistent with the last modification time of file if (cacheData.isUseLocalConfigInfo () & & path.exists () & & cacheData.getLocalConfigInfoVersion ()! = path .lastModified ()) {String content = LocalConfigInfoProcessor.getFailover (agent.getName (), dataId, group, tenant) Final String md5 = MD5Utils.md5Hex (content, Constants.ENCODE); cacheData.setUseLocalConfigInfo (true); cacheData.setLocalConfigInfoVersion (path.lastModified ()); cacheData.setContent (content); LOGGER.warn ("[{}] [failover-change] failover file changed. DataId= {}, group= {}, tenant= {}, md5= {}, content= {} ", agent.getName (), dataId, group, tenant, md5, ContentUtils.truncateContent (content);}}

The local check is mainly by using the local configuration, then looking for the persistent cache file, and then judging whether the last modification event of the file is consistent with the version of the local cache to determine whether it is changed or not.

By tracking the checkLocalConfig method, you can see that Nacos saves the cache configuration information in the

~ / nacos/config/fixed- {address} _ 8848_nacos/snapshot/DEFAULT_GROUP/ {dataId}

In this file, let's take a look at the contents saved in this file, as shown in the following figure:

two。 Server inspection

Then get the dataId list of value changes from the server through the checkUpdateDataIds () method

Through the getServerConfig method, the latest configuration information is obtained from the dataId to the server, and then the latest configuration information is saved to CacheData.

Finally, call the checkListenerMd5 method of CacheData, and you can see that this method has also been called in the first part, so we need to focus on it.

/ / check the server configuration List changedGroupKeys = checkUpdateDataIds (cacheDatas, inInitializingCacheList); if (! CollectionUtils.isEmpty (changedGroupKeys)) {LOGGER.info ("get changedGroupKeys:" + changedGroupKeys);} for (String groupKey: changedGroupKeys) {String [] key = GroupKey.parseKey (groupKey); String dataId = key [0]; String group = key [1]; String tenant = null If (key.length = = 3) {tenant = key [2];} try {/ / get the latest configuration of the relevant id from the server [] ct = getServerConfig (dataId, group, tenant, 3000L); CacheData cache = cacheMap.get (). Get (dataId, group, tenant); cache.setContent (ct [0]) If (null! = ct [1]) {cache.setType (ct [1]) } LOGGER.info ("[{}] [data-received] dataId= {}, group= {}, tenant= {}, md5= {}, content= {}, type= {}", agent.getName (), dataId, group, tenant, cache.getMd5 (), ContentUtils.truncateContent (ct [0]), ct [1]) } catch (NacosException ioe) {String message = String .format ("[% s] [get-update] get changed config exception. DataId=%s, group=%s, tenant=%s ", agent.getName (), dataId, group, tenant); LOGGER.error (message, ioe) }} for (CacheData cacheData: cacheDatas) {if (! cacheData.isInitializing () | inInitializingCacheList .verification (GroupKey.getKeyTenant (cacheData.dataId, cacheData.group, cacheData.tenant)) {/ / verify MD5 value cacheData.checkListenerMd5 (); cacheData.setInitializing (false);}} inInitializingCacheList.clear (); executorService.execute (this) Catch (Throwable e) {/ / If the rotation training task is abnormal, the next execution time of the task will be punished LOGGER.error ("longPolling error:", e); executorService.schedule (this, taskPenaltyTime, TimeUnit.MILLISECONDS)

We also found that when the client pulls the configuration file from the server, the configuration file will be cached locally, so generally, the local configuration will be preferred. If the local file does not exist or the content is empty, the configuration will be pulled from the remote end through the HTTP GET method and saved to the local cache.

Private String getConfigInner (String tenant, String dataId, String group, long timeoutMs) throws NacosException {group = null2defaultGroup (group); ParamUtils.checkKeyParam (dataId, group); ConfigResponse cr = new ConfigResponse (); cr.setDataId (dataId); cr.setTenant (tenant); cr.setGroup (group); / / give priority to local configuration String content = LocalConfigInfoProcessor.getFailover (agent.getName (), dataId, group, tenant) If (content! = null) {LOGGER.warn ("[{}] [get-config] get failover ok, dataId= {}, group= {}, tenant= {}, config= {}", agent.getName (), dataId, group, tenant, ContentUtils.truncateContent (content)); cr.setContent (content); configFilterChainManager.doFilter (null, cr); content = cr.getContent (); return content } try {String [] ct = worker.getServerConfig (dataId, group, tenant, timeoutMs); cr.setContent (ct [0]); configFilterChainManager.doFilter (null, cr); content = cr.getContent (); return content } catch (NacosException ioe) {if (NacosException.NO_RIGHT = = ioe.getErrCode ()) {throw ioe;} LOGGER.warn ("[{}] [get-config] get from server error, dataId= {}, group= {}, tenant= {}, msg= {}", agent.getName (), dataId, group, tenant, ioe.toString ()) } LOGGER.warn ("[{}] [get-config] get snapshot ok, dataId= {}, group= {}, tenant= {}, config= {}", agent.getName (), dataId, group, tenant, ContentUtils.truncateContent (content)); content = LocalConfigInfoProcessor.getSnapshot (agent.getName (), dataId, group, tenant); cr.setContent (content); configFilterChainManager.doFilter (null, cr); content = cr.getContent () Return content;} 2. Add Listener

Now we can add a Listener for ConfigService, and finally call the addTenantListeners method of ClientWorker, as shown in the following figure:

/ * Add listeners for tenant. * * @ param dataId dataId of data * @ param group group of data * @ param listeners listeners * @ throws NacosException nacos exception * / public void addTenantListeners (String dataId, String group, List

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.