Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is Dubbo metadata center". In daily operation, I believe many people have doubts about what Dubbo metadata center is. The editor consulted all kinds of materials and sorted out simple and easy operation methods. I hope to help you answer the doubts about "what is Dubbo metadata center"! Next, please follow the small series to learn together!

preface

If you were to build a Dubbo application locally, what additional middleware would you need to build? If I'm not mistaken, your first instinct should be a registry, and most Dubbo-like service governance frameworks have a concept of a registry. You can deploy a Zookeeper, or a Nacos, depending on your preference. But after Apache Dubbo version 2.7, two additional middleware were introduced: the metadata center and the configuration center.

When Dubbo 2.7 was first released at the beginning of this year, I wrote an article entitled "Detailed explanation of the three new features of Dubbo 2.7", which introduced the three new features including metadata center transformation, but some details were not presented in detail. In this article, I will take Dubbo as an example to discuss the significance and integration details of metadata center in the service governance framework.

Introduction to Metadata Center

Metadata in service governance refers to service grouping, service version, service name, method list, method parameter list, timeout time, etc., which will be stored in the metadata center. A concept on an equal footing with metadata is registration information for services, i.e. service groupings, service versions, service names, address lists, etc., which will be stored in the registry. A slight comparison shows that the metadata center and registry store some common service information, such as service names. The metadata center also stores a method list, i.e., a parameter list, and the registry stores service addresses. The overview above illustrates the different roles metadata centers and registries play in service governance. For an intuitive comparison, I have compiled the following table:

metadata registration information responsibility description service, definition of basic attributes of service storage address list change frequency is basically unchanged along with the service up and down line continuously change data size data interaction/storage model consumer/provider reporting, console query PubSub model, provider reporting, consumer subscription main use scenarios service test, service MOCK service invocation availability requirements metadata center availability requirements are not high, do not affect the main process registry availability requirements are high, Main process affecting service invocation

Below I'll analyze each comparison point separately to deepen my understanding of the metadata center.

duties

Before Dubbo 2.7, there was no concept of a metadata center, where registration information and metadata were coupled together. The service configuration of Dubbo Provider has nearly 30 configuration items, excluding some parameters required for service governance of registry, most of which are only used by the Provider itself and do not need to be transmitted to consumers;Dubbo Consumer also has more than 20 configuration items. In the registry, the list of service consumers only needs to focus on a small number of configurations such as application, version, group, ip, dubbo version, etc. This part of the data does not need to enter the registry, but only needs to be persisted in the metadata center in the form of key-value. From the perspective of responsibilities, storing the data of different responsibilities in the corresponding components will make the logic clearer.

frequency of change

The coupling of registration information and metadata will lead to the expansion of the data volume of the registry, which will increase the network overhead of the registry and directly cause negative effects such as slow service address push. Service online and offline will occur at any time. What changes is actually registration information, and metadata is relatively unchanged.

amount of data

Because metadata contains the list of methods and parameters for the service, this data results in metadata that is much larger than the registry information. Registration information is designed to be lean and has a direct impact on SLAs for service delivery.

Data interaction/storage model

Registry uses PubSub model, which belongs to everyone's consensus, so the selection of registry components generally requires it to have notify mechanism. The metadata center does not have the requirement of notify, and generally only requires components to provide key-value storage structures.

Main usage scenarios

In service governance, the registry acts as an address book, enabling consumers to find providers in complex distributed scenarios. The metadata stored in the metadata center is mainly applicable to scenarios such as service test and service MOCK, which all have demands on method list and parameter list. In the following sections, I'll also describe usage scenarios in more detail.

availability requirements

A registry outage or network outage directly affects service availability by affecting the primary path of service invocation. However, in general, problems with the metadata center do not affect service invocation, and the capabilities it provides can be degraded. This also explains why many users did not configure the metadata center in Dubbo 2.7 and did not affect normal use. Metadata centers play a role in service governance that is icing on the cake. When selecting components, we generally have higher availability requirements for the registry, and the metadata center can relax the requirements.

The value of metadata centres

Children only see right and wrong, adults only see pros and cons. The introduction of an additional metadata center will inevitably bring about problems such as operation and maintenance costs, understanding costs, migration costs, etc., so what value does it have to convince everyone to choose it? When we introduced the metadata center above, we have already mentioned scenarios such as service test and service MOCK. In this section, we will focus on the value of metadata center.

Reduce latency of address push

Since the registry uses the PubSub model, the size of the data will directly affect the service address push time. I wonder if you have encountered an error of No provider available? Obviously, the provider has been started, but due to the slow push of the registry, many problems will be caused. On the one hand, it will affect the availability of the service, and on the other hand, it will increase the difficulty of troubleshooting.

In a Hangzhou Dubbo Meetup, Netease Koala shared their transformation of Zookeeper, the root of which is

Large push volume-> Large storage data volume-> Large network transmission volume-> Serious delay

This practical case proves that metadata transformation is not a demand generated out of thin air, but a pain point.

Service Test & Service MOCK

Before Dubbo 2.7, although the registry stored a lot of data that should belong to metadata, it also missed some metadata, such as the method list of services and the parameter list. These are the data necessary for service testing and service MOCK, and to use these capabilities, a metadata center must be introduced. For example, the open source Dubbo Admin implements service testing functionality, allowing users to perform functional tests on published service providers on the console. You may have wondered why only Dubbo 2.7 supports service testing. Aha, the reason is that Dubbo 2.7 has the concept of metadata center. Of course, the same goes for service mocks.

other scenarios

It can be understood that any function that depends on metadata needs to be supported by metadata centers. Other scenarios include gateway applications fetching metadata for generalized invocations, service automation testing, and so on. Describe another possible scenario, and throw a brick at the jade. At a Nanjing Dubbo Meetup, the author of dubbo.js mentioned a scenario where he hoped to automatically generate NodeJs code based on metadata to simplify the amount of front-end development, which is also one of the roles of metadata. This is where your imagination comes in.

Dubbo Configuration Metadata Center

Currently, the latest version of Dubbo is 2.7.4. Several metadata centers currently supported can be found in the source code (the official documentation has not been updated):

Support consume, etcd, nacos, redis, zookeeper these five components.

The configuration is as follows:

dubbo.metadata-report.address=nacos://127.0.0.1:8848

Analysis of metadata storage format

Earlier we introduced the origin and value of metadata center, or floating in the sky concept, this section will let the concept land. What format is metadata stored in?

Take DemoService as an example:

First observe how the registry stores information about this service in Dubbo 2.6.x:

dubbo://30.5.120.185:20880/com.alibaba.dubbo.demo.DemoService?

anyhost=true&

application=demo-provider&

interface=com.alibaba.dubbo.demo.DemoService&

methods=sayHello&

bean.name=com.alibaba.dubbo.demo.DemoService&

dubbo=2.0.2&executes=4500&

generic=false&owner=kirito&

pid=84228&retries=7&side=provider×tamp=1552965771067

For example, bean.name and owner attributes are definitely not necessary to register.

Next, we use best practices in Dubbo 2.7 to configure simplified=true for registry:

Looking at the registry data later, it has become quite streamlined:

dubbo://30.5.120.185:20880/org.apache.dubbo.demo.api.DemoService?

application=demo-provider&

dubbo=2.0.2&

release=2.7.0&

timestamp=1552975501873

The data that is simplified and omitted does not mean that it is not used, but transferred to the metadata center. Let's observe the data in the metadata center at this time:

best practices

Metadata centers are a key component of service governance, but a relatively new concept for most users, so I've compiled what I think are best practices to share with you.

When migrating from Dubbo 2.6 to Dubbo 2.7, you can adopt a three-step strategy to smoothly migrate metadata. Step 1: Dubbo 2.6 + Registry, Step 2: Dubbo 2.7 + Registry + Metadata, Step 3: Dubbo 2.7 + Registry (simplified=true)+ Metadata. In future upgrades of Dubbo, the simplified default value of registry will change to true, currently false, to allow users time to upgrade.

When the application starts, metadata will be published once, after which there will be a timer, metadata synchronization once a day, the above report those run-time generated beans, currently users can not configure the metadata reporting cycle, but can be turned off by-Dcycle.report.

Recommended metadata centers: Nacos and Redis.

Dubbo 2.7 also has a lot of interesting features, if you have any interesting questions about Dubbo, welcome to leave a message at the end of the article or in the background, I will continue to update the Dubbo series articles later.

At this point, the study of "what is Dubbo metadata center" is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.