Shulou(Shulou.com)05/31 Report--

It is believed that many inexperienced people are at a loss about the process of opening the service on the Dubbo-go Server side. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

The following describes the basic use of the dubbo-go framework and the introduction of server end source code from the point of view of export call chain.

When you get a framework, a good way to read the source code is as follows: start with the most basic helloworld demo source code-> check the configuration file-> enable various dependent services (such as zk and consul)-> open the server-> then call the server through client-> print the complete request log and response package. After the call is successful, according to the design model of the framework, starting from the parsing of the configuration file, read the call stack of the entire framework from the top down.

For the rpc request in Cpact S mode, the entire call stack is split into two parts: client and server, so you can read from the configuration file parsing on the server side to the listening startup on the server side and an invoker Call call from the configuration file parsing on the client side. Such a complete request became clear.

Run helloworld-demo1 provided by the official website. Dubbo-go version 2.7 QuickStart1) start a go-server service

Clone the warehouse locally

$git clone https://github.com/dubbogo/dubbo-samples.git

Enter the dubbo directory

$cd dubbo-samples/golang/helloworld/dubbo

After entering the directory, you can see four folders, client that supports go and java, and server. We try to run a server of go. Enter the app subfolder and you can see that the go file is saved inside.

$cd go-server/app

Sample file structure

You can see three folders in go-server: app, assembly, and profiles.

The go source code is saved under the app folder, the optional build script for the specific environment is saved under the assembly folder, and the configuration file is saved under profiles. For the dubbo-go framework, configuration files are very important, and the absence of files will cause the service to fail to start.

Set the environment variable that points to the configuration file

Because the dubbo-go framework relies on the configuration file to start, the way for the framework to locate the configuration file is through the environment variable. For the server side, you need to configure two environment variables: CONF_PROVIDER_FILE_PATH and APP_LOG_CONF_FILE, which should point to the server configuration file and the log configuration file, respectively.

In sample, we can use the dev environment, that is, profiles/dev/log.yml and profiles/dev/server.yml files. Under app/, specify these two files from the command line:

$export CONF_PROVIDER_FILE_PATH= ".. / profiles/dev/server.yml"

$export APP_LOG_CONF_FILE= ".. / profiles/dev/log.yml"

Set up the go proxy and run the service

$go run.

If you prompt timeout, you need to set up the goproxy proxy.

$export GOPROXY= "http://goproxy.io"

Run go run again to start the service.

2) run zookeeper

Install zookeeper and run zkServer, which defaults to port 2181.

3) run go-client to invoke server service

Enter the source directory of go-client

$cd go-client/app

Similarly, configure environment variables under / app

$export CONF_CONSUMER_FILE_PATH= ".. / profiles/dev/client.yml"

$export APP_LOG_CONF_FILE= ".. / profiles/dev/log.yml"

Configure the go agent:

$export GOPROXY= "http://goproxy.io"

Run the program

$go run.

The printed request result can be found in the log:

Response result: & {A001 Alex Stocks 18 2020-10-28 14 CST 52 CST 49.131 + 49.131}

Similarly, in the running server, you can find the printed request in the log:

Req: [] interface {} {"A001"}

Rsp:main.User {Id: "A001", Name: "Alex Stocks", Age:18, Time:time.Time {...}

Congratulations! One rpc call based on dubbo-go was successful.

4) FAQ

When a log with both profiderInit and ConsumerInit failures appears at the beginning of the log, check whether the configuration path and configuration file in the environment variable are correct.

When register fails in the log, it usually fails to register with the registry. Check whether the registry is open or not, and check whether the port of register in the configuration file is correct.

The default open port of sample is 20000, which ensures that there is no occupation before startup.

two。 Configure the environment variable export APP_LOG_CONF_FILE= ".. / profiles/dev/log.yml" export CONF_CONSUMER_FILE_PATH= ".. / profiles/dev/client.yml" 3. Server source code 1) directory structure

The directory provided by example of the dubbo-go framework is as follows:

The source code is stored in the app/ folder, and you can write your own environment variable configuration script buliddev.sh

The assembly/ folder stores build scripts for different platforms

Profiles/ folder stores configuration files for different environments

Executable files are stored in the target/ folder

2) key source code

The source code is placed in the app/ folder and mainly contains two files, server.go and user.go. As the name implies, server.go is used to open services and register the transport protocol using the framework; user.go defines the rpc-service structure and the structure of the transport protocol.

User.go

Func init () {config.SetProviderService (new (UserProvider)) / /-for hessian2- hessian.RegisterPOJO (& User {})} type User struct {Id string Name string Age int32 Time time.Time} type UserProvider struct {} func (u * UserProvider) GetUser (ctx context.Context, req [] interface {}) (* User, error) {

As you can see, there is an init function in user.go, which is the first part of the server code to be executed. User is a user-defined transport structure, and UserProvider is a user-defined rpc_service; that contains a rpc function, GetUser. Of course, users can customize other rpc functions.

In the init function, call the SetProviderService function of config to register the current rpc_service with the framework config.

You can view the blueprints provided in the dubbo official documentation:

Below the service layer is the config layer, and the user service will register down layer by layer, and finally realize the server-side exposure.

After the rpc-service is registered, call the hessian API to register the transport structure User.

At this point, the init function has been executed.

Server.go

/ / they are necessary:// export CONF_PROVIDER_FILE_PATH= "xxx" / / export APP_LOG_CONF_FILE= "xxx" func main () {hessian.RegisterPOJO (& User {}) config.Load () initSignal ()} func initSignal () {signals: = make (chan os.Signal, 1)...

The main function is then executed.

There are only two operations done in the main function, first registering the User structure with the hessian registration component (a slight repetition as before), so that you can unpack it later using getty.

Then call the config.Load function, which is located in the framework config/config_loader.go, which is the starting point of the entire framework service. The important configuration process in this function will be described in detail below. After executing the Load () function, the configuration file is read into the framework, and then the registered service is implemented into the configuration structure according to the contents of the configuration file, and then the Export is called to expose the specific registry, and then the specific service is opened for tcp listening on the corresponding port, and the service is successfully started and exposed.

Finally turn on signal listening initSignal () gracefully ends the startup process of a service.

4. Client source code

The client contains two files, client.go and user.go, in which user.go is exactly the same as the server, so I won't repeat it.

Client.go

/ they are necessary:// export CONF_CONSUMER_FILE_PATH= "xxx" / / export APP_LOG_CONF_FILE= "xxx" func main () {hessian.RegisterPOJO (& User {}) config.Load () time.Sleep (3e9) println ("\ n\ n\ nstart to test dubbo") user: = & User {} err: = userProvider.GetUser (context.TODO (), [] interface {} {"A001"}) User) if err! = nil {panic (err)} println ("response result:% v\ n", user) initSignal ()}

The main function is similar to the server side, first registering the transport structure with hessian, and then calling the config.Load () function. As described below, the client and server execute the specific functions loadConsumerConfig () and loadProviderConfig () in config.Load () according to the configuration type, thus achieving the purpose of "opening the service" and "invoking the service".

After loading the configuration, the corresponding function of the client instance userProvider is rewritten by implementing the service, adding the function proxy, applying for registry and reloadInvoker pointing to the server ip. In this case, by calling the GetUser function, you can directly call the open server through invoker to implement the rpc process.

The following describes the process of service startup, registry registration and invocation in detail from both the server side and the client side.

5. Custom profile (non-environment variable) method 1) Server-side custom profile

Var providerConfigStr = xxxxx// configuration file contents, you can refer to log and client. Here you can define how to get configuration files, such as configuration center and local file reading.

Log address: https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/log.yml

Client address: https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/client.yml

Set the configuration before config.Load (), for example:

Func main () {hessian.RegisterPOJO (& User {}) providerConfig: = config.ProviderConfig {} yaml.Unmarshal ([] byte (providerConfigStr), & providerConfig) config.SetProviderConfig (providerConfig) defaultServerConfig: = dubbo.GetDefaultServerConfig () dubbo.SetServerConfig (defaultServerConfig) logger.SetLoggerLevel ("warn") / info,warn config.Load () select {} 2) client customization profile

Var consumerConfigStr = xxxxx// configuration file contents, you can refer to log and clien. Here you can define how to get configuration files, such as configuration center and local file reading.

Set the configuration before config.Load (), for example:

Func main () {p: = config.ConsumerConfig {} yaml.Unmarshal ([] byte (consumerConfigStr), & p) config.SetConsumerConfig (p) defaultClientConfig: = dubbo.GetDefaultClientConfig () dubbo.SetClientConf (defaultClientConfig) logger.SetLoggerLevel ("warn") / / info,warn config.Load () user: = & User {} err: = userProvider.GetUser (context.TODO (), [] interface {} {"A001"}) User) if err! = nil {log.Print (err) return} log.Print (user)} Server side

The service exposure process involves several encapsulation and exposure of the original rpcService, and the diagrams of other articles on the Internet feel too general. Here, we briefly draw a data flow diagram of a user-defined service:

1. Load configuration 1) Framework initialization

Before loading the configuration, the framework provides many defined protocols, factories and other components, which will be registered on the extension module in the corresponding module init function for selection in the following configuration files.

The important ones are:

Default function proxy factory: common/proxy/proxy_factory/default.go

Func init () {extension.SetProxyFactory ("default", NewDefaultProxyFactory)}

Its function is to encapsulate the original rpc-service to form proxy_invoker, which makes it easier to implement remote call calls. For more information, please see its invoke function.

Registry Registration Agreement: registry/protocol/protocol.go

Func init () {extension.SetProtocol ("registry", GetProtocol)}

It is responsible for exposing the invoker to the corresponding registry, such as the zk registry.

Zookeeper Registration Protocol: registry/zookeeper/zookeeper.go

Func init () {extension.SetRegistry ("zookeeper", newZkRegistry)}

It incorporates base_resiger and is responsible for registering the service with the zookeeper registry during service exposure, thus providing invocation methods for callers.

Dubbo Transport Protocol: protocol/dubbo/dubbo.go

Func init () {extension.SetProtocol (DUBBO, GetProtocol)}

It is responsible for listening to the corresponding port, exposing the specific service, and starting the corresponding event handler, passing the event event of remote invocation to the invoker, calling the local invoker and returning the execution result.

Filter wrapper call chain protocol: protocol/protocolwrapper/protocol_filter_wrapper.go

Func init () {extension.SetProtocol (FILTER, GetProtocol)}

It is responsible for packaging the proxy invoker during service exposure, forming a call chain through the configured filter, and delivering it to the dubbo protocol for exposure.

The components that have been implemented by the above pre-registered framework will be used throughout the service exposure invocation chain and will take what they need according to the configuration.

2) configuration file

The important configuration required by the server side has three fields: services, protocols, and registries.

Profiles/dev/server.yml:

Registries: "demoZk": protocol: "zookeeper" timeout: "3s" address: "127.0.0.1 zookeeper 2181" services: "UserProvider": # multiple registry can be specified, separated by commas Do not specify that registry: "demoZk" protocol: "dubbo" # is equivalent to interface interface: "com.ikurento.user.UserProvider" loadbalance: "random" warmup: "100th" cluster: "failover" methods:-name: "GetUser" retries: 1 loadbalance: "random" protocols: "dubbo": name: "dubbo" port: 20000 in dubbo.xml

Where service specifies the rpc-service name to be exposed ("UserProvider"), the exposed protocol name ("dubbo"), the registered protocol name ("demoZk"), the interface where the exposed service is located, the load balancing policy, the cluster failure policy and the method to be invoked, and so on.

The protocol name of the intermediary service needs to correspond to the mapkey under the registries, and the exposed protocol name needs to correspond to the mapkey under the protocols.

As you can see in the above example, dubbo is used as the exposure protocol, zookeeper is used as the intermediate registration protocol, and the port is given. If zk needs to set a user name and password, it can also be written in the configuration.

3) Reading and checking of configuration files

Config/config_loader.go:: Load ()

In the main function of the above example, there is a direct call to the config.Load () function. The execution details of this function are as follows:

/ / Load Dubbo Initfunc Load () {/ / init router initRouter () / / init the global event dispatcher extension.SetAndInitGlobalDispatcher (GetBaseConfig () .EventDispatcherType) / / start the metadata report if config set if err: = startMetadataReport (GetApplicationConfig () .MetadataType, GetBaseConfig () .MetadataReportConfig) Err! = nil {logger.Errorf ("Provider starts metadata report error, and the error is {% # v}", err) return} / / reference config loadConsumerConfig () / / service config loadProviderConfig () / / init the shutdown callback GracefulShutdownInit ()}

In this article, we focus on the loadConsumerConfig () and loadProviderConfig () functions.

For the provider side, you can see the code of the loadProviderConfig () function as follows:

The first half is the read-in and check of the configuration, and after entering the for loop, it is the exposure starting point of a single service.

As mentioned earlier, you have written all kinds of information about the service to be exposed in the configuration file, such as service name, interface name, method name, and so on. Within the for loop in the figure, all the service services are implemented in turn.

In the first line of the for loop, call the GetProviderService function according to key to get the registered rpcService instance, which corresponds to the rpc-service instance manually registered by the user in the init function mentioned above:

This object also becomes the rpcService variable in the for loop, writes the object registration to sys (ServiceConfig type) through the Implement function, sets the key and protocol group of sys, and finally calls the Export method of sys.

Here corresponds to the portion of the flowchart:

At this point, the framework configuration structure has all the service-related configurations, as well as user-defined rpc-service instances, which trigger the Export method to expose its own instances. This is the starting point of the Export call chain.

two。 The original service is encapsulated into proxy_invoker

Config/service_config.go:: Export ()

Next go to the ServiceConfig.Export () function.

This function performs some detailed operations, such as assigning random ports to different protocols. If multiple center registration protocols are specified, the service will be exposed through the registryProtocol of multiple center registration protocols. We are only concerned about how a registration protocol operates. There are also operations such as generating calls to url and registering url to prepare for exposure.

1) first generate the corresponding registryUrl and serviceUrl through configuration

At present, this regUrl only contains information related to registration (zk). Later, it will be added to ServiceIvk, that is, information related to service invocation, including method name, parameters, etc.

2) for a registration protocol, register the incoming rpc-service instance in common.ServiceMap

This Register function registers the service instance twice, once to write to the interface service group with Interface for key, and once to write to a specific unique service for key with interface and proto.

This instance will be taken out of common.Map later.

3) get the default proxy factory, encapsulate the instance into the proxy invoker// and get a proxyInvoker. The url of this invoker is the incoming regUrl, and this place encapsulates the service instance registered above into invoker//. The default returned by GetProxyFactory is common/proxy/proxy_factory/default.go//, which calls GetInvoker to get the default proxyInvoker. Saved the current registration urlinvoker: = extension.GetProxyFactory (providerConfig.ProxyFactory). GetInvoker (* regUrl) / / exposed to generate exporter, turn on tcp snooping / / here you should jump to the Export called by registry/protocol/protocol.go registryProtocol, and export the current proxyInvoker to exporter = c.cacheProtocol.Export (invoker)

You can go to the common/proxy/proxy_factory/default.go::ProxyInvoker.Invoke () function and see the section that calls svc for common.Map and the function that actually calls Call for the corresponding Method of svc as follows:

At this point, the invoker returned by the above GetInvoker (* regUrl) is proxy_invoker, which encapsulates the user-defined rpc_service and encapsulates the specific calling logic in the Invoke function.

Why use Proxy_invoker to call?

Through this proxy_invoke to call the user's function, the call mode will be more abstract, you can see in the code, through ins and outs to define the input and output parameters, the entire call logic is abstracted into the invocation structure, and the specific function name selection, parameter down transfer and reflect reflection process are encapsulated in the invoke function, this design is more conducive to later remote calls. Personally, I think this is the design idea of dubbo Invoke call chain.

At this point, the corresponding part of the figure is implemented:

3. Registry protocol exposes the above proxy_invoker on zkRegistry

Above, we executed exporter = c.cacheProtocol.Export (invoker).

The cacheProtocol here is designed for a layer of cache, which corresponds to the original demo, and here is the registryProtocol implemented by default.

Registry/protocol/protocol.go:: Export ()

Multiple EventListener are constructed in this function, which is very java-like.

We only care about the process of service exposure, ignoring these listeners first.

1) obtain registration url and service url

2) obtain registry instance zkRegistry

One layer of cache operation, if the cache does not need to retrieve the zkRegistry from the common.

3) zkRegistry calls the Registry method to register dubboPath on zookeeper

The above gets a specific zkRegistry instance, which is defined in: registry/zookeeper/registry.go.

The structure combines the registry.BaseRegistry structure, and the base structure defines the basic functions of the registry, such as Registry, Subscribe, etc., but inside these default defined functions, the specific implementation functions of the facade layer (zkRegistry layer) will still be called. This design model can introduce the implementation of the outer functions while ensuring that the existing functions do not need to be defined repeatedly, similar to the structure inheriting and reusing the code. This design pattern is worth learning.

We look at the registry/protocol/protocol.go:: Export () function above and call it directly:

/ / 1. Through the zk registry, call the Register () function to register the existing @ root@rawurl on zk err: = reg.Register (* registeredProviderUrl)

Register the existing RegistryUrl with zkRegistry.

This step calls the Register function of baseRegistry, which in turn calls the DoRegister function of zkRegister, which in turn calls:

In this function, a new node will be created corresponding to root.

And write specific node information. Node is the result of url passing through encode, including the invocation mode of the server.

This part of the code is more complex, you can see baseRegistry's processURL () function: http://t.tb.cn/6Xje4bijnsIDNaSmyPc4Ot.

At this point, the server call url is registered on the zookeeper, and if the client wants to get the url, all it needs to do is pass in a specific dubboPath and request from the zk. At present, client can obtain the access method, but the specific service on the server side has not been started, and the listening of the specific protocol port has not been enabled. This is what the registry/protocol/protocol.go:: Export () function will do next.

4) proxy_invoker encapsulates wrapped_invoker and gets filter call chain / / invoker encapsulated into warppedInvoker wrappedInvoker: = newWrappedInvoker (invoker, providerUrl) / / after adding filter call chain to invoker, and then using dubbo protocol Export, service is enabled and Exporter is returned. / / export_1 cachedExporter = extension.GetProtocol (protocolwrapper.FILTER) .Export (wrappedInvoker)

Create a new WrappedInvoker for subsequent chained calls.

Get the ProtocolFilterWrapper that is implemented and registered in advance, call the Export method, and further expose.

Protocol/protocolwrapped/protocol_filter_wrapper.go:Export ()

Protocol/protocolwrapped/protocol_filter_wrapper.go:buildInvokerChain

It can be seen that according to the content of the configuration, through the construction of chain calls, the proxy_invoker is wrapped layer by layer at the bottom of the call chain, and finally a call chain invoker is returned.

Corresponding to the part in the figure:

At this point, we have the filter call chain and look forward to exposing this chain to a specific port for the corresponding request event.

5) expose wrapped_invoker through dubbo protocol

Protocol/protocolwrapped/protocol_filter_wrapper.go:Export ()

/ / export_2 return pfw.protocol.Export (invoker) called through dubbo protocol Export dubbo_protocol

Going back to the last line of the above Export function, the Export method of dubboProtocol is called to actually expose the above chain.

The specific implementation of the Export method is: protocol/dubbo/dubbo_protocol.go: Export ().

This function does two things: construct the trigger and start the service.

The incoming Invoker call chain is further encapsulated into an exporter, and then the export is saved in map. Be careful! Here you put the exporter into the SetExporterMap, and when the following service starts, the exporter will be taken out in the form of a registered event listener!

Call the openServer method of dubboProtocol to turn on a listening for a specific port.

As shown in the figure above, a Session is passed in to enable event listening on the corresponding port.

At this point, the exporter is constructed, and the part of the figure is completed:

4. Register trigger action

The above only starts the service, but the details of the trigger event have not been seen yet. Click the s.newSession above to see that the dubbo protocol uses the following configuration by default for the session of a getty:

One of the most important configurations is EventListener, which is passed in the default rpcHandler of dubboServer.

Protocol/dubbo/listener.go:OnMessage ()

RpcHandler has a well-implemented OnMessage function, and according to getty's API, OnMessage is triggered when client calls the port.

/ / OnMessage notified when RPC server session got any message in connectionfunc (h * RpcServerHandler) OnMessage (session getty.Session, pkg interface {}) {

This function implements a series of processing after the getty session receives the rpc call:

Parsing of incoming packets

Construct the request url according to the request packet

Get the corresponding request key and find the exporter to be called

Get the corresponding Invoker

Construct invocation

Call

Return

The whole transfer process was done in one fell swoop. Realized from the call event of getty.Session to the call of invoker after layers of encapsulation.

At this point, a rpc call is returned correctly.

On the layer-by-layer Packaging of Invoker

It can abstract a call into an invoke;, it can abstract a protocol into an encapsulation for invoke, and encapsulate the specific changes made for an invoke into the invoke function, which can reduce the coupling between modules. The logic of layer upon layer encapsulation is clearer.

About the abstraction of URL

The extreme abstraction of the unified request object URL for dubbo is unprecedented. Personally, I think this encapsulation can ensure the simplification and consistency of the request parameter list. However, in the process of development, the abuse of extremely abstract interfaces may result. The difficulties of debug? And do not know which fields are currently encapsulated and which are useless.

Understanding of the agreement

The previous understanding of the protocol is still too specific, and with regard to the dubbo-go protocol for dubboProtocol, I think it is based on the further encapsulation of getty, which defines what specific operations the client and server should have for the session of getty, so as to ensure the consistency of the main tone and the tuned protocol, and this guarantee is also the embodiment of the protocol, which is regulated by the dubbo protocol.

After reading the above, have you mastered the method of opening the service on the Dubbo-go Server side? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.