Shulou(Shulou.com)06/03 Report--

Author: Chen Mingjie (sandyskies)

Tars is a unified application framework of the background logic layer that Tencent has been using since 2008. Currently, it supports the Java Magi PHP language NodejsMagi Golang. The framework provides users with a complete set of solutions related to development, operation and maintenance, and testing, which helps a product or service to develop, deploy, test and launch quickly. It integrates scalable protocol coding and decoding, high-performance RPC communication framework, name routing and discovery, release monitoring, log statistics, configuration management and so on. Through it, we can quickly build our own stable and reliable distributed applications in the way of micro-services, and achieve complete and effective service governance. At present, the framework is in Tencent, the major core businesses are in use, very popular, based on the framework deployment and operation of tens of thousands of service nodes.

Tars opened source in April 2017 and joined the Linux Foundation in June 2018.

TarsGo is the Goto language implementation version of Tars, open source in September 2018, the project address is https://github.com/TarsCloud/TarsGo, welcome to star.

New version of TarsGo released

After the last open source, some users responded to some requirements, based on the requirements of user feedback, we implemented and released version 1.1.0. This release added: support for pb, support for zipkin distributed tracking, support for filter (custom plug-in writing), support for context, etc., in addition to a series of optimization and bugfix.

What's new: PB support

Protocol Buffers (PB for short) is a data exchange format of Google, which is independent of language and platform. It was first published in July 2008. With the development of micro-service architecture and its excellent performance, ProtoBuf can be used in many fields, such as network transmission, configuration files, data storage and so on.

For existing users who already use grpc, use proto files, and want to convert to tars protocols, you need to translate the above proto files into Tars files. This kind of translation will be tedious and error-prone. For this reason, we decided to write a plug-in that supports proto files to directly generate tars's rpc logic. Protoc-gen-go code logic is reserved for plug-in writing specifications, referring to grpc, mainly grpc/grpc.go and an import plug-in link_grpc.go. Here we write tarsrpc/tarsrpc.go and link_tarsrpc.go

Aspects of use:

Put these two files under protoc-gen-go, and go install regenerates the protoc-gen-go binary

Define the proto file

Use the recompiled installed protoc-gen-go to generate serialization and rpc-related interface code protoc--go_out=plugins=tarsrpc:. Helloworld.proto

Write tars client-side and server-side code, the parameters use the structure generated by pb, and the rest of the code logic is the same as the normal tars service.

For detailed principles and usage documents, read Tencent Cloud Community articles about the new feature: filter mechanism, which supports zipkin distributed tracking.

In order to support users to write plug-ins, we support the filter mechanism, which is divided into server-side filters and client-side filters. Users can implement their own TarsGo plug-ins based on this mechanism.

/ / server filter, incoming dispatch, and f, used to call user code, req, and resp for incoming user requests and server corresponding packet body type ServerFilter func (ctx context.Context, d Dispatch, f interface {}, req * requestf.RequestPacket, resp * requestf.ResponsePacket, withContext bool) (err error) / / client filter, input msg (including obj information, adapter information, req and resp packet body) There are also user-set call timeout type ClientFilter func (ctx context.Context, msg * Message, invoke Invoke, timeout time.Duration) (err error) / registered server filter / / func RegisterServerFilter (f ServerFilter) / / registered client filter / / func RegisterClientFilter (f ClientFilter)

With the filter, we can do some filtering on requests from both the server and the client, such as opentracing's span using hook for distributed tracking.

Let's take a look at the client-side filter example:

/ generate the client tars filter, and register this filter to realize the injection of span func ZipkinClientFilter () tars.ClientFilter {return func (ctx context.Context, msg * tars.Message, invoke tars.Invoke, timeout time.Duration) (err error) {var pCtx opentracing.SpanContext req: = msg.Req / / first check whether there is any information passed to invoke the chain from the context called by the client. / / if so Then take this as the parent span, if not, a new span,span name is the function name of the RPC request if parent: = opentracing.SpanFromContext (ctx) Parent! = nil {pCtx = parent.Context ()} cSpan: = opentracing.GlobalTracer (). StartSpan (req.SFuncName, opentracing.ChildOf (pCtx), ext.SpanKindRPCClient,) defer cSpan.Finish () cfg: = tars.GetServerConfig () / / set span information, such as the ip address of the client we called and the requested interface Method, protocol, client version, etc. CSpan.SetTag ("client.ipv4", cfg.LocalIP) cSpan.SetTag ("tars.interface", req.SServantName) cSpan.SetTag ("tars.method", req.SFuncName) cSpan.SetTag ("tars.protocol", "tars") cSpan.SetTag ("tars.client.version", tars.TarsVersion) / / inject span into the Status of the request packet Status is a map [strint] string structure if req.Status! = nil {err = opentracing.GlobalTracer () .Inject (cSpan.Context (), opentracing.TextMap, opentracing.TextMapCarrier (req.Status)) if err! = nil {logger.Error ("inject span to status error:") Err)}} else {s: = make (map [string] string) err = opentracing.GlobalTracer () .Inject (cSpan.Context (), opentracing.TextMap, opentracing.TextMapCarrier (s)) if err! = nil {logger.Error ("inject span to status error:" Err)} else {req.Status = s}} / / nothing else needs to be modified If the client calls err = invoke (ctx, msg, timeout) if err! = nil {/ / call error, record the error message of span ext.Error.Set (cSpan, true) cSpan.LogFields (oplog.String ("event", "error"), oplog.String ("message", err.Error ())} return err}

The server will also register a filter, and the main function is to extract the context of the call chain from the status of the request packet and use this as the parent span to record the call information.

One effect as a whole:

For more information, see TarsGo/tars/plugin/zipkintracing

For complete client and server examples of zipkin tracing, please see ZipkinTraceClient and ZipkinTraceServer under TarsGo/examples.

New feature: support for context

TarsGo previously did not use context in the generated client code or in the implementation code passed in by the user. This makes it difficult for us to transmit some information about the framework, such as client-side ip, port, etc., or for users to pass some information about the call chain to the framework. Through a refactoring of the interface, context is supported, and all these contextual information will be implemented through context. In order to be compatible with the old user behavior, this refactoring adopts a fully compatible design.

The server uses context

Type ContextTestImp struct {} / / just add the ctx context.Context parameter func (imp * ContextTestImp) Add (ctx context.Context, an int32, b int32, c * int32) (int32, error) {/ / We can get the information passed by the framework through context, such as getting ip below, and even returning some information to the framework For details, see the interface ip under tars/util/current, ok: = current.GetClientIPFromContext (ctx) if! ok {logger.Error ("Error getting ip from context")} return 0, nil} / / used to use AddServant, but now you just need to change it to AddServantWithContextapp.AddServantWithContext (imp, cfg.App+ "." + cfg.Server+ ".ContextTestObj").

The client uses context

Ctx: = context.Background () c: = make (map [string] string) c ["a"] = "b" / / used to use app.Add for client calls. Here, as long as it becomes app.AddWithContext, you can pass context to the framework. If you want to set the context// to the tars request, you can pass in multiple parameters, such as c, which is optional. The format is... [string] string ret, err: = app.AddWithContext (ctx, I, iTunes 2, & out, c)

For complete examples of server and client, please see TarGo/examples

Other optimizations and repairs change the Sbuffer field of request package from vector to vector, solve communication problems with other languages, repair stat monitoring and reporting problems, log level from remote update, repair routing refresh, optimize the protocol pool scheme in extreme cases, and add a protocol pool scheme to repair the panic problem caused by the start-up sequence of the go program. Most of the golint code

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.