Shulou(Shulou.com)06/01 Report--

This article analyzes "how to get started with go-kit". The content is detailed and easy to understand. Friends who are interested in "how to get started with go-kit" can follow the editor's idea to read it slowly and deeply. I hope it will be helpful to everyone after reading. Let's follow the editor to learn more about how to get started with go-kit.

The first principle

Let's create a minimal Go-kit service. Now we will use a separate main.go file.

Your business logic

Your service starts with your business logic. In Go kit, we model services as interfaces.

/ StringService provides operations on strings.import "context" type StringService interface {Uppercase (string) (string, error) Count (string) int}

The interface will have an implementation.

Import ("context"errors"strings") type stringService struct {} func (stringService) Uppercase (s string) (string, error) {if s = = "{return", ErrEmpty} return strings.ToUpper (s) Nil} func (stringService) Count (s string) int {return len (s)} / / ErrEmpty is returned when input string is emptyvar ErrEmpty = errors.New ("Empty string") request and response

In Go-kit, the main messaging mode is RPC. Therefore, each method in the interface will be modeled as a remote procedure call request and response (request and response) structure, capturing all input and output parameters, respectively.

Type uppercaseRequest struct {S string `json: "s" `} type uppercaseResponse struct {V string `json: "v" `Err string `json: "err,omitempty"` / errors don't JSON-marshal, so we use a string} type countRequest struct {S int `json: "s" `} type countResponse struct {V int `json: "v"`} Endpoint

Gokit provides functionality through an abstract concept called endpoint.

The endpoint is defined as follows (you don't have to put it in your code, it's provided by go-kit):

Type Endpoint func (ctx context.Context, request interface {}) response interface {}, err error)

It represents a separate RPC. Is a method in the service interface. We will write a simple adapter that converts each method of the service into an endpoint. Each adapter accepts an StringService and returns an endpoint corresponding to one of the methods.

Import ("context"github.com/go-kit/kit/endpoint") func makeUppercaseEndpoint (svc StringService) endpoint.Endpoint {return func (_ context.Context, request interface {}) (interface {}, error) {req: = request. (uppercaseRequest) v, err: = svc.Uppercase (req.S) if err! = nil {return uppercaseResponse {v Err.Error ()}, nil} return uppercaseResponse {v, ""}, nil}} func makeCountEndpoint (svc StringService) endpoint.Endpoint {return func (_ context.Context, request interface {}) (interface {}, error) {req: = request. (countRequest) v: = svc.Count (req.S) return countResponse {v} Nil}} Transmission

Now, we need to expose your service to the outside world so that it can be invoked. Your organization may already have its own views on how services should communicate with each other. Maybe you use Thrift, or a custom JSON on HTTP. Go kit supports many transport protocols out of the box.

For this smallest example service, let's use JSON over HTTP. Go kit provides a helper structure in the package transport/HTTP.

Import ("context"encoding/json"log"net/http" httptransport "github.com/go-kit/kit/transport/http") func main () {svc: = stringService {} uppercaseHandler: = httptransport.NewServer (makeUppercaseEndpoint (svc), decodeUppercaseRequest, encodeResponse ) countHandler: = httptransport.NewServer (makeCountEndpoint (svc), decodeCountRequest, encodeResponse,) http.Handle ("/ uppercase", uppercaseHandler) http.Handle ("/ count", countHandler) log.Fatal (http.ListenAndServe (": 8080", nil))} func decodeUppercaseRequest (_ context.Context, r * http.Request) (interface {} Error) {var request uppercaseRequest if err: = json.NewDecoder (r.Body) .Decode (& request) Err! = nil {return nil, err} return request, nil} func decodeCountRequest (_ context.Context, r * http.Request) (interface {}, error) {var request countRequest if err: = json.NewDecoder (r.Body). Decode (& request) Err! = nil {return nil, err} return request, nil} func encodeResponse (_ context.Context, w http.ResponseWriter, response interface {}) error {return json.NewEncoder (w) .Encode (response)} string SVC1

So far, the complete service is the string SVC1.

$go get github.com/go-kit/kit/examples/stringsvc1 $stringsvc1 $curl-XPOST-d' {"s": "hello, world"} 'localhost:8080/uppercase {"v": "HELLO, WORLD"} $curl-XPOST-d' {"s": "hello, world"}' localhost:8080/count {"v": 12} intermediate software

Without thorough logging and testing, no service can be considered to be available for production.

Separation of concerns

As you continue to increase the number of your service endpoint, we separate the calls of each project into separate service layer files in order to make it easier to read go-kit projects. Our first example [string SVC1] places all these layers in a single master file. Before adding complexity, let's divide the code into the following files and leave all the remaining code in main.go

Include the following functions and types of service.go files for your service.

Type StringServicetype stringServicefunc Uppercasefunc Countvar ErrEmpty

Turn your transport layer into a transport.go file with the following functions and types.

Func makeUppercaseEndpointfunc makeCountEndpointfunc decodeUppercaseRequestfunc decodeCountRequestfunc encodeResponsetype uppercaseRequesttype uppercaseResponsetype countRequesttype countResponse transfer log

Any component that requires logging should have logger as a dependency, just like a database connection. We construct our logger in our func main and pass it to the components that need it. We never use global-scoped logger.

We can pass a logger directly into our stringService implementation, but there is a better method of middleware, also called a decorator, which is a function that accepts the endpoint and returns the endpoint.

Type Middleware func (Endpoint) Endpoint

Note that the middleware type is provided by go-kit.

In between, it can do anything. Below you can see how to implement a basic logging middleware (you don't need to copy / paste this code anywhere):

Func loggingMiddleware (logger log.Logger) Middleware {return func (next endpoint.Endpoint) endpoint.Endpoint {return func (ctx context.Context, request interface {}) (interface {}, error) {logger.Log ("msg", "calling endpoint") defer logger.Log ("msg", "called endpoint") return next (ctx Request)}

To use the go-kit log package and delete the standard library log, you need to remove the log.Fatal from the bottom of the main.go file.

Import ("github.com/go-kit/kit/log")

And connect it to each of our handlers. The following code does not compile until you follow the application logging section, which defines the chapter for loggingMiddleware logging middleware.

Logger: = log.NewLogfmtLogger (os.Stderr) svc: = stringService {} var uppercase endpoint.Endpointuppercase = makeUppercaseEndpoint (svc) uppercase = loggingMiddleware (log.With (logger, "method", "uppercase")) (uppercase) var count endpoint.Endpointcount = makeCountEndpoint (svc) count = loggingMiddleware (log.With (logger, "method", "count")) (count) uppercaseHandler: = httptransport.NewServer (uppercase, / /) countHandler: = httptransport.NewServer (count) / /.)

It turns out that this technology is not only applicable to logging, many Go-kit components are implemented as endpoint middleware.

Application logging

But what if we want to log in our application domain, such as the parameters passed in? In fact, we can define a middleware for our service and get the same good and composable results. Since our StringService is defined as an interface, we just need to create a new type to wrap the existing StringService and perform additional logging tasks.

Type loggingMiddleware struct {logger log.Logger next StringService} func (mw loggingMiddleware) Uppercase (s string) (output string, err error) {defer func (begin time.Time) {mw.logger.Log ("method", "uppercase", "input", s, "output", output "err", err, "took", time.Since (begin),)} (time.Now ()) output Err = mw.next.Uppercase (s) return} func (mw loggingMiddleware) Count (s string) (n int) {defer func (begin time.Time) {mw.logger.Log ("method", "count", "input", s, "n", n "took", time.Since (begin),)} (time.Now ()) n = mw.next.Count (s) return}

Link to our code

Import ("os"github.com/go-kit/kit/log" httptransport "github.com/go-kit/kit/transport/http") func main () {logger: = log.NewLogfmtLogger (os.Stderr) var svc StringService svc = stringService {} svc = loggingMiddleware {logger, svc} / /. UppercaseHandler: = httptransport.NewServer (makeUppercaseEndpoint (svc), / /...) CountHandler: = httptransport.NewServer (makeCountEndpoint (svc), / /...)}

Use endpoint middleware for transport domain problems, such as circuit breakers and rate limiting. Use service middleware for business domain issues, such as logging and detection. Speaking of testing...

Application instrument

In Go-kit, instrumentation means using software package metrics to record statistics on the behavior of the service runtime, to count the number of jobs processed, to record the duration after the request is completed, and to track the number of operations being performed, all of which are considered tools.

We can use the same middleware pattern as logging.

Type instrumentingMiddleware struct {requestCount metrics.Counter requestLatency metrics.Histogram countResult metrics.Histogram next StringService} func (mw instrumentingMiddleware) Uppercase (s string) (output string, err error) {defer func (begin time.Time) {lvs: = [] string {"method", "uppercase", "error" Fmt.Sprint (err! = nil)} mw.requestCount.With (lvs...) .add (1) mw.requestLatency.With (lvs...) .Observe (time.Since (begin). Seconds ())} (time.Now ()) output Err = mw.next.Uppercase (s) return} func (mw instrumentingMiddleware) Count (s string) (n int) {defer func (begin time.Time) {lvs: = [] string {"method", "count", "error" "false"} mw.requestCount.With (lvs...) .add (1) mw.requestLatency.With (lvs...) .Observe (time.Since (begin) .Seconds () mw.countResult.Observe (float64 (n))} (time.Now ()) n = mw.next.Count (s) return}

Connect it to our service.

Import (stdprometheus "github.com/prometheus/client_golang/prometheus kitprometheus" github.com/go-kit/kit/metrics/prometheus "github.com/go-kit/kit/metrics") func main () {logger: = log.NewLogfmtLogger (os.Stderr) fieldKeys: = [] string {"method", "error"} requestCount: = kitprometheus.NewCounterFrom (stdprometheus.CounterOpts {Namespace: "my_group" Subsystem: "string_service", Name: "request_count", Help: "Number of requests received.",}, fieldKeys) requestLatency: = kitprometheus.NewSummaryFrom (stdprometheus.SummaryOpts {Namespace: "my_group", Subsystem: "string_service" Name: "request_latency_microseconds", Help: "Total duration of requests in microseconds.",}, fieldKeys) countResult: = kitprometheus.NewSummaryFrom (stdprometheus.SummaryOpts {Namespace: "my_group", Subsystem: "string_service", Name: "count_result" Help: "The result of each count method.",}, [] string {}) / / no fields here var svc StringService svc = stringService {} svc = loggingMiddleware {logger, svc} svc = instrumentingMiddleware {requestCount, requestLatency, countResult, svc} uppercaseHandler: = httptransport.NewServer (makeUppercaseEndpoint (svc), decodeUppercaseRequest, encodeResponse ) countHandler: = httptransport.NewServer (makeCountEndpoint (svc), decodeCountRequest, encodeResponse,) http.Handle ("/ uppercase", uppercaseHandler) http.Handle ("/ count", countHandler) http.Handle ("/ metrics", promhttp.Handler ()) logger.Log ("msg", "HTTP", "addr" ": 8080") logger.Log ("err", http.ListenAndServe (": 8080", nil))} string SVC2

So far, the complete service is the string SVC2.

$go get github.com/go-kit/kit/examples/stringsvc2 $stringsvc2msg=HTTP addr=:8080$ curl-XPOST-d' {"s": "hello, world"} 'localhost:8080/uppercase {"v": "HELLO, WORLD"} $curl-XPOST-d' {"s": "hello, world"}' localhost:8080/count {"v": 12} method=uppercase input= "hello, world" output= "HELLO, WORLD" err=null took= 2.455 μ smethod = count input= "hello, world" nasty 12 took=743ns call other services

Few services exist in the vacuum without relying on other services. Typically, you need to invoke other services. This is where Go kit shines. We provide transmission middleware to solve many problems that arise.

Suppose we want the string service to invoke different string services to satisfy the uppercase method. In effect, the request is delegated to another service. Let's implement proxy middleware as service middleware, the same as logging or detection middleware.

/ / proxymw implements StringService, forwarding Uppercase requests to the// provided endpoint, and serving all other (i.e. Count) requests via the// next StringService.type proxymw struct {next StringService / / Serve most requests via this service... Uppercase endpoint.Endpoint / /... except Uppercase, which gets served by this endpoint} client endpoint

We get exactly the same endpoint as we already know, but we will use it to invoke instead of the service requesting the client. In order to invoke the client endpoint, we only need to do some simple transformations.

Func (mw proxymw) Uppercase (s string) (string, error) {response, err: = mw.uppercase (uppercaseRequest {S: s}) if err! = nil {return ", err} resp: = response. (uppercaseResponse) if resp.Err! =" {return resp.V, errors.New (resp.Err)} return resp.V, nil}

Now, to build one of these proxy middleware, we convert a proxy URL string to an endpoint.

Import (httptransport "github.com/go-kit/kit/transport/http") func proxyingMiddleware (proxyURL string) ServiceMiddleware {return func (next StringService) StringService {return proxymw {next, makeUppercaseProxy (proxyURL)} func makeUppercaseProxy (proxyURL string) endpoint.Endpoint {return httptransport.NewClient (GET, mustParseURL (proxyURL), encodeUppercaseRequest, decodeUppercaseResponse Endpoint ()} Service Discovery and load balancing

It would be nice if we only had one remote service. But in fact, we may have many service instances available. We want to discover them through some kind of service discovery mechanism and spread our load across all of these instances. If any of these examples starts to perform badly, we have to deal with it without affecting the reliability of our own services.

Go-kit provides adapters for different service discovery systems to obtain the latest set of instances, which are exposed as a single endpoint.

Type Subscriber interface {Endpoints () ([] endpoint.Endpoint, error)}

Internally, the subscriber uses the provided factory function to convert each discovered instance string (usually host: port) to an available endpoint.

Type Factory func (instance string) (endpoint.Endpoint, error)

So far, our factory function makeUppercaseProxy has only called URL directly. However, some safety middleware, such as circuit breakers and speed limiters, should also be installed in the factory.

Var e endpoint.Endpointe = makeUppercaseProxy (instance) e = circuitbreaker.Gobreaker (gobreaker.NewCircuitBreaker (gobreaker.Settings {})) (e) e = kitratelimit.NewTokenBucketLimiter (jujuratelimit.NewBucketWithRate (float64 (maxQPS), int64 (maxQPS)

Now that we have a set of endpoints, we need to choose one. The load balancer wraps the subscriber and selects one of the multiple endpoints. Go kit provides two basic load balancers, and if you need more advanced inspiration, you can easily write your own endpoints.

Type Balancer interface {Endpoint () (endpoint.Endpoint, error)}

Now, we can choose the endpoint based on some inspiration. We can use it to provide users with a single, logical, robust endpoint. Retry the policy wrapper load balancer and return an available endpoint. The retry policy retries the failed request until the maximum number of attempts is reached or the timeout occurs.

Func Retry (max int, timeout time.Duration, lb Balancer) endpoint.Endpoint

Let's connect the last proxy middleware. For simplicity, let's assume that the user will use a flag to specify multiple comma-separated instance endpoints.

Func proxyingMiddleware (instances string, logger log.Logger) ServiceMiddleware {/ / If instances is empty, don't proxy. If the instance is empty, do not proxy if instances = "" {logger.Log ("proxy_to", "none") return func (next StringService) StringService {return next}} / / Set some parameters for our client. Some of our customer's parameters var (qps = 100,100 / beyond which we will return an error maxAttempts = 3 / / per request, before giving up maxTime = 250* time.Millisecond / / wallclock time, before giving up) / / Otherwise, construct an endpoint for each instance in the list And add / / it to a fixed set of endpoints. In a real service, rather than doing this / / by hand, you'd probably use package sd's support for your service / / discovery system. Var (instanceList = split (instances) subscriber sd.FixedSubscriber) logger.Log ("proxy_to", fmt.Sprint (instanceList)) for _ Instance: = range instanceList {var e endpoint.Endpoint e = makeUppercaseProxy (instance) e = circuitbreaker.Gobreaker (gobreaker.NewCircuitBreaker (gobreaker.Settings {})) (e) e = kitratelimit.NewTokenBucketLimiter (jujuratelimit.NewBucketWithRate (float64 (qps), int64 (qps) (e) subscriber = append (subscriber, e)} / / Now, build a single, retrying Load-balancing endpoint out of all of / / those individual endpoints. Balancer: = lb.NewRoundRobin (subscriber) retry: = lb.Retry (maxAttempts, maxTime, balancer) / / And finally, return the ServiceMiddleware, implemented by proxymw. Return func (next StringService) StringService {return proxymw {next, retry} string SVC3

So far, the complete service is the string SVC3.

$go get github.com/go-kit/kit/examples/stringsvc3 $stringsvc3-listen=:8001 & listen=:8001 caller=proxying.go:25 proxy_to=nonelisten=:8001 caller=main.go:72 msg=HTTP addr=:8001 $stringsvc3-listen=:8002 & listen=:8002 caller=proxying.go:25 proxy_to=nonelisten=:8002 caller=main.go:72 msg=HTTP addr=:8002 $stringsvc3-listen=:8003 & listen=:8003 caller=proxying.go:25 proxy_to=nonelisten=:8003 caller=main.go:72 msg=HTTP addr=:8003 $stringsvc3-listen=:8080-proxy=localhost:8001,localhost:8002 Localhost:8003listen=:8080 caller=proxying.go:29 proxy_to= "[localhost:8001 localhost:8002 localhost:8003]" listen=:8080 caller=main.go:72 msg=HTTP addr=:8080$ for s in foo bar baz Do curl-d "{\" s\ ":\" $s\ "}" localhost:8080/uppercase Done {"v": "FOO"} {"v": "BAR"} {"v": "BAZ"} listen=:8001 caller=logging.go:28 method=uppercase input=foo output=FOO err=null took= 5.168 μ s =: 8080 caller=logging.go:28 method=uppercase input=foo output=FOO err=null took=4.39012mslisten=:8002 caller=logging.go:28 method=uppercase input=bar output=BAR err=null took= 5.445 μ s =: 8080 caller=logging.go:28 method=uppercase input=bar output=BAR err=null took=2.04831mslisten=:8003 caller=logging.go:28 method=uppercase input=baz output=BAZ err=null took= 3.285 μ s =: 8080 caller=logging.go:28 method=uppercase input=baz output=BAZ err=null took=1.388155ms Advanced topic interspersed context

The context object is used to pass information across conceptual boundaries within the scope of a single request. In our example, we have not yet penetrated the context into the business logic. But it's almost always a good idea. It allows you to pass requested information between business logic and middleware, which is a complex task necessary to achieve more functionality, such as fine-grained distributed tracking annotations.

Specifically, this means that your business logical interface looks like

Type MyService interface {Foo (context.Context, string, int) (string, error) Bar (context.Context, string) error Baz (context.Context) (int, error)} distributed tracking

Once your infrastructure has grown to a certain size, it becomes important to track requests through multiple services so that you can identify hotspots and troubleshoot. See [package tracking] package tracing for more information.

Create a client package

You can use Go kit to create a client package for your service to make it easier to use your service from other Go programs. In fact, your client package will provide an implementation of the service interface that invokes the remote service instance using a specific transport. See addsvc/cmd/addcli or package profilesvc/clien for example.

About how to get started on go-kit to share here, I hope that the above content can make you improve. If you want to learn more knowledge, please pay more attention to the editor's updates. Thank you for following the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.