Shulou(Shulou.com)06/03 Report--

Editor to share with you how to operate Go etcd, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to understand it!

Etcdetcd introduction

Etcd is an open source, highly available distributed key-value storage system developed in the Goto language, which can be used to configure sharing and service registration and discovery.

Similar projects include zookeeper and consul.

Etcd has the following characteristics:

Full replication: every node in the cluster can use a full archive

High availability: Etcd can be used to avoid a single point of hardware failure or network problems

Consistency: each read returns the latest writes across multiple hosts

Simple: includes a well-defined, user-oriented API (gRPC)

Security: automated TLS with optional client certificate authentication

Fast: base speed of 10000 writes per second

Reliable: use Raft algorithm to achieve strong consistency and high availability of service storage catalog

Etcd Application scenario Service Discovery

Service discovery is also one of the most common problems in distributed systems, that is, how can processes or services in the same distributed cluster find each other and establish connections. In essence, service discovery wants to know if there are processes in the cluster listening on udp or tcp ports, and can find and connect by name.

Configuration center

Put some configuration information on etcd for centralized management.

Such scenarios are usually used like this: when the application starts, it takes the initiative to obtain configuration information from etcd. At the same time, it registers a Watcher on the etcd node and waits. Each time the configuration is updated, etcd will notify the subscriber in real time, so as to obtain the latest configuration information.

Distributed lock

Because etcd uses the Raft algorithm to maintain strong data consistency, the values stored in the cluster at a time must be globally consistent, so it is easy to implement distributed locks. The lock service can be used in two ways, one is to remain exclusive, the other is to control the timing.

Keep it exclusive, that is, only one of all users who acquire the lock will eventually get it. For this reason, etcd provides a set of API to implement distributed lock atomic operation CAS (CompareAndSwap). By setting the exit value, you can ensure that only one is successful when multiple nodes go to create a directory at the same time. The successful user can be considered to have acquired the lock.

Control timing, that is, all users who want to acquire locks are scheduled to execute, but the order in which locks are acquired is also globally unique and determines the order of execution. Etcd also provides a set of API (automatically create ordered keys), which is specified as a POST action when creating a value for a directory, so that etcd will automatically generate a current maximum value under the directory as the key and store the new value (client number). You can also use API to list all the key values in the current directory in order. At this point, the values of these keys are the timing of the client, and the values stored in these keys can be numbers that represent the client.

Why use etcd instead of ZooKeeper?

All these functions realized by etcd can be realized by ZooKeeper. So why use etcd instead of using ZooKeeper directly?

Why not choose ZooKeeper?

Deployment and maintenance are complex, and the Paxos strong consistency algorithm used is complex and difficult to understand. The official interface is only provided for Java and C languages.

Writing in Java introduces a large number of dependencies. It is troublesome for the operation and maintenance personnel to maintain.

The development has been slow in recent years, not as good as rising stars such as etcd and consul.

Why choose etcd?

simple. Using Go language to write and deploy is simple; supporting HTTP/JSON API, easy to use; using Raft algorithm to ensure strong consistency so that users can understand easily.

Etcd default data is persisted as soon as it is updated.

Etcd supports SSL client security authentication.

Finally, as a young project, etcd is in the process of high-speed iteration and development, which is both an advantage and a disadvantage. The advantage is that it has unlimited possibilities in the future, and the disadvantage is that it can not be tested by the long-term use of large projects. However, well-known projects such as CoreOS, Kubernetes, and CloudFoundry all use etcd in a production environment, so overall, etcd is worth a try.

Etcd cluster

As a highly available key storage system, etcd is naturally designed for clustering. Because the Raft algorithm requires the majority of nodes to vote when making decisions, etcd generally recommends an odd number of nodes in a cluster, and the recommended number of nodes is 3, 5 or 7 nodes to form a cluster.

Set up an example of a 3-node cluster:

Cluster members are specified in each etcd node, and it is best to configure a unique token in order to distinguish between different clusters.

Here is the pre-defined cluster information, where N1, N2, and n3 represent three different etcd nodes.

TOKEN=token-01CLUSTER_STATE=newCLUSTER=n1= http://10.240.0.17:2380,n2=http://10.240.0.18:2380,n3=http://10.240.0.19:2380

Execute the following command on the N1 machine to start etcd:

Etcd-data-dir=data.etcd-- name N1\-- initial-advertise-peer-urls http://10.240.0.17:2380-- listen-peer-urls http://10.240.0.17:2380\-- advertise-client-urls http://10.240.0.17:2379-- listen-client-urls http://10.240.0.17:2379\-- initial-cluster ${CLUSTER}\-- initial- Cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN}

On the N2 machine, execute the following command to start etcd:

Etcd-data-dir=data.etcd-- name N2\-- initial-advertise-peer-urls http://10.240.0.18:2380-- listen-peer-urls http://10.240.0.18:2380\-- advertise-client-urls http://10.240.0.18:2379-- listen-client-urls http://10.240.0.18:2379\-- initial-cluster ${CLUSTER}\-- initial- Cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN}

On the machine N3, execute the following command to start etcd:

Etcd-data-dir=data.etcd-- name N3\-- initial-advertise-peer-urls http://10.240.0.19:2380-- listen-peer-urls http://10.240.0.19:2380\-- advertise-client-urls http://10.240.0.19:2379-- listen-client-urls http://10.240.0.19:2379\-- initial-cluster ${CLUSTER}\-- initial- Cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN}

The etcd official website provides an etcd storage address that can be accessed by the public network. You can get the directory of the etcd service with the following command and use it as the-discovery parameter.

Curl https://discovery.etcd.io/new?size=3https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92 # grab this tokenTOKEN=token-01CLUSTER_STATE=newDISCOVERY= https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92 etcd-data-dir=data.etcd-- name N1\-- initial-advertise-peer-urls http://10.240.0.17:2380-- listen-peer-urls http://10.240.0.17:2380\-- advertise-client-urls http://10.240.0.17 2379-- listen-client-urls http://10.240.0.17:2379\-- discovery ${DISCOVERY}\-- initial-cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN} etcd-- data-dir=data.etcd-- name N2\-- initial-advertise-peer-urls http://10.240.0.18:2380-- listen-peer-urls http://10.240.0.18:2380\ -- advertise-client-urls http://10.240.0.18:2379-- listen-client-urls http://10.240.0.18:2379\-- discovery ${DISCOVERY}\-- initial-cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN} etcd-- data-dir=data.etcd-- name n3\-- initial-advertise-peer-urls http://10.240.0.19:2380-- -listen-peer-urls http://10.240.0.19:2380\-- advertise-client-urls http://10.240.0.19:2379-- listen-client-urls http:/10.240.0.19:2379\-- discovery ${DISCOVERY}\-- initial-cluster-state ${CLUSTER_STATE}-- initial-cluster-token ${TOKEN}

At this point, the etcd cluster is set up, and you can use etcdctl to connect to the etcd.

Export ETCDCTL_API=3HOST_1=10.240.0.17HOST_2=10.240.0.18HOST_3=10.240.0.19ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379 etcdctl-endpoints=$ENDPOINTS member lisGo language operating etcd

The official etcd/clientv3 package is used here to connect to etcd and perform related operations.

Install go get go.etcd.io/etcd/clientv3put and get operation

The put command is used to set key-value pairs of data, and the get command is used to obtain values based on key.

Package main import ("context"fmt"time"go.etcd.io/etcd/clientv3") / / etcd client put/get demo// use etcd/clientv3 func main () {cli, err: = clientv3.New (clientv3.Config {Endpoints: [] string {"127.0.0.1 time 2379"}, DialTimeout: 5 * time.Second,}) if err! = nil {/ / handle error! Fmt.Printf ("connect to etcd failed, err:%v\ n", err) return} fmt.Println ("connect to etcd success") defer cli.Close () / / put ctx, cancel: = context.WithTimeout (context.Background (), time.Second) _, err = cli.Put (ctx, "q1mi", "dsb") cancel () if err! = nil {fmt.Printf ("put to etcd failed, err:%v\ n" Err) return} / / get ctx, cancel = context.WithTimeout (context.Background (), time.Second) resp, err: = cli.Get (ctx, "q1mi") cancel () if err! = nil {fmt.Printf ("get from etcd failed, err:%v\ n", err) return} for _, ev: = range resp.Kvs {fmt.Printf ("% soren% s\ n", ev.Key Ev.Value)}} watch operation

Watch is used to get notifications of future changes.

Package main import ("context"fmt"time"go.etcd.io/etcd/clientv3") / / watch demo func main () {cli, err: = clientv3.New (clientv3.Config {Endpoints: [] string {"127.0.0.1 Endpoints 2379"}, DialTimeout: 5 * time.Second,}) if err! = nil {fmt.Printf ("connect to etcd failed, err:%v\ n") Err) return} fmt.Println ("connect to etcd success") defer cli.Close () / / watch key:q1mi change rch: = cli.Watch (context.Background () "q1mi") / / etcdctl.exe-- endpoints= http://127.0.0.1:2379 put q1mi "dsb2" OK etcd > etcdctl.exe-- endpoints= http://127.0.0.1:2379 del q1mi1 etcd > etcdctl.exe-- endpoints= http://127.0.0.1:2379 put q1mi "dsb3" OK

All the above programs can receive the following notification.

Watch > watch.execonnect to etcd successType: PUT Key:q1mi Value:dsb2Type: DELETE Key:q1mi Value:Type: PUT Key:q1mi Value:dsb3

Lease lease

Package main import ("fmt"time") / / etcd lease import ("context"log"go.etcd.io/etcd/clientv3") func main () {cli, err: = clientv3.New (clientv3.Config {Endpoints: [] string {"127.0.0.1 VRV 2379"}, DialTimeout: time.Second * 5 }) if err! = nil {log.Fatal (err)} fmt.Println ("connect to etcd success.") Defer cli.Close () / / create a 5-second lease resp, err: = cli.Grant (context.TODO (), 5) if err! = nil {log.Fatal (err)} / / 5 seconds later, / nazha/ the key will be removed _, err = cli.Put (context.TODO (), "/ nazha/", "dsb") Clientv3.WithLease (resp.ID)) if err! = nil {log.Fatal (err)}}

KeepAlive

Package main import ("context", "fmt", "log", "time"go.etcd.io/etcd/clientv3") / / etcd keepAlive func main () {cli, err: = clientv3.New (clientv3.Config {Endpoints: [] string {"127.0.0.1 log 2379"}, DialTimeout: time.Second * 5 }) if err! = nil {log.Fatal (err)} fmt.Println ("connect to etcd success.") Defer cli.Close () resp, err: = cli.Grant (context.TODO (), 5) if err! = nil {log.Fatal (err)} _, err = cli.Put (context.TODO (), "/ nazha/", "dsb", clientv3.WithLease (resp.ID)) if err! = nil {log.Fatal (err)} / / the key 'foo' will be kept forever ch Kaerr: = cli.KeepAlive (context.TODO (), resp.ID) if kaerr! = nil {log.Fatal (kaerr)} for {ka: =

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.