Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to use Consul to achieve distributed system coordination in C#. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

When coordinating distributed applications, you will mainly encounter the following application scenarios:

Service discovery (service discovery)

Find out which services and nodes are available in the distributed system

Name Service (name service)

Know the corresponding resource by the given name

Configuration Management (configuration management)

How to share configuration files in distributed nodes to ensure consistency.

Fault discovery and failover (failure detection and failover)

When a node fails, how to detect and notify other nodes, or transfer the desired services to other available nodes

Leadership election (leader election)

How to elect a leader among many nodes to coordinate all nodes

Distributed locks (distributed exclusive lock)

How to synchronize in distributed services through locks

Message and Notification Service (message queue and notification)

How to deliver messages in distributed services and respond proactively to events in the form of notifications

Consul

Consul is a distributed service coordination and management tool developed with Go. It provides service discovery, health check, Key/Value storage and other functions, and supports cross-data center functions. Some key features provided by consul:

Service discovery:consul makes it easy to register and discover services through DNS or HTTP interfaces, as can some external services, such as those provided by saas.

Health checking: health check enables consul to quickly alert operations in the cluster. Integration with service discovery can prevent services from being forwarded to failed services.

Key/value storage: a system for storing dynamic configurations. Provides a simple HTTP interface that can be operated anywhere.

Multi-datacenter: no complex configuration is required to support any number of areas.

Consul API based on HTTP can be easily bound with various languages, while C # language binds https://github.com/PlayFab/consuldotnet.

Consul runs an Agent on each node on the Cluster, and this Agent can use either Server or Client mode. Client is responsible for efficient communication to Server and is relatively stateless. Server is responsible for electing leadership nodes, maintaining the status of cluster, responding to all queries, communicating across data centers, and so on.

The consul website already has compiled binaries that support a variety of platforms: win, linux, etc., download packages that match your platform: here, download the package: 0.5.2_windows_386.zip. After decompression, there is only one consul file.

D:\ GitHub\ consuldotnet\ Consul.Test > consul

Usage: consul [--version] [--help] []

Available commands are:

Agent Runs a Consul agent

Configtest Validate config file

Event Fire a new event

Exec Executes a command on Consul nodes

Force-leave Forces a member of the cluster to enter the "left" state

Info Provides debugging information for operators

Join Tell Consul agent to join cluster

Keygen Generates a new encryption key

Keyring Manages gossip layer encryption keys

Leave Gracefully leaves the Consul cluster and shuts down

Lock Execute a command holding a lock

Maint Controls node or service maintenance mode

Members Lists the members of a Consul cluster

Monitor Stream logs from a Consul agent

Reload Triggers the agent to reload configuration files

Version Prints the Consul version

Watch Watch for changes in Consul

After consul is installed, agent can be started, agent can run in server or client mode, and at least one agent in each data center runs in server mode. It is generally recommended to have 3 or 5 server. Deploying a single server is very bad because data loss is inevitable in a failure scenario. This article covers the creation of a new data center where all other agents runs in client mode, a very lightweight service registration process that runs health monitoring and forwards query results to the service. Agent must run on every node in the cluster.

Consul.exe agent-config-file test_config.json

Let's first run an agent in server mode:

D:\ GitHub\ consuldotnet\ Consul.Test > consul.exe agent-config-file test_config.json

= > WARNING: Bootstrap mode enabled! Do not enable unless necessary

WARNING: Windows is not recommended as a Consul server. Do not use in production.

= > WARNING: It is highly recommended to set GOMAXPROCS higher than 1

= = > Starting Consul agent...

= = > Starting Consul agent RPC...

= > Consul agent running!

Node name: 'GEFFZHANG-NB'

Datacenter: 'dc1'

Server: true (bootstrap: true)

Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS:-1, DNS: 8600, RPC: 8400)

Cluster Addr: 192.168.1.4 (LAN: 8301, WAN: 8302)

Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false

Atlas:

= > Log data will now stream in as it occurs:

2015-08-09 09:14:48 [INFO] serf: EventMemberJoin: GEFFZHANG-NB 192.168.1.4

2015-08-09 09:14:48 [INFO] serf: EventMemberJoin: GEFFZHANG-NB.dc1 192.168.1.4

2015-08-09 09:14:48 [INFO] raft: Node at 192.168.1.4:8300 [Follower] entering Follower state

09:14:48 on 2015-08-09 [INFO] consul: adding server GEFFZHANG-NB (Addr: 192.168.1.4 adding server GEFFZHANG-NB) (DC: dc1)

09:14:48 on 2015-08-09 [INFO] consul: adding server GEFFZHANG-NB.dc1 (Addr: 192.168.1.4 adding server GEFFZHANG-NB.dc1) (DC: dc1)

2015-08-09 09:14:48 [ERR] agent: failed to sync remote state: No cluster leader

2015-08-09 09:14:50 [WARN] raft: Heartbeat timeout reached, starting election

2015-08-09 09:14:50 [INFO] raft: Node at 192.168.1.4:8300 [Candidate] entering Candidate state

2015-08-09 09:14:50 [DEBUG] raft: Votes needed: 1

2015-08-09 09:14:50 [DEBUG] raft: Vote granted. Tally: 1

2015-08-09 09:14:50 [INFO] raft: Election won. Tally: 1

2015-08-09 09:14:50 [INFO] raft: Node at 192.168.1.4:8300 [Leader] entering Leader state

2015-08-09 09:14:50 [INFO] consul: cluster leadership acquired

2015-08-09 09:14:50 [INFO] consul: New leader elected: GEFFZHANG-NB

09:14:50 on 2015-08-09 [INFO] raft: Disabling EnableSingleNode (bootstrap)

2015-08-09 09:14:50 [DEBUG] raft: Node 192.168.1.4 updated peer set 8300 updated peer set (2): [192.168.1.4 Node 8300]

2015-08-09 09:14:50 [DEBUG] consul: reset tombstone GC to index 2

09:14:50 on 2015-08-09 [INFO] consul: member 'GEFFZHANG-NB' joined, marking health alive

09:14:51 on 2015-08-09 [INFO] agent: Synced service 'consul'

09:16:03 on 2015-08-09 [DEBUG] agent: Service 'consul' in sync

09:17:30 on 2015-08-09 [DEBUG] agent: Service 'consul' in sync

09:18:38 on 2015-08-09 [DEBUG] agent: Service 'consul' in sync

09:19:47 on 2015-08-09 [DEBUG] http: Request / v1/status/peers (0)

09:19:52 on 2015-08-09 [DEBUG] agent: Service 'consul' in sync

As you can see, consul agent has started and printed some logs to the terminal, from which you can see that our agent is already running in server mode and is the lead node of the entire cluster. Also, local members have been marked as healthy members in the cluster. At this point, you can see the members of the entire cluster by running consul members in another terminal. At this time you can only see yourself, because no other members have been added to our cluster.

The output already shows your own node information, including address information, health status, role in the cluster, and some version information. If you want to view some metadata, you can add the-detailed tag.

The information output by the consul members command is based on the gossip protocol and is ultimately consistent.

Basic operation of KV

Consul provides a simple build V storage system that can be used to dynamically retrieve configuration, coordinate services, elect master nodes, other developers can think of, and so on.

Var client = new Client ()

Var kv = client.KV

Var key = GenerateTestKeyName ()

Var value = Encoding.UTF8.GetBytes ("test")

Var getRequest = kv.Get (key)

Assert.IsNull (getRequest.Response)

Var pair = new KVPair (key)

{

Flags = 42

Value = value

}

Var putRequest = kv.Put (pair)

Assert.IsTrue (putRequest.Response)

GetRequest = kv.Get (key)

Var res = getRequest.Response

Assert.IsNotNull (res)

Assert.IsTrue (StructuralComparisons.StructuralEqualityComparer.Equals (value, res.Value))

Assert.AreEqual (pair.Flags, res.Flags)

Assert.IsTrue (getRequest.LastIndex > 0)

Var del = kv.Delete (key)

Assert.IsTrue (del.Response)

GetRequest = kv.Get (key)

Assert.IsNull (getRequest.Response)

Service Discovery (Service Discovery) and Health check (Health Check)

Another major function of Consul is to manage distributed services, where users can register for a service and provide the ability to perform health checks on the service.

Service definition: a service can dynamically add, delete, or modify services by providing service definition profiles or by calling HTTP API.

Service query: once the agent is started and the service is synchronized, we can use DNS or HTTP API to query.

Service upgrade: the service definition can be upgraded by modifying the service definition configuration file and then sending a SIGHUP signal to agent, which allows you to upgrade the service without agent downtime or unreachable service. Or through the HTTP API interface to dynamically add, delete, modify services.

Consul supports three modes of Check:

Call an external script (Script). In this mode, consul periodically invokes an external script to obtain the health status of the corresponding service through the return of the script.

Call HTTP, in which consul periodically invokes a HTTP request and returns 2XX for health; 429 (Too many request) is a warning. Everything else is unhealthy.

Actively report. In this mode, the service needs to actively invoke a HTTP PUT request provided by consul to report the health status.

C # API provides the corresponding interface

Client.Agent.Service

Client.Agent.Check

Consul's Health Check, which checks the status of the service by calling scripts, HTTP or active reporting, is more flexible and can obtain more information, but more work needs to be done.

Fault detection (Failure Detection)

Consul provides the concept of Session, and Session can be used to check whether the service is alive or not. We can create a session object for each service. Note that we set ttl,consul to check the survival of session continuously with the value of ttl as the interval. Accordingly, in the service, we need a continuous renew session to ensure that the session is legal.

Var client = new Client ()

Var sessionRequest = client.Session.Create (new SessionEntry () {TTL = TimeSpan.FromSeconds (10)})

Var id = sessionRequest.Response

Assert.IsTrue (sessionRequest.RequestTime.TotalMilliseconds > 0)

Assert.IsFalse (string.IsNullOrEmpty (sessionRequest.Response))

Var tokenSource = new CancellationTokenSource ()

Var ct = tokenSource.Token

Client.Session.RenewPeriodic (TimeSpan.FromSeconds (1), id, WriteOptions.Empty, ct)

TokenSource.CancelAfter (3000)

Task.Delay (3000, ct) .Wait (ct)

Var infoRequest = client.Session.Info (id)

Assert.IsTrue (infoRequest.LastIndex > 0)

Assert.IsNotNull (infoRequest.KnownLeader)

Assert.AreEqual (id, infoRequest.Response.ID)

Assert.IsTrue (client.Session.Destroy (id) .Response)

Note here that because it is based on ttl (minimum 10 seconds) detection, there is a delay of at least 10 seconds from service interruption to detection, which is not applicable for scenarios that require real-time response.

Leadership elections and distributed locks

This document introduces how to use the KV storage of Consul to achieve Leader Election, using the KV function of Consul, it is very convenient to achieve the function of leader election and lock.

WEB UI

Consul also supports the web interface, which can be used to view all services and nodes, all health checks and their current status, and read the values that set the K _ web V system. UI automatically supports multiple datacenter by default. These UI are static html you do not need to run a separate web server, consul agent itself can be configured with a web service.

Download the UI component: WEB UI

After the download is completed, there is a 0.5.2_web_ui.zip compressed file, and when unzipped, there is a dist directory. Then restart agent by adding the-ui-dir parameter and the-client parameter.

D:\ GitHub\ consuldotnet\ Consul.Test > consul.exe agent-config-file test_config.json-ui-dir=D:\ GitHub\ consuldotnet\ Consul.Test\ 0.5.2_web_ui\ dist

Enter http://127.0.0.1:8500 in the browser to access UI

There are services, nodes, Kramp V, acl, datacenter management, a very perfect system.

This is how to use Consul to achieve distributed system coordination in C# shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.