Shulou(Shulou.com)05/31 Report--

This article introduces you how to carry out etcd analysis, the content is very detailed, interested friends can refer to, hope to be helpful to you.

Mechanism concept

The algorithm adopted by Raft:etcd to ensure the strong consistency of distributed system

Node: an instance of a Raft state machine

Member: an etcd instance that manages a Node and can serve client requests

Cluster: an etcd cluster composed of multiple Member that can work together

Peer: a term for another Member in the same etcd cluster

Client: the client that sends the HTTP request to the etcd cluster

WAL: pre-written log, log format used by etcd for persistent storage

Snapshot:etcd prevents snapshots set by too many WAL files and stores the state of etcd data

A mode of Proxy:etcd that provides reverse proxy services for etcd clusters

In Leader:Raft, a node generated by election to handle all data submissions.

Follower: as subordinate nodes in Raft, the losing node provides strong consistency guarantee for the algorithm.

Candidate: when Follower fails to receive the heartbeat of Leader for more than a certain period of time, it is changed to Candidate to start the campaign.

Term: a node becomes a Leader until the next election time, which is called a Term

Index: data item number Raft uses Term and Index to locate data

Raft of Etcd

Because the Raft algorithm requires the majority of nodes to vote when making decisions, etcd generally recommends an odd number of nodes in a cluster, and the recommended number of nodes is 3, 5 or 7 nodes to form a cluster.

Brain fissure solution

Introduce a new concept, region leader

Region leader is a logical concept.

There must be only one region leader for a region at any given time.

Each region leader tries to update region leader's lease within raft group at t intervals during its term of office.

All read and write requests must be completed through region leader

It is worth noting, however, that region leader and raft leader may not be the same node. When region leader and raft leader do not coincide, region leader forwards the request to the current raft leader

When a network is partitioned, the following situations occur:

Region leader falls on the majority side, while the old raft leader falls on the majority side.

Region leader falls on the side of the majority, while the old raft leader falls on the side of the minority.

Region leader falls in the minority, and the old raft leader is on the side of the majority.

Region leader is on the minority side, and old raft leader is on the minority side.

Storage (v3)

V3 is so different from the previous version that it only prevails.

Backend:BoltDB

Basic characteristics

Single machine

Support transaction

Key-value pair

Transaction processing

Add, delete, read: all need to get a transaction

Read-only transactions: read-only, can only find Bucket and K _ swap V

Read-write transaction: readable and writable

At the end: if there is no exception, the transaction is committed

There is an exception: abandon the whole transaction

MVCC structure hierarchy

Revision: each modification to the same key corresponds to a revision-main: each transaction + 1-sub: each operation on the same transaction + 1 generation:-an key may be frequently deleted during the life cycle-all revision collections created from a certain time to that deletion form a generation key: each key is composed of multiple generation versions of keyIndex: the structure of organizing this multi-version key is called keyIndex

Maintain

Multiple versions are maintained at the same time and need to be cleaned from time to time

Delete the old version

Ectd provides command line tools

Implement type keyIndex struct {key [] byte modified revision// latest version revision generations [] generation / / Generation} type generation struct {ver int64 created revision// create the first version of this generation revs [] revision// version array} type revision struct {main int64 / / transaction id, global increment sub int64 / / intra-transaction increment}

Example insert data

(key1, value1) (key2, value2)

Update data

(key1, update1) (key2, update2)

Stored record

Rev= {10}, key=key1, value= "value1" rev= {10}, key=key2, value= "value2" rev= {20}, key=key1, value= "update1" rev= {21}, key=key2, value= "update2" memory index: KeyIndex

Basic characteristics

Open source based on Google

Based on btree

module

HTTP Server: used to handle API requests sent by users and requests for synchronization and heartbeat information from other etcd nodes.

Store: used to handle transactions of various functions supported by etcd, including data indexing, node state change, monitoring and feedback, event handling and execution, etc., is the concrete implementation of most of the API functions provided by etcd to users.

The implementation of Raft:Raft strong consistency algorithm is the core of etcd.

WAL:Write Ahead Log (pre-written log) is the data storage method of etcd.

In addition to storing the state of all data in memory and the index of the node, etcd persists through WAL

In WAL, all data is logged before it is submitted

Snapshot is a status snapshot to prevent too much data.

Entry indicates the specific contents of the stored log

This is the end of the analysis on how to carry out etcd. I hope the above content can be helpful to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.