Shulou(Shulou.com)06/03 Report--

1. Role description of the cluster:

Role

Description

Leader (leader)

The leader is responsible for publishing decisions, mainly dealing with write requests

Follower (follower)

Follower is used to receive client requests and return results to the client (only read requests can be processed, if a write request is received

Forward the write request to leader), when the leader goes down, initiates the elector, has the right to vote and be voted

Observer (observer)

Observer can receive client connections and forward write requests to leader nodes to help follower reduce the pressure of reading.

But Observer does not participate in elections and be elected. The directory of Observer is designed to expand the system and improve the reading speed

Client

Request initiator

2. The selection process of zookeeper

(1) the choice of the new cluster:

Select the master according to the order of startup and id (more than half mechanism: the master is selected only when more than half of the clusters in the cluster are available)

Take hadoop01 (id=1)-hadoop02 (id=2)-hadoop03 (id=3)-hadoop04 (id=r)-hadoop05 (id=5) as an example:

When hadoop01 starts, it has only one server at this time, and the report it fetches does not correspond to anything, and all its elections have been looking status.

Hadoop02 startup: it communicates with the original hadoop01 and exchanges its election results with each other. Since neither of them has historical data, the server with a higher id value wins, but because more than half of the servers agree to elect it (more than half of the servers in this example are 3), all hadoop01 and hadoop02 still remain looking.

Hadoop03 started, and according to the previous analysis, server 3 became the boss of server 1, Hadoop03 2, and 3, but unlike the above, three servers (more than half) elected it, so server 3 became the leader of this election.

Start with hadoop04. According to the above analysis, hadoop04 should be the largest id in the server, but hadoop03 has been elected by nearly half of the servers, so hadoop04 can only be follower.

Hadoop05 startup, like hadoop04, is also a follower

There are three working states of zookeeper server:

LOOKING: currently, Server doesn't know who leader is, it's searching, it's electing.

LEADING: the current Server is the elected leader, which is responsible for coordinating affairs

FOLLOWING:leader has been elected, and the current Server is synchronized with it and obeys the orders of leader

(2) the selection of non-brand-new clusters

Leader has been elected, and the current Server is synchronized with it and obeys the command of leader, but the primary node is down for some reason:

At this time, we choose the master according to three dimensions: data version, serverid, and logical clock.

Data version: the new version of data is large. Every time the data is updated, its version will be updated at the same time.

Serverid: this is the value in the myid we configured, one for each machine.

Logical clock: this value starts from 0, and each election corresponds to a value. That is to say, if in the same election, this value should be consistent, the larger the logical clock, it means that the process of this election leader person is updated, that is, each election has a zxid, and the voting result only takes the latest zxid.

Election criteria:

The election result with a small logic clock is ignored and revoted.

Big win of data version after unified logic clock

The logic clock is unified, version is the same, and server id wins.

According to the above rules, quickly select the master node of the cluster.

3. Zookeeper's process of writing data:

The client sends a request to write data, which is eventually processed by leader

Leader will write the data first, and notify follower to synchronize the data after the write is completed.

Follower will start data synchronization (parallel, multiple follower parallel synchronization)

Each follower will send a data synchronization success message to leader as soon as the data synchronization is completed.

After receiving more than half of the successful messages, leader thinks that the data is written successfully this time.

Other nodes synchronize slowly and do not provide read and write services in the process of data synchronization.

4. Synchronization process of zookeeper data

Follower connects to leader and sends its largest zixd

Leader compares its largest zxid with the zxid sent by follower. If the zxid of leader is greater than that of follower, notify follower for data synchronization.

Follower sends data synchronization request

Leader determines the data synchronization point of the current follower (data needs to be synchronized from the largest zxid in follower to the largest zxid in leader)

Follower begins to synchronize data, and this process does not provide read and write services.

Follower synchronization completed, send message to leader

Leader will change the status of the current follower to update. At this time, follower can accept read and write requests from the client, but can only read. If it is a write request, it needs to be forwarded to leader.

5. Job responsibilities of each role in ZooKeeper

(1) Leader

Recover data

Maintain the heartbeat with follower, receive follower requests and determine the request message type of follower

Different processing is performed according to different message types.

(2) follower

Send a request to leader (synchronous data, write request)

Receive the message from leader and process it accordingly

Receive read and write requests from client. If the request is written, it is forwarded to leader for processing.

Return the read request of client and the result of the query

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.