Shulou(Shulou.com)05/31 Report--

This article will explain the example analysis of ceph monitor for you in detail. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

Ceph monitor analysis

A brief introduction to monitor

As the core service of ceph cluster, monitor is responsible for self-maintaining the health status of ceph cluster, rados cluster metadata mdsmap, monmap, osdmap, log, auth, health and so on. Based on the improved paxos algorithm, it ensures the consistency of data state among cluster nodes at the same time. Generally speaking, Monitor is responsible for collecting cluster information, updating cluster information, and publishing cluster information.

Figure 1 monito architecture diagram

You can see that the interior of monitor is actually a distributed kv database. From the bottom up are MonitorDBStore, Paxos, and PaxosService. PaxosService is responsible for ensuring that only one proposal enters the paxos process at a time. The Paxos module implements the multi-Paxos algorithm. MonitorDBStore is an abstract encapsulation of the underlying DB, encapsulating the basic operation transactions of DB into a unified interface. The current DB uses rocksdb by default.

II. Monitor startup process

The startup process of monitor service mainly includes: loading basic configuration, checking and loading db, initializing network module, mon registering network monitoring, and monitor bootstrap startup.

Figure 3 prepare phase a

Figure 5 accept phase a

Figure 7 learn phase

2) Multi paxos:

The original Paxos algorithm (Basic Paxos) can only make a decision on one value, and the formation of the decision requires at least two network back and forth. In high concurrency cases, it may require more network back and forth, and in extreme cases, it may even form a livelock. If you want to determine multiple values in succession, Basic Paxos can't handle it. Therefore, Basic Paxos is almost only used for theoretical research, and is not directly used in practical engineering. In practical applications, it is almost necessary to determine multiple values continuously, and we hope to have higher efficiency. Multi-Paxos is proposed to solve this problem. Multi-Paxos makes two improvements based on Basic Paxos:

1. Run an instance of Paxos algorithm (Instance) for each value to be determined to form a decision.

2. Elect a Leader among all the Proposers, and the Leader uniquely submits the Proposal to the Acceptors for voting. In this way, there is no Proposer competition, and the problem of live lock is solved. When there is only one Leader in the system for Value commit, the Prepare phase can be skipped, thus turning the two phases into one phase and improving efficiency.

Ceph monitor uses the Multi paxos algorithm.

Monitor election process

Monitor::bootstrap () is the election entrance, and the whole process is made up of a series of state changes. After each Monitor starts, according to the host ip list in the configuration file, find other monitor and get the latest log version number of other nodes, determine whether it is necessary to pull db data from other nodes for synchronization according to the version number, then select leader and peon, then do recovery, and finally shutdown according to the received signal.

Fig. 9 probe process

After the synchronization is completed, the election starts if the number of replies exceeds half of the number of nodes, and the node enters the STATE_ELECTING state. First broadcast the OP_PROPOSE message, the peer receives the message, and if it finds that its rank is greater than the rank value of the sender, it replies to the OP_ACK message. In this way, the node with the smallest rank will receive the most ack messages. If the number of ack messages received is the same as the number of current active nodes, the local node becomes leader, and then notifies each node that it wins, and the other nodes will become peon.

Figure 11 recovery process

The commit process that implements paxos in Monitor is done by begin (). Leader first updates its pending_v and pending_pn, and then sends an OP_BEGIN message to all peon nodes. If the accepted_pn of peon is less than that sent by leader, accept the proposal, save the data locally, and reply to the OP_ACCEPT message. If the number of accept received by Leader is equal to the quorum of the cluster, the proposal is submitted locally to the leader and each peon is notified to complete the commit operation. At the same time, update the lease extend_lease (). After renewing the lease, there will be a scheduled task between leader and peon. Leader will renew the lease every 3 seconds by default, and if it expires for 10 seconds, it will be re-elected. Peon also does not receive a lease message, and it will be re-elected after 10 seconds.

Figure 12 proposal submission process

As you can see, the launch of monitor has gone through the process of probe, sync data, election and recovery, in which recovery involves the submission of proposals, and all subsequent monitor proposals are submitted through begin (). From this point of view, the first phase of paxos is completed in bootstrap (), and subsequent proposals can be submitted directly through leader to peon, in line with the description of multi-paxos.

This is the end of this article on "sample Analysis of ceph monitor". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.