Shulou(Shulou.com)05/31 Report--

This article shows you what is the common application scenario of Paxos algorithm in large systems. The content is concise and easy to understand. It can definitely make you shine. I hope you can gain something through the detailed introduction of this article.

In the field of distributed algorithms, there is a very important algorithm called Paxos. How important is it? Google Chubby [1] mentioned

all working protocols for asynchronous consensus we have so far encountered have Paxos at their core.

The detailed description of Paxos algorithm is more introduced in Wikipedia. The Chinese version introduces the rules of choose value [2], the English version introduces the process of Paxos 3 phase commit [3], and the Chinese version is not translated from the English version but written independently, so it is very complementary. The Paxos algorithm was proposed by Leslie Lamport, who wrote in Paxos Made Simple[4].

The Paxos algorithm, when presented in plain English, is very simple.

When you've been studying Paxos for a long time and you're still a little confused, it might be a little frustrating to see this sentence. However, it is recognized that its algorithm is still relatively cumbersome, especially when you have to use the rigorous thinking of programmers to sort out all the details, your mind will be full of question marks. Leslie Lamport also spent nine years perfecting the theory of this algorithm.

In fact, for the average developer, we don't need to know all the details of Paxos and how to implement it, just know that Paxos is a distributed election algorithm. This article mainly introduces Paxos commonly used applications, perhaps one day when your system has grown to a certain scale, you know that there is such a technology that can help you solve some technical problems correctly and elegantly.

1. Database replication, log replication, etc., such as bdb data replication is to use paxos-compatible algorithms. The biggest use of Paxos is to keep data consistent across multiple nodes.

2. naming service, for example, there are usually multiple interfaces within a large system, and services call each other.

1)The usual implementation is to write the ip/hostname of the service in the configuration, and when the service fails, manually change the configuration file or modify the DNS pointing method to solve it. Disadvantages are poor maintainability, the more internal units, the greater the failure rate.

2)LVS dual redundancy mode, the disadvantage is that all units need double resources input.

By using Paxos algorithm to manage all naming services, it can ensure that high available services are allocated to clients. ZooKeeper also provides watch functionality, i.e. watch objects will automatically send notification when they change, so that all clients can use consistent, highly available interfaces.

3.config Configuration Management

1)Usually, the method of manually modifying the configuration file is error-prone and requires manual intervention to take effect, so the state of the node cannot be consistent at the same time.

2)Large-scale applications will implement their own configuration services, such as http web services to achieve configuration centralization. Its disadvantage is that all clients cannot know immediately after the update, and the order of loading of each node cannot be guaranteed, resulting in different configurations in the system.

4. membership User role/access control list, for example, in permission settings, once a user sets a certain permission, such as changing from administrator to ordinary identity, it should take effect immediately on all remote CDN servers, otherwise it will lead to unacceptable consequences.

5. Number assignment. Often the simple solution is to increment the ID with the database, which makes it difficult to partition the database, or to generate a GUID programmatically, which usually results in an ID that is too long. A more elegant approach is to use paxos algorithm to select one of multiple replicas as master, and assign numbers through master. When a master fails, paxos selects another master.

Here are some common Paxos applications, for which other solutions, on the one hand, do not provide automatic high availability solutions, and on the other hand, they are far less simple and elegant than Paxos.

Yahoo! ZooKeeper [5] is an open-source implementation of Paxos-like. Its programming interface looks a lot like a distributed small file system that provides strong consistency guarantees. It applies to all of the above situations. Unfortunately, ZooKeeper does not follow the Paxos protocol, but is based on a 2-phase commit protocol designed and optimized by ZooKeeper itself, so its theory [6] has not been fully proved. But ZooKeeper is still on Yahoo! Internals have been successfully applied in HBase, Yahoo! Message Broker, Fetch Service of Yahoo! Crawler and other systems, so it can be used with confidence.

Also select a paragraph from Paxos made live [7] for the end of the implementation experience.

* There are significant gaps between the description of the Paxos algorithm and the needs of a real-world system. In order to build a real-world system, an expert needs to use numerous ideas scattered in the literature and make several relatively small protocol extensions. The cumulative effort will be substantial and the final system will be based on an unproven protocol.

* Because chubby fills in some details not mentioned in Paxos 'paper, the final implementation system is not a theoretically fully verified system.

* The fault-tolerance computing community has not developed the tools to make it easy to implement their algorithms.

* Distributed fault-tolerant algorithms lack supporting tools to help implement algorithms. For example, although the compilation field is complex, tools such as yacc and ANTLR have reduced the difficulty of this field to a minimum.

* The fault-tolerance computing community has not paid enough attention to testing, a key ingredient for building fault-tolerant systems.

* Lack of testing tools in distributed fault-tolerant algorithms

As a background, proving distributed fault-tolerant algorithms is often more difficult than implementing them. Google can't prove Chubby is reliable. Yahoo! There is no guarantee that Zookeeper theory is correct. Most systems have been running in practice for a long time before they can be cautiously expressed, and the current system has basically found no major problems.

What is the common application scenario of Paxos algorithm in large systems? Have you learned knowledge or skills? If you want to learn more skills or enrich your knowledge reserves, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.