Shulou(Shulou.com)06/01 Report--

How to carry out distributed system architecture and CAP principle analysis, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

What is a distributed system?

With regard to the definition of distributed systems, the book "principles and paradigms of distributed systems" defines distributed systems as follows: "A distributed system is a collection of independent computers that are like a single system to users."

(the picture is selected from "Common Technologies and case Analysis of distributed Systems")

There are many aspects to consider in designing a distributed system. For example, how does the system disassemble the molecular system? How to plan the communication in the subsystem? Is the communication secure? Is the system scalable? How does the subsystem achieve reliability? How to ensure the consistency of the data? Wait, it's not easy to solve.

But for our ordinary users, the open source world already has perfect solutions and tools. For example, when we design communications, we can use message-oriented middleware, such as Apache ActiveMQ, RabbitMQ, Apache RocketMQ, Apache Kafka, etc., as well as similar RPC frameworks such as Google Protocol Buffer and Thrift. When designing distributed computing, we can use MapReduce, Apache Hadoop, Apache Spark, Apache Flink and so on. In big data distributed storage, we can choose HDFS, Apache HBase, Apache Cassandra, Memcached, Redis, MongoDB and so on. In the aspect of distributed monitoring, the common technologies include Nagios, Zabbix, Consul, ZooKeeper and so on.

Second, a little understanding of CAP

In July 2000, Professor Eric Brewer of the University of California, Berkeley, put forward the CAP conjecture at the ACM PODC conference. Two years later, Seth Gilbert and Nancy Lynch of the Massachusetts Institute of Technology proved CAP theoretically. After that, CAP theory has officially become a recognized theorem in the field of distributed computing.

First, let's take a look at what CAP stands for.

Consistency: after the update operation is successful and the return client is completed, all distributed nodes have the same data at the same time. (note: here we are talking about strong consistency. )

Availability: both read and write operations can be successful. At any time, the user's request can be returned within the valid time.

Partition fault tolerance: can the system continue to serve when the distributed nodes cannot communicate with each other due to the network failure?

It should be noted that in a distributed system, no design can meet the three characteristics of consistency, availability and partition fault tolerance at the same time. The three CAP are not equal, in which P is the foundation, and CA needs to be weighed before.

If there's anything special about Spanner, it's Google's wide area network. Google ensures P through the establishment of private networks and strong network engineering capabilities. On the basis of years of operational improvement, it can minimize the occurrence of partitions in the production environment, thus achieving high availability.

The father of CAP wrote in "Spanner, True time, CAP Theory"

In the global wide area geographical distribution environment (global scale distributed system), network zoning is a natural fact, and even some people think it is inevitable. Used to explain why P is the foundation. )

The consistency mentioned above can be divided into several categories: strong consistency, monotonous consistency, weak consistency, session consistency, and final consistency.

Strong consistency

Any user can read the last successfully updated data at any time.

Monotone consistency

At any time, once a user reads the value of a data after an update, it will no longer read a value older than that value. In other words, the order of available data must be monotonously increasing.

Weak consistency

The user cannot read the latest updated value within a certain time. Session consistency

Once a user reads the value of a certain data after an update in a certain session, he or she will no longer read an older value than this value in this session. Session consistency is a further relaxation of constraints on the basis of monotonous consistency, which only guarantees the monotonicity of a single user within a single session, but not between different users or different sessions of the same user. Final consistency

Users can only read the value after an update, but the system ensures that the data will eventually reach a completely consistent state, but the time required cannot be guaranteed.

III. Summary

The article also mentioned at the beginning of the article, now the open source world has a complete distributed system solution, we do not need to repeat the wheel, the difficulty can be imagined. We can reverse which two features of CAP are satisfied by this system in the process of using it. For example, Zookpper is the foundation of CP,P, and all distributed systems need to give priority to this. Zookeeper ensures data consistency from the aspects of sequential consistency, atomicity, single mirror, persistence and real-time performance. But it will re-elect the Leader in some cases (restart or network node failure), when the entire cluster is unavailable. The problem is that the election leader takes too long, 30-120s, and the entire zk cluster is not available during the election period, which leads to the paralysis of the registration service during the election period. Although the service can be finally restored, the long-term unavailability of registration caused by the long election time cannot be tolerated. Therefore, ZooKeeper cannot guarantee the availability of services. So Zookepper guarantees CP, not AP.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.