Shulou(Shulou.com)06/02 Report--

What is the principle of Memcached? For this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more small partners who want to solve this problem find a simpler and easier way.

I. Memcached Profile

Memcached is a high-performance distributed memory cache server. It caches database query results and reduces database access times to improve the speed and scalability of dynamic Web applications. It is similar to Redis. Memcached is based on a hashmap that stores key/value (K/V) pairs. Clients can be written in any language and communicate with daemons via the memcached protocol.

Memcached's API uses a 32-bit cyclic redundancy check (CRC-32) to compute key values and distribute the data across different machines. When the table is full, the next new data is replaced by the LRU mechanism.

Second, Memcached distributed algorithm

1, remainder hash (remainder calculation dispersion):

Principle: Disperse according to the remainder of the number of servers: first obtain the integer hash value (int hashCode) of the key, convert the key into an integer through the Hash function, and then divide the hash value by the number of servers, and select the server according to the remainder.

For example: For a total of 10 servers (numbered 0~9), first obtain the integer hash value of the key of the data: hashCode=hash(key)=13, and then divide by the total number of servers, mod 10 (13)=3, that is, select the server numbered 3 for storage.

Disadvantages: When adding or reducing servers, re-hashing is required, which will cause the original server number to change, and the remainder obtained may also change, resulting in a decrease in Memcached hit rate, resulting in more access to the database server to adjust data, increasing the load on the database server.

2. Consistent hash:

Principle:

Consistent hashing is the organization of the entire hash value space into a virtual ring, such as assuming that the value space of a hash function H is 0~(2^32 -1)(i.e., the hash value is a 32-bit unsigned integer). This hash space is a ring. Then let each machine occupy a sector of space. (Consistent hashing, such that the space of hash values stored by the server is a range, rather than a specific series of residues.) This reduces the impact of server additions.)

The hash space is organized clockwise. To determine the spatial location of each server, each server is hashed according to its hostname or IP address. Then you need to use the hash algorithm to determine which server the data should be stored in: First, calculate the hash value h according to the key value using the same function H, determine the position of the data on the ring according to h, and look clockwise down from this position. The first server encountered is the server where it should be stored.

Fault tolerance: For the following figure, when Server3 fails, the storage indication D of the data changes and is stored on Server2. The storage data of the system changes the least (because the data is clockwise to find the server storage, so the D position data is found on the Server2 server)

Scalability: Assuming that Server4 is added and placed in the middle of BC according to the rules, only B is stored on Server4, and the overall impact only occurs in the interval part of the new node.

Memcached's data removal algorithm

Memcached starts the LRU (Least Recently Used) algorithm to weed out old data items.

Each slab maintains a queue, where newly inserted data is at the head of the queue, and frequently obtained data is also moved to the head of the queue, so that older or less accessed data is relatively left at the end of the queue. The algorithm starts elimination at the end of the queue. When slab does not allocate enough memory, it first checks the queue tail for stale data. If there is one, it will be overwritten directly as a new object, if not, it will start to eliminate the object at the end of the queue.

IV. Workflow

Check whether the client's request data is in memcached, if yes, return the request data directly, if no, access the database server for operation;

If the requested data is not in memcached, check the database, return the data obtained from the database to the client, and cache a copy of the data in memcached (memcached client is not responsible, it needs to be explicitly implemented by the program);

Every time the database is updated, the data in memcached is updated to ensure consistency; when the memory space allocated to memcached is exhausted, the LRU (Least Recently Used) policy is used plus the expiration policy, and the expired data is replaced first, and then the recently unused data is replaced.

What is the difference between Memcached and Redis?

1. Data storage location: Redis supports the persistence of data, and can keep the data in memory in disk. It can be loaded again for use when restarting. Memecache stores all data in memory;

2. Speed: Redis is much faster than memcached;

3. Multi-threading: Memcached supports multi-threading, while Redis uses a single-threaded IO multiplexing model;

Supported data types: Memcached only supports simple data types. To store complex data types, you must convert complex data types into simple data types. Redis not only supports simple k/v data, but also provides storage for string(string), list(list), set(set), zset(sortedset --ordered set) and hash (hash type) data structures.

About Memcached principle is what the answer to the question is shared here, I hope the above content can be of some help to everyone, if you still have a lot of doubts not solved, you can pay attention to the industry information channel to learn more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.