Shulou(Shulou.com)06/01 Report--

Mongodb and memcached are not in the same category. Mongodb is a document-based non-relational database, which has the advantage of powerful query function and can store large amounts of data. There is no question of who replaces whom between mongodb and memcached. MongoDB is actually a simplified version of MySQL,Mongodb 's interface, which is actually similar to encapsulating SQL. Mongodb can only support simple single query statements, no join, no subqueries. The actual performance of MySQL is similar to that of Mongodb. MySQL does not perform very well in the case of high concurrency, and MongoDB will be much better. Mongodb mode is free, easy to build distributed clusters, suitable for storage.

Closer to memcached is redis. They are all memory databases, and the data is stored in memory and accessed directly through tcp. The advantage is high speed and high concurrency, but the disadvantage is that the data type is limited and the query function is not strong, so it is generally used as cache. In our team's project, we started with memcached and then replaced it with redis.

Compared to memcached:

1. Redis has a persistence mechanism, which can persist the data in memory to the hard disk on a regular basis.

2. Redis has binlog function, which can write all operations to the log. When the redis fails, the data can be recovered according to binlog.

3. Redis supports virtual memory, which can limit the amount of memory used. When the data exceeds the threshold, the least commonly used data in memory will be saved to the page file of the hard disk through an algorithm similar to LRU.

4. Redis natively supports more data types and uses more imagination space.

5. The consistent hash mentioned by a friend earlier is used in the sharding of redis. It is generally used when the load is very high and horizontal scaling is needed. We haven't used this function yet. for general projects, stand-alone is enough.

The support is concurrent. Redis 3. 0 will introduce cluster, which is more powerful.

6. For more advantages of redis, please go to the official website for more information.

7. Performance

According to the authors of Redis, the average performance on a single core, Redis is better when a single piece of data is small. Why? the reason is that Redis runs in a single thread.

Because it is single-threaded, the overall performance is bound to be low compared to Memcached's multithreading.

Because it runs in a single thread, IO is serialized, network IO and memory IO, so when a single piece of data is too large, you need to wait for all IO of a command to complete before performing subsequent commands, so performance will be affected.

If you simply compare the difference between Redis and Memcached, most will get the following point of view:

1 Redis not only supports simple KBE data, but also provides storage of data structures such as list,set,hash.

2 Redis supports data backup, that is, data backup in master-slave mode.

3 Redis supports data persistence, which can keep the data in memory on disk and can be loaded and used again when rebooting.

In Redis, not all data is stored in memory all the time. This is the biggest difference compared with Memcached. Redis only caches all key information. If Redis finds that the memory usage exceeds a certain threshold, it will trigger the operation of swap, and Redis calculates which key corresponding value needs to be swap to disk based on "swappability = age*log (size_in_memory)". The value corresponding to these key is then persisted to disk and cleared in memory. This feature allows Redis to maintain more data than the memory of its machine itself. Of course, the machine's own memory must be able to maintain all the key, after all, this data will not be swap operation. At the same time, when Redis swap the data in memory to disk, the main thread providing the service and the sub-thread performing the swap operation will share this part of memory, so if the data of swap is needed to update, Redis will block the operation until the child thread completes the swap operation.

You can refer to the comparison before and after using the Redis-specific memory model:

VM off: 300k keys, 4096 bytes values: 1.3G usedVM on: 300k keys, 4096 bytes values: 73M usedVM off: 1 million keys, 256 bytes values: 430.12M usedVM on: 1 million keys, 256 bytes values: 160.09M usedVM on: 1 million keys, values as large as you want, still: 160.09M used

When reading data from the Redis, if the value corresponding to the read key is not in memory, then the Redis needs to load the data from the swap file and then return it to the requester. There is a problem with the I / O thread pool. By default, Redis will block, that is, all swap files will not be loaded until they are loaded. This strategy is suitable for batch operations because of a small number of clients. However, if you apply Redis to a large website application, it is obviously not enough to meet the situation of large concurrency. So we set the size of the Redis O thread pool to concurrently operate on read requests that need to load the corresponding data from the swap file to reduce blocking time.

Comparison of redis, memcache and mongoDB

Redis, memcache and mongoDB are compared from the following dimensions. Welcome to clap bricks.

1. Performance

All of them are relatively high, and performance should not be a bottleneck for us.

Generally speaking, in terms of TPS, redis is similar to memcache, but larger than mongodb.

2. Convenience of operation

Memcache data structure is single

Redis is richer. In terms of data operation, redis is better and has fewer network IO times.

Mongodb supports rich data expression, index, most similar to relational database, and supports a wide range of query languages.

3. The size of memory space and the amount of data

Redis adds its own VM feature after version 2.0, which breaks through the limitation of physical memory; you can set the expiration time of key value (similar to memcache)

Memcache can modify the maximum available memory, using LRU algorithm

MongoDB is suitable for large amount of data storage, relying on the operating system VM for memory management, eating memory is also relatively strong, services should not be together with other services.

4. Availability (single point of problem)

For a single point problem

Redis relies on the client to achieve distributed read and write; in master-slave replication, every time the slave node reconnects to the master node, it depends on the entire snapshot, no incremental replication, due to performance and efficiency problems.

Therefore, the single point problem is more complex; does not support automatic sharding, we need to rely on the program to set a consistent hash mechanism.

One alternative is to use your own active replication (multiple copies of storage) without redis's own replication mechanism, or change to incremental replication (which needs to be implemented by yourself), a tradeoff between consistency and performance.

Memcache itself has no data redundancy mechanism and is not necessary. For fault prevention, we use mature hash or ring algorithm to solve the jitter problem caused by single point of failure.

MongoDB supports master-slave,replicaset (internal paxos election algorithm, automatic fault recovery), auto sharding mechanism, and shields the client from the failover and segmentation mechanism.

5. Reliability (persistence)

For data persistence and data recovery

Redis support (snapshot, AOF): relying on snapshots for persistence, aof enhances reliability while affecting performance

Memcache does not support it and is usually used for caching to improve performance.

MongoDB has adopted binlog since version 1.8 to support the reliability of persistence.

6. Data consistency (transaction support)

Memcache uses cas to ensure consistency in concurrent scenarios

Redis transaction support is relatively weak, so it can only guarantee the continuous execution of each operation in the transaction.

MongoDB does not support transactions

7. Data analysis

MongoDB has built-in data analysis function (mapreduce), which is not supported by others.

8. Application scenarios

Redis: more performance operations and operations with small amounts of data

Memcache: used to reduce database load and improve performance in dynamic systems; cache to improve performance (suitable for reading more and writing less; for large amounts of data, sharding can be used)

MongoDB: mainly solve the problem of access efficiency of massive data

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.