Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "Redis persistence case Analysis". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Interviewer: tell me what you think of Redis, young man.

Me: Ah, opinion, sitting or lying down. Redis is very small? Soon? But it lasts?

Interviewer: seriously, I suspect you are driving, not only driving but also making color.

Me:.

Interviewer: go. My time is limited. Don't talk nonsense. Back to the point, how much do you know about Redis?

Me: lightweight and small, memory-based is very fast, RDB with AOF persistence makes it just as strong and durable.

Interviewer: tell me something specific.

Me: please look at the text.

Text

Brief introduction

Redis is an open source, high-performance, key-value pair-based cache and storage system, which provides a variety of key data types to meet the cache and storage needs in different scenarios. At the same time, many high-level functions of Redis make it suitable for different roles such as message queue, task queue, and so on. In addition, Redis supports external module extensions that can be used as a primary database in certain scenarios.

Because the reading and writing speed of memory is much faster than that of hard disk, even the current thinking of solid state disk is probably developing towards the mode of thinking of memory. Maybe I am a layman, but I still use mechanical disk for long-term storage. So all the data in the Redis database is stored in memory, which is pretty fast. There is also a certain risk, which can lead to data loss, but with RDB and AOF persistence can reduce the risk.

First acquaintance with Redis1, installation under linux (Redhat7 series) 1.1, installation

What is prepared here is the source code package. The version is not up-to-date, but stable.

The rest of the version is available on the official website or on its hosted platform github. The following is the download address of the official website of Redis.

Https://redis.io/download

Redis-6.0.8.tar.gz# installs tar-zxvf redis-6.0.8.tar.gz# to compile make & & make install1.2, troubleshoot make [1]: * * [server.o] error 1

1.3, solution

1.3.1. Installation depends on the environment

Yum-y install centos-release-sclyum-y install devtoolset-9-gcc devtoolset-9-gcc-c++ devtoolset-9-binutils

1.3.2. Add environment variables and take effect

Scl enable devtoolset-9 bashecho "/ opt/rh/devtoolset-9/enable" > > / etc/profile

Reread the environment variable configuration file

Source / etc/profile

Recompile to resolve the problem

# change to the installation directory of Redis. The general source code package installation will be placed under / usr/local/. See how individuals use cd / opt/redis-6.0.8/# to compile make & & make install.

For common basic command exercises, you can refer to the rookie tutorial

Https://www.runoob.com/redis/redis-commands.html

1.4. Startup and login

Start the redis-server server

# start the redis service nohup / opt/redis-6.0.8/src/redis-server &

Log in to the redis-cli client

# Log in to redis-cli/opt/redis-6.0.8/src/redis-cli

The test verifies that the redis under linux has been launched successfully. The basic usage will be introduced below.

Pingpong

1.5. Set the password

There is no open password setting by default, and you need to manually turn on the commented-out parameter configuration.

# Edit the original configuration file vim / opt/redis-6.0.8/redis.conf# is commented out, copy a line and change it to the password you set # requirepass foobaredrequirepass 1234562, install 2.1under Windows, install Redis-x64-3.2.100.zip

2.1.1. Decompress under Windows or install msi directly.

2.1.2. Set the service command (register as a service form, self-starting)

Installation service

Redis-server-service-install redis.windows-service.conf-loglevel verbose

Uninstall the service

Redis-server-- service-uninstall2.2, starting and shutting down redis-server redis.windows.conf

2.2.1. Enable the service

Redis-server-service-start

2.2.2, stop the service

Redis-server-- service-stop2.3, start the redis service # also run cmdredis-server as administrator in the directory unzipped or installed by redis-- service-start2.4, run test login under cmd # run cmdredis-cli.exe-h 127.0.0.1-p 637 login as administrator in the directory unzipped or installed by redis or execute redis-cli# directly execute redis-cli# login test ping

5. Rdm, a management tool under Windows, is a visual interface.

Https://redisdesktop.com/download

Basic knowledge 1. Interviews often ask questions

Interviewer: what are the data types in redis? can you talk about them?

Me: string (string type), hash (hash type), list (list type), set (collection type), zset (ordered collection type), stream (flow type)

Stream is a new feature support for redis5.0.

Interviewer: Ooh, the young man has something. He knows a lot about it, even the type of stream stream.

Me: I look confused.

Third, advanced 1. Persistence.

Interviewer: do you know some advanced features of Redis?

Me: a little bit.

Interviewer: can you talk about it in detail?

Me: quickly summed up by reading books before brain searchers. Caching and persistence are coming.

Redis is used as the cache server, but the performance will be greatly affected after the cache is penetrated, and all caches fail the cache avalanche at the same time, making the service unresponsive.

We hope that Redis will be able to synchronize the data from memory to disk in some form so that it can be restarted and recovered according to the records on disk. This process is persistence.

Interviewer: do you know what common persistence methods are available for Redis?

Me: RDB persistence enabled by Redis by default, while AOF persistence needs to be enabled manually.

Redis supports two kinds of persistence. One is RDB, the other is AOF. The former stores the data in memory on the hard disk according to the specified rule "timing", while the latter records the command book after each execution of the command. For these two persistence methods, you can use either alone, but in most cases you combine the two closely.

At this time, the interviewer looked forward to me with a sparkling look. Please continue.

2. RDB mode

To continue, RDB adopts snapshot mode. The default setting is custom snapshot [automatic synchronization]. The default configuration is as follows.

You can also synchronize manually.

# it is not recommended to use SAVE# asynchronous form BGSAVE# based on custom snapshot FLASHALL3 and AOF in production environment

When using Redis to store non-temporary data, it is generally necessary to turn on AOF persistence to reduce data loss caused by process termination. AOF can append every command executed by Redis to the hard disk file, which obviously degrades the performance of Redis in the process, but in most cases this is acceptable. It is emphasized here that using a hard disk that reads and writes faster can improve the performance of AOF.

It is not enabled by default, so you need to open AOF manually. When you view the redis.conf file, you will also find that appendonly is configured with no.

Appendonly yes

When AOF persistence is turned on, each execution of a command will change the directory of the data in Redis, and Redis will write the command to the AOF file on disk. The AOF file is saved in the same location as the RDB file, both of which are set by the dir parameter. The default file name is appendonly.aof, which can be modified by the appendfilename parameter.

Appendfilename "appendonly.aof

In fact, this is exactly what Redis does. Whenever a certain condition is met, Redis automatically rewrites the AOF file. This condition can be set in the redis.conf configuration file:

Auto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb

At startup, Redis executes the commands in the AOF file line by line to load the data from the hard disk into memory, which is slower than RDB.

Although each time you perform an operation that changes the contents of the database, AOF records the command in the AOF file. But in fact, due to the caching mechanism of the operating system, the data is not actually written to the hard disk, but into the hard disk cache of the operating system. By default, the operating system performs a synchronization every 30 seconds to write the contents of the hard disk cache to the hard disk.

In Redis, you can set the timing of synchronization through appendfsync:

# appendfsync always# is set to everysecappendfsync everysec# appendfsync no by default

Redis allows both AOF and RDB to be enabled at the same time. This not only ensures the security of the data, but also friendly to the backup and other operations. When Redis is restarted at this point, the AOF file is used to recover the data. Because of the persistence of AOF mode, the probability of data loss is minimized.

4. Redis replication

Through persistence, Redis ensures that data is not lost (a small part of it) even if the server is restarted. However, the database is stored on a single server, it is inevitable that a variety of emergencies will not occur, such as hard disk failure, sudden server downtime and so on, which will also lead to data loss.

To avoid failures as much as possible, it is common practice to replicate multiple copies of the database for deployment on different servers. In this way, even if one fails, other servers can still provide services. For this reason, Redis provides the replication function. That is, after the data in one database is updated, the updated data is automatically synchronized to other databases.

At this time, students who are familiar with MySQL feel that it is very similar to the master-slave replication of MySQL, so as to turn on binary log binlog to achieve synchronous replication.

It is easier to use replication in Redis than in MySQL. You only need to add slaveof to the slave database address when booting from the library.

# configure the slaveof master_database_ip_addr# test in the slave library, add nohup and & to put it in the background, and output the log to nohup.outnohup / opt/redis-6.0.8/src/redis-server in / root/ directory-- 6380-- slaveof 192.168.245.147 6379 &

4.1. Principle

Copy initialization. The main principle here is to boot from the library and send a SYNC command to the master library. At the same time, after receiving the SYNC command, the main library will begin to save the snapshot in the background, that is, the process of RDB persistence, and cache the commands received during the snapshot. When the snapshot is complete, Redis sends the snapshot file and all cached commands to the slave database. When received from the database, the snapshot file is loaded and the cache command received is executed.

The replication synchronization phase runs through the entire master-slave synchronization process until the master-slave relationship is terminated. Snapshots play a vital role in the replication process, and snapshots are taken as long as the replication is performed, even if RDB persistence is turned off by deleting all save parameters.

4.2. Optimistic replication

Redis adopts an optimistic replication strategy (optimistic replication). Tolerate that the content of the master-slave database is different in a certain period of time, but the data of the two will eventually be synchronized. Specifically, the process of Redis copying data between master and slave databases is asynchronous, which means that after the master database executes the command requested by the client, the master database will immediately feedback the execution result of the command in the master database to the client, and asynchronously synchronize the data to the slave database, and will not wait for the command to be received from the database before returning to the client.

It is writable when the data is synchronized to at least the specified number of slave libraries, and is specified by parameters:

# set minimum limit 3min-slaves-to-write limits setting allows min-slaves-max-lag 104.3 to lose connection time from data, incremental replication

Based on the following three points

The slave library stores the running ID (run id) of the master library. Each Redis running instance has a unique running ID, and each time the instance is restarted, a new running ID is automatically generated. Similar to MySQL, it is identified from the unique ID configured by the node.

In the replication synchronization phase, when a command from the master library is transferred to the slave library, the command is also stored in a backlog queue (backlog) to record the offset range of commands stored in the current backlog queue.

When a command from the main library is received from the library, the offset of the command is recorded.

4.4. Attention

When the primary database crashes, the situation is slightly complicated. When restoring master database data manually from a database database, you need to strictly follow the following principles:

Use the SLAVEOF NO ONE command in the slave database to promote the slave library to the master database to continue the service.

Start the master library that crashed before, and then use the SLAVEOF command to set it as a slave to the new master library.

Note: when replication is enabled and database persistence is turned off, do not use supervisor and similar process management tools to cause the main library to crash and restart. Similarly, when the server where the main library is located is shut down due to a failure, avoid a direct restart. Because when the main library is restarted, persistence is not enabled, and all data in the database is emptied. At this time, the slave library will still receive data from the master database, which causes all slave libraries to be emptied, resulting in loneliness in the persistence of the database.

Manual maintenance is really troublesome, but Redis provides an automated solution: Sentinels to implement this process to avoid error-prone problems with manual maintenance.

5. Sentinel (sentinel)

From the replication calendar of Redis, we know that in a typical one-master and multi-slave Redis system, the slave library plays the role of redundant backup and read-write separation in the whole system. When the master library encounters an abnormal interruption of service, the developer will manually upgrade the master to make the system continue to serve. The process is relatively complex and difficult to automate. The Sentinel tool can be used at this time.

The role of sentinels

Monitor the operation of Redis system

Monitor whether the master library and slave library are running properly

Master library gg Smecta, automatically upgrade slave library to master library, Meizi

Of course, there are multiple Sentinel monitoring master-slave database modes, and Sentinels also monitor each other, as shown below:

First of all, you need to build an one-master and multi-slave model, and then turn on the configuration sentinel.

# the main library sentinel monitor master 127.0.0.1 6379 sets up configuration files, such as sentinel.confredis-sentinel / opt/path/to/sentinel.conf

That's all I have to say about the Sentinel, and now I have an impression in my mind. At least know that there is such a thing, you can talk to a beautiful interviewer more.

6. Cluster (cluster)

The cluster feature has been added since Redis3.0.

Even with the use of sentinels, every database in the Redis cluster still contains all the data in the cluster, resulting in the total data storage of the cluster being limited to the database node with the least available memory, followed by the bucket effect. Because all data in Redis is based on memory storage, the problem is already prominent, especially when Redis is used as a persistent storage service.

There is such a scene. As far as capacity expansion is concerned, after the client is sharded, if more nodes are added, the database needs to be migrated manually. In the process of migration, in order to ensure the consistency of the data, we need to temporarily offline the entry into the group, which is relatively complex.

At this time, considering that the Redis is very small, ah, no slip of the tongue, it is lightweight. Pre-fragmentation (presharding) can be used to avoid problems to some extent. In other words, in the early stages of deployment, consider the future storage size in advance and create enough instances.

From the above theoretical knowledge, Sentinel and cluster are similar, but Sentinel and cluster are two independent functions. If you want to expand the capacity horizontally, the cluster is a good choice.

Configure the cluster and open the cluster-enabled in the configuration file redis.conf

Cluster-enabled yes

Configure the cluster each node configures a different working directory, or modify the persistence file

Cluster-config-file nodes-6379.conf

Cluster testing can be configured, or refer to other books, it is not difficult to implement. As long as you know how it works.

IV. Redis for Java

Example

Package com.jedis;import redis.clients.jedis.Jedis;import redis.clients.jedis.JedisPool;import redis.clients.jedis.JedisPoolConfig;public class Test {@ org.junit.Test public void demo () {Jedis jedis = new Jedis ("127.0.0.1", 6379); jedis.set ("name", "sky"); String params = jedis.get ("jedis") System.out.println (params); jedis.close ();} @ org.junit.Test public void config () {/ / get the configuration object of the connection pool JedisPoolConfig config = new JedisPoolConfig (); / / set the maximum number of connections config.setMaxTotal (30) / / set the maximum number of idle connections config.setMaxIdle (10); / / get the connection pool JedisPool pool = new JedisPool (config, "127.0.0.1", 6379); / / get the core object Jedis jedis = null Try {/ / get connection through connection pool jedis = pool.getResource (); / / set object jedis.set ("poolname", "pool") / / get object String pools = jedis.get ("poolname"); System.out.println ("values:" + pools);} catch (Exception e) {e.printStackTrace () } finally {/ / release resource if (jedis! = null) {jedis.close ();} if (pool! = null) {pool.close () This is the end of the content of "Redis persistence instance Analysis". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.