Shulou(Shulou.com)06/01 Report--

This article mainly introduces what redis persistent storage is, which is very detailed and has a certain reference value. Friends who are interested must read it!

Overview of redis

REmote DIctionary Server (Redis) is a persistent database storage system based on key-value key-value pairs. Redis is similar to the famous Memcached caching service software, but redis supports a variety of data storage types than memcached, including strings (string), lists (list), sets (collection) and sorted sets (ordered collection).

These data types support push/pop,add/remove and rendezvous, union and difference sets, and richer operations, all of which are atomic. On this basis, redis supports a variety of different sorting methods. Like the memcached caching service, data is cached in memory to ensure efficiency. Unlike memcached, redis persistent caching service periodically writes updated data to disk and appends modified operation records to files and records. Compared with memcached, redis also supports master-slave (master-slave) synchronization, which is very similar to MySQL master-slave replication function.

Redis is an open source log database written in C (more than 30, 000 lines of code) that supports the network, memory-based and persistent, Key-Value database, and provides API in multiple languages. Since March 15, 2010, the development of Redis has been presided over by VMware.

The emergence of Redis software, to a certain extent, makes up for the lack of key-value memory cache services such as memcached, and can play a good complementary role to relational databases on some occasions. Redis provides a Python,Ruby,Erlang,PHP client

1.2 redis featur

Key-value key type storage supports reliable data storage and landing single-process single-thread high-performance server crash safe & recovery slow stand-alone qps can reach 10W suitable for high-speed read and write access with small amount of data

1.3 benefits of Redis

Unlike memcached, Redis can persistently store data with high performance: Redis can support more than 10W per second read and write frequency.

Rich data types: Redis supports binary Strings,Lists,Hashes,Sets and sorted Sets data type operations

Atom: all operations of Redis are atomic, and Redis also supports atomicity after several operations have been merged.

Rich features: Redis also supports publish/subscribe (publish / subscribe), notification, key expiration, and so on. Redis supports off-machine master-slave replication.

1.4 redis defects and traps

The system is running with burrs.

The delay of different commands varies greatly.

High memory management overhead (setting is lower than physical memory 3ax 5)

Buffer io causes system OOM (memory overflow)

Data type of 1.5 redis

As a Key- value storage system database, Redis provides key (Key) and value (value) mapping. However, in addition to regular numeric values or strings, the key value of Redis can also be one of the following forms, and the following are the most commonly used data types:

String string Hash hash table List list Set collection Sorted set ordered collection

1.6 redis persistence

Typically, Redis stores data in memory or is configured to use virtual memory. Data persistence can be achieved in two ways: using snapshots (snapshot), constantly writing data in memory to disk, or using MySQL-like binlog logs (aof but not for master-slave synchronization) to log each update. The former has high performance, but may cause a certain degree of data loss; the latter is the opposite.

1.7 redis application scenarios

The best application scenario of redis

The best trial scenario for Redis is full data in-memoryRedis. More scenarios are used as a substitute for Memcached. Data is more important, a business that has certain requirements for data consistency. Redis is more appropriate when you need more data type support than key/value. Need to provide master-slave synchronization and load balancing distributed application scenarios (redis master-slave synchronization)

Lessons from 1.8redis production

Be sure to configure Master-slave master-slave synchronization, which can be switched in the event of a service failure.

Disable data persistence on master, just configure data persistence on slave

Physical memory + virtual memory is insufficient, dump has been dead at this time, and the machine hung up after a long time. This situation is a disaster!

When the use of Redis physical memory exceeds 3 / 5 of the total memory capacity, it will be more dangerous, and you will start to do swap, which has a large memory fragment.

When the maximum memory is reached, the key with the expiration time is emptied, even if the key has not reached the expiration time.

Redis and DB synchronous write problem, first write DB, then write redis, because there is basically no problem with write memory.

Rapid deployment of a redis environment

2.1 build the Redis deployment environment

Hostname eth0 usage Master-redis01 10.0.0.135 Master RedisSlave-redis02 10.0.0.136 Slave Redis

2.2 start installing the redis service

Download the latest stable version of redis from redis's official website (http://www.redis.io).

Wget-Q http://download.redis.io/releases/redis-2.8.9.tar.gz

# perform the following operations on both redis01 and redis02

[root@redis01 ~] # tar xf redis-2.8.9.tar-C / usr/src/ [root@redis01 ~] # cd / usr/src/redis-2.8.9/ [root@redis01 redis-2.8.9] # make MALLOC=jemalloc [root@redis01 redis-2.8.9] # make PREFIX=/usr/local/redis install [root@redis01 redis-2.8.9] # LANG=en [root@redis01 redis-2.8.9] # tree / usr/local/redis/bin/ / usr/local/redis/bin/ ├── redis-benchmark ├── redis-check-aof ├── redis-check-dump ├── redis-cli └── redis-server0 directories 5 files

After the command is executed, five executables are generated in the / usr/local/redis/bin/ directory, namely:

Redis-server,redis-cli,redis-benchmark,redis-check-aof,redis-check-dump

Their functions are as follows:

Redis-server # Redis server daemon launcher redis-cli # Redis command operation tool. Of course, you can also use telnet to operate the redis-benchmark # Redis performance testing tool according to its plain text protocol to test the read and write performance of Redis on your system and your configuration. Redis-check-aof # checks whether the update log appendonly.aof is available, similar to the mysql binlog checking tool redis-check-dump # for checking local database rdb files

2.3 configure and start the redis service

(1) configure startup commands

Operation procedure:

[root@redis01 redis-2.8.9] # ln-s / usr/local/redis/bin/* / usr/local/bin/

(2) View the command help:

[root@redis01 redis-2.8.9] # redis-server-hUsage:. / redis-server [/ path/to/redis.conf] [options]. / redis-server-(read config from stdin). / redis-server-v or-- version. / redis-server-h or-- help. / redis-server-- test-memory Examples:. / redis-server (run the server with default conf) . / redis-server / etc/redis/6379.conf. / redis-server-- port 7777. / redis-server-- port 7777-- slaveof 127.0.0.1 8888. / redis-server / etc/myredis.conf-- loglevel verboseSentinel mode:. / redis-server / etc/sentinel.conf-- sentinel

(3) start the redis service

Operation procedure:

# copy redis.conf from the source program directory to the program installation directory

[root@redis01 redis-2.8.9] # cd / usr/src/redis-2.8.9/ [root@redis01 redis-2.8.9] # pwd/usr/src/redis-2.8.9 [root@redis01 redis-2.8.9] # mkdir / usr/local/redis/conf [root@redis01 redis-2.8.9] # cp redis.conf / usr/local/redis/conf/

# start the redis service

[root@redis01 redis-2.8.9] # redis-server / usr/local/redis/conf/redis.conf &

# check the startup of the redis process

[root@redis01 redis-2.8.9] # ps-ef | grep redis | grep-v greproot 3169 1288 0 10:17 pts/0 00:00:00 redis-server *: 6379

Special hint:

After redis starts successfully, the following warning message appears at the end:

[3169] 02 Oct 10:17:30.689 # Server started, Redis version 2.8.9 [3169] 02 Oct 10:17:30.690 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory=1' to / etc/sysctl.conf and then reboot or run the command' sysctl vm.overcommit_memory=1' for this to take effect. [3169] 02 Oct 10 etc/sysctl.conf and then reboot or run the command 17 etc/sysctl.conf and then reboot or run the command 30.690 * The server is now ready to accept connections on port 6379

# warning roughly means:

Overcommit_memory is set to 0. 0. Background saving may fail if there is not enough memory; to fix this, you need to set vm.overcommit_memory to 1 in the / etc/sysctl.conf configuration file, or modify it with the command "sysctl vm.overcommit_memory=1".

So, after we do some processing, we start the redis process

[root@redis01 redis-2.8.9] # pkill redis [root@redis01 redis-2.8.9] # sysctl vm.overcommit_memory=1vm.overcommit_memory = 1 [root@redis01 redis-2.8.9] # redis-server / usr/local/redis/conf/redis.conf &

After processing, start redis again without any warning.

Vm.overcommit_memory parameter description:

According to the kernel documentation, this parameter has three values, which are:

0: when user space requests more memory, the kernel tries to estimate the remaining available memory.

1: when this parameter is set to 1, the kernel allows overuse of memory until it is used up, mainly for scientific computing.

2: when this parameter is set to 2, the kernel uses an algorithm that does not overuse memory, that is, the entire memory address space of the system cannot exceed the swap+50% RAM value, and the 50% parameter is set in overcommit_ratio.

Test the command to shut down the redis service

Redis-cli shutdown shuts down the redis process

[root@redis01 redis-2.8.9] # ps-ef | grep redis | grep-v greproot 3200 1288 0 10:38 pts/0 00:00:08 redis-server *: 6379 [root@redis01 redis-2.8.9] # redis-cli shutdown [3200] 02 Oct 12 User requested shutdown... 46.621 # User requested shutdown... [3200] 02 Oct 12 User requested shutdown.... [ 3200] 02 Oct 12 Oct 43 Oct 43 Redis is now ready to exit 46.630 * DB saved on disk [3200] 02 Bye bye... [1] + Done redis-server / usr/local/redis/conf/redis.conf [root@redis01 redis-2.8.9] # ps-ef | grep redis | grep-v grep [root@redis01 redis-2.8.9] # redis-server / usr/local/redis/conf/redis.conf

Redis self-startup script

[root@ser02 redis] # vim redserver.shrunken raceme Bash stop () {/ data/redis/bin/redis-cli-a redis shutdown} start () {/ data/redis/bin/redis-server / data/redis/conf/redis.conf &} conn () {/ data/redis/bin/redis-cli-a redis} case $1 in start) start;; stop) stop;; restart) stop start;; conn) conn *) echo "Usage:$0 (start | stop | restart)" esac [root @ ser02 redis] # chmod + x redserver.sh [root@ser02 redis] # vim / etc/profileexport PATH=/data/redis/:$PATH [root@ser02 redis] # source / etc/profileredserver.sh start # # enable redis service

Redserver.sh conn # # Log in to redis

2.4 operating the redis database through the client

Let's simply manipulate the database.

Insert data: set a key-value pair

[root@redis01 redis-2.8.9] # redis-cli # write a piece of data key (id) through the client connection local redis 127.0.0.1 redis 6379 > set id 001 # Value (001) OK127.0.0.1:6379 > get id # values key (id) "001" # display the corresponding value of key 127.0.0.1id 6379 > del id # Delete key (id) (integer) 1 # 1 indicates success 127.0.0.1get id 6379 > exists id # verify the existence of key (integer) 0 # 0 indicates that there is no 127.0.0.1 key 6379 > get id # take the value of key (nil) # error message 127.0.0.1 get user001 6379 > set user001 benetOK127.0.0.1:6379 > set user002 yunjisuanOK127.0.0.1:6379 > set user003 yun123OK127.0.0.1:6379 > get user001 "benet" 127.0.0.1 benet 6379 > get user002 "yunjisuan" 127.0.0.1 benet 6379 > keys * # View all key1 in redis) "user003" 2) "user002" 3) "user001"

More operation methods and command help

(1) Table schema of redis database

127.0.0.1 keys * # View all key1) "user003" 2) "user002" 3) "user001" 127.0.0.1 user001 > select 1 # switch to Table 1 mode OK127.0.0.1:6379 [1] > keys * # query all key (empty list or set) # nothing 127.0.0.1 user003 6379 [1] > set name wangwu # write a key-value pair OK127.0.0.1:6379 [1] > keys * # View all key1) "name" # key (name) already has 127.0.0.1 key [1] > get name # View the value of key (name) "wangwu" 127.0.0.1 key 6379 [1] > select 0 # switch back to Table 0 mode (initial mode) OK127.0.0.1:6379 > keys * # View all key1) "user003" 2) "user002" 3) "user001"

(2) remote connection of redis-cli client and non-interactive operation of database

[root@redis01 redis-2.8.9] # redis-cli-h 10.0.0.135-p 637910.0.0.135 root@redis01 redis-2.8.9 6379 > quit [root@redis01 redis-2.8.9] # redis-cli-h 10.0.0.135-p 6379 set aaa 111OK [root@redis01 redis-2.8.9] # redis-cli-h 10.0.0.135-p 6379 get aaa "111"

Redis security

(1) set the external link password for the redis client

Warning:

Because redis is quite fast, under a good server, an external user can try tens of thousands of passwords in a second, which means you need to specify a very powerful password to prevent brute force cracking.

Set the password:

[root@ser02 bin] # vim / data/redis/conf/redis.conf requirepass redis

(2) renaming dangerous orders

Rename-command set "sset" # rename set to sset

Install the redis client extension for php

(1) obtain the source code package

Wget https://github.com/nicolasff/phpredis/archive/master.zip

(2) installation

[root@redis01] # ls-l phpredis-master.tar.gz-rw-r--r--. 1 root root 164509 Oct 2 19:23 phpredis-master.tar.gz [root@redis01 ~] # tar xf phpredis-master.tar.gz-C / usr/src/ [root@redis01 ~] # cd / usr/src/phpredis-master/ [root@redis01 phpredis-master] # / usr/local/php/bin/phpize [root@redis01 phpredis-master] #. / configure-with-php-config=/usr/local/php/bin/php-config [root@redis01 phpredis-master] # make & & Make installvim / etc/php.ini extensions = / usr/lib64/php/modules/redis.so [root@ser02 modules] # systemctl restart httpd

Test:

[root@ser02 redis] # cd / var/www/html/ [root@ser02 html] # vim 1.php [root@ser02 html] # systemctl restart httpd [root@ser02 html] # php 1.php anliu

Install Python redis client operation redis

Wget https://pypi.python.org/packages/source/r/redis/redis-2.10.1.tar.gztar xf redis-2.10.1.tar.gzcd redis-2.10.1python setup.py install

Develop python programs to operate redis

Please comment out the modified redis commands in the previous redis configuration file before operation, otherwise an error will be reported.

[root@redis01 redis-2.10.1] # pythonPython 2.6.6 (r266Nov 84292, Nov 22 2013, 12:16:22) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2Type "help", "copyright", "credits" or "license" for more information. > > import redis # quote redis support Library > r = redis.Redis (host='10.0.0.135',port='6379') Password='yunjisuan') # establish the connection object of redis database (object-oriented) > r.set ('name' 'benet') # Operand calls set method to write data True > > r.get (' name') # Operand calls get to read data 'benet' > r.dbsize () # Operand view the number of data entries in redis database 1L > r.keys () # View all key [' name'] > > exit () # exit

2.11 display redis by connecting Python programs through Web interface

Develop Python scripts

[root@redis01 scripts] # cat python-redis.py # / usr/bin/pythonfrom wsgiref.simple_server import make_serverimport redisdef get_redis (): r = redis.Redis (host='10.0.0.135',port='6379',password='yunjisuan',db=0) r.set ('name','yunyunyun') return r.get (' name') def hello_world_app (environ,start_response): status = '200 OK' # HTTP Status headers = [(' Content-type') 'text/plain')] # HTTP Headers start_response (status,headers) # The returned object is going to be printed return get_redis () httpd = make_server (', 8000 hellogic worldview app) print "Serving on port 8000..." # Server until process is killedhttpd.serve_forever ()

Start the python script

Notice to close iptables [root@redis01 scripts] # python python-redis.py Serving on port 8000... # listening on port 8000

These are all the contents of the article "what is redis persistent Storage?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.