Shulou(Shulou.com)06/01 Report--

Relational database and non-relational database relational database:

An institutionalized database, created on the basis of a relational model, generally oriented to records

Including oracle, mysql, sqlserver, db2

Non-relational databases:

Except for mainstream relational databases, unexpected databases are artificially non-relational.

Including redis, mongdb, hbase, couhdb

Background of non-relational databases: high concurrent read and write requirements for databases; requirements for efficient storage and access of massive data; A brief introduction to Redis for high scalability and high availability of databases

Redis runs based on memory and supports persistence

Adopt the storage form of key-value (key-value pair)

Advantages: extremely high data read and write speed, support rich data types, support data persistence atomicity, support data backup, install and configure Redis1, install necessary environment components And install yum install gcc gcc-c++ make [root @ root ~] # yum install gcc gcc-c++ make-y # # installation environment components [root@localhost ~] # mkdir / mnt/tools [root@localhost ~] # mount.cifs / / 192.168.100.100/tools / mnt/tools/ # # mount Password for root@//192.168.100.100/tools: [root@localhost ~] # cd / mnt/tools/redis/ [root@localhost redis] # lsredis-5.0.7.tar .gz [root@localhost redis] # tar xf redis-5.0.7.tar.gz-C / opt/ # # extract [root@localhost redis] # cd / opt/ [root@localhost opt] # lsredis-5.0.7 rh [root@localhost opt] # cd redis-5.0.7/ [root@localhost redis-5.0.7] # make # compile. / / omit the process [root@localhost redis-5.0.7 ] # make PREFIX=/usr/local/redis/ install # # install. / / omit process 2. Execute the configuration Redis profile script And configure [root@localhost redis-5.0.7] # cd utils/ [root@localhost utils] #. / install_server.sh # # execute script to configure Welcome to the redis service installerThis script will help you easily set up a running redis serverPlease select the redis port for this instance: [6379] # # default port Selecting default: 6379Please select the redis config file name [/ etc/redis/6379.conf] # # configuration file Selected default-/ etc/ Redis/6379.confPlease select the redis log file name [/ var/log/redis_6379.log] # # Log file Selected default-/ var/log/redis_6379.logPlease select the data directory for this instance [/ var/lib/redis/6379] # # data file Selected default-/ var/lib/redis/6379Please select the redis executable path [] / usr/local/redis/bin/redis-server## executable file path Selected config:Port : 6379Config file: / etc/redis/6379.confLog file: / var/log/redis_6379.logData dir: / var/lib/redis/6379Executable: / usr/local/redis/bin/redis-serverCli Executable: / usr/local/redis/bin/redis-cliIs this ok? Then press ENTER to go on or Ctrl-C to abort.Copied / tmp/6379.conf = > / etc/init.d/redis_6379Installing service...Successfully added to CHK configs successful added to runlevels 345 starting Redis server...Installation successful! [root@localhost utils] # ln-s / usr/local/redis/bin/* / usr/local/bin/ # # it is easy for the system to identify [root@localhost utils] # netstat-ntap | grep 6379 # # View the listening port tcp 00 127.0.0.1usr/local/redis/bin/* 6379 0.0.0.0 root@localhost utils * LISTEN 18004/redis-server [root@localhost utils] # / etc/init. D/redis_6379 stop # # close redisStopping... Redis stopped [root@localhost utils] # netstat-ntap | grep 6379 # # View the listening port tcp 0 0127.0.0.1 TIME_WAIT-[root@localhost utils] # / etc/init.d/redis_6379 start # # enable redisStarting Redis server... [root@localhost utils] # netstat-ntap | | grep 6379tcp 0 0127.0.0.1 TIME_WAIT 6379 0.0.0.0 TIME_WAIT * LISTEN 18091/redis-server tcp 0 0127.0.1 TIME_WAIT 6379 127.0.1 TIME_WAIT-[root@localhost utils] # vim / etc/redis/6379.conf | # # modify the configuration file bind 127.0.0.1 192.168.52.149 # # set the listening address [root@localhost utils] # / etc/init.d/redis_6379 restart # # restart the redis service Stopping... Redis stoppedStarting Redis server...###Redis database basic operation [root@localhost utils] # redis-cli-h 192.168.52.149-p 6379 # # Log in redis192.168.52.149:6379 > help @ List # # get the help list BLPOP key [key...] Timeout summary: Remove and get the first element in a list, or block until one is available since: 2.0.0.According to / omitting part of content RPUSHX key value summary: Append a value to a list Only if the list exists since: 2.2.0192.168.52.149SET key value 6379 > help set # # help help SET key value [expiration EX seconds | PX milliseconds] [NX | XX] summary: Set the string value of a key since: 1.0.0 group: string192.168.52.149:6379 > set name zhangsan # # set key values for OK192.168.52.149:6379 > set net wwwOK192.168.52.149:6379 > KEYS * # View all keys 1) "name" 2) "net" 192.168.52.149 KEYS 6379 > KEYS nails? # # View key is followed by two characters 1) "net" 192.168.52.149KEYS 6379 > KEYS n * # View key begins with n 1) "name" 2) "net" 192.168.52.149KEYS 6379 > GET net # View the value of the key "www" 192.168.52.149aster 6379 > EXISTS net # View Whether the key exists (integer) 1 # # 1 is present 0 is 192.168.52.149del net 6379 > EXISTS nat (integer) 0192.168.52.149del net # # delete key (integer) 1192.168.52.149del net * 1) "name" 192.168.52.149del net * 1) "name" 192.168.52.149 > type name # # View key type string192.168.52.149:6379 > rename name N1 # rename key OK192.168.52.149:6379 > KEYS * 1) "N1" 192.168.52.149) : 6379 > get N1 "zhangsan" 192.168.52.149 hset person score 6379 > hset person score 80 # # establish a key value pair by hash (integer) 1192.168.52.149hset person score 6379 > hset person name zhangsan # # establish a key value pair by hash (integer) 1192.168.52.149integer 6379 > hset person age 30 # # establish a key value pair by hash (integer) 1192.168.52.1490 hash 6379 > KEYS * 1) "person" 192.168.52.149hset person score 6379 > hget person Name # # get the key value "zhangsan" 192.168.52.149hget person age # # get the key value "30" 192.168.52.149hget person age 6379 > set name lisi # # get the key value OK192.168.52.149:6379 > KEYS * 1) "name" 2) "person" 192.168.52.1496379 > EXPIRE name 10 # set the automatic deletion time of the key 10s (integer) 1192.168.52.1496379 > KEYS * # # View all keys within # # 10s 1) "name" 2) "person" 192.168.52.149KEYS 6379 > KEYS * # 10s view all keys 1) "person" 192.168.52.149KEYS 6379 > exit # # exit 3, Perform pressure test [root@localhost utils] # redis-benchmark-h 192.168.52.149-p 6379-c 100-n 1000000 # concurrent 100 100000 requests = SET = 100000 requests completed in 1.14 seconds # # time spent 100 parallel clients 3 bytes payload keep alive: 184.66% MOVE N1 10 # # Mobile key pair to the 11th library (integer) 1192.168.52.149seconds 6379 > KEYS * 1) "mylist" 2) "counter:__rand_int__" 3) "key:__rand_int__" 4) "myset:__rand_int_" _ "192.168.52.149GET 6379 > SELECT 10 # # enter the 11th library OK192.168.52.149:6379 [10] > KEYS * # View key 1)" N1 "192.168.52.149 GET 6379 [10] > FLUSHDB # # clear the database data OK192.168.52.149:6379 [10] > KEYS * # View all keys (empty list Or set) 192.168.52.149OK192.168.52.149:6379 6379 [10] > SELECT 0 # # switch to the first library OK192.168.52.149:6379 > KEYS * # View all keys 1) "myset:__rand_int__" 2) "mylist" 3) "key:__rand_int__" 4) "counter:__rand_int__" 192.168.52.149 OK192.168.52.149:6379 6379 > exit [root@localhost utils] # Redis persistence

Redis is running in memory, and the data in memory is lost due to power outage.

In order to reuse Redis data later, or to prevent system failure, we need to write the data in Redis to disk space, that is, persistence

Persistent classification RDB mode: create a snapshot to obtain copies of all the data in the Redis at a certain time AOF mode: write the executed write command to the end of the file, log the changes in the data RDB persistence

Default persistence mode of Redis

Default file name dump.rdb

Trigger condition: perform a specified number of write operations (profile control), execute save or bgsave (asynchronous) commands, execute flushall commands, empty all data in the database and execute shutdown commands within a specified time interval to ensure that the server is shut down normally and does not lose any data advantages and disadvantages: suitable for large-scale data recovery if the business does not require high data integrity and consistency RDB is a good choice for data integrity and consistency. It takes up memory to recover data from RDB files during backup.

Copy the dump.rdb file to the bin directory of the installation directory of redis, and restart the redis service

Configure RDB persistence [root@localhost utils] # vim / etc/redis/6379.conf # at least one write within 900 seconds save 900 write operations occur at least 10 times within 300 seconds save 300 write operations occur at least 10000 times within 60 seconds save 60 1000 write operations trigger snapshot operations as long as one of them is satisfied Note all save entries means to turn off RDB#RDB file name dbfilename dump.rdb#RDB file path dir / var/lib/redis/6379# enable compression rdbcompression yesAOF persistence

Redis is not enabled by default

Make up for the deficiency of RDB (data inconsistency)

Record each write in the form of a log and append it to the file

Redis restart will execute the write instruction from front to back according to the contents of the log file to complete the data recovery.

Recover data from AOF files

Copy the appendonly.aof file to the bin directory of the redis installation directory and restart the redis service

AOF persistence configuration [root@localhost utils] # vim / etc/redis/6379.conf # enable AOF persistence appendonly yes#AOF file name appendfilename "appendonly.aof" # always: synchronous persistence, every data change will be immediately written to disk # appendfsync always#everysec: recommended by default, asynchronous records per second (default) appendfsync everysec#no: out of sync Leave it to the operating system to decide how to synchronize the rewriting mechanism of # appendfsync no# ignoring the last potentially problematic instruction aof-load-truncated yesAOF

The working principle of AOF is to append write operations to the file, so that there will be more and more redundant content in the file.

When the size of the AOF file exceeds the set threshold, Redis will compress the contents of the AOF file

The principle of AOF rewriting

Redis will fork a new process, read the data in memory (not the old file), rewrite it to a temporary file, and finally replace the old aof file.

AOF rewrite configuration [root@localhost utils] # vim / etc/redis/6379.conf # when the log is BGREWRITEAOF, if it is set to yes, it means that the new write operation is not synchronized and the fsync,# is only temporarily stored in the buffer to avoid disk I0 operation conflicts and write after the rewrite is completed. By default in redis, the current AOF file size of nono-appendfsync-on-rewrite no# is twice the size of the AOF file when the last log rewrite occurs. The BGREWRITEAOF operation auto-aof-rewrite-percentage 10 executes the minimum BGREWRITEAOF command of the current AOF file. # avoid frequent BGREWRITEAOFauto-aof-rewrite-min-size 64mbRedis performance management due to small file size when starting Reids # # View redis memory usage [root@localhost utils] # / usr/local/redis/bin/redis-cli127.0.0.1:6379 > info memory memory fragmentation rate

● the memory value assigned by the system used_ memory rss divided by the memory value used by redis

Used _ memory calculates

● memory fragmentation is caused by inefficient allocation / recycling of physical memory by the operating system

Discontiguous physical memory allocation

● tracking memory fragmentation rates is very important to understand the resource performance of redis instances.

It is reasonable for the memory fragmentation rate to be slightly greater than 1, which indicates that the memory fragmentation rate is relatively low.

The memory fragmentation rate is more than 1.5, indicating that redis consumes 150% of the actual physical memory, of which 50% is the memory fragmentation rate.

If the memory fragmentation rate is less than 1, the Redis memory allocation exceeds the physical memory, and the operating system is swapping memory.

Memory utilization

The memory usage of the ● redis instance exceeds the maximum available memory, and the operating system will begin to

Exchange of memory and swap space

● avoids memory swapping

Select for cached data size

Use Hash data structures whenever possible

Set the expiration time of key

Recycle key

● ensures rational allocation of redis's limited memory resources

● when memory usage reaches the maximum set threshold, you need to choose a key recycling strategy.

By default, the recycling policy is to disable deletion

Modify the maxmemory-policy property value in the redis.conf configuration file

Volatile-lru: use LRU algorithm to phase out data from data sets that have set expiration time volatile-ttl: select data that is about to expire from data sets that have set expiration time (recommended) volatile-random: randomly pick data from data sets with expiration time to phase out allkeys-lru: use LRU algorithm to phase out data from all data sets allkeys-random: from data sets Select data to phase out no-enviction: data elimination is prohibited

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.