Shulou(Shulou.com)06/02 Report--

1. SQL and NoSQL1, SQL

1. A structured database, created on the basis of relational model, is generally record-oriented.

2. Including Oracle, Mysql, SQL Server, Microsoft Access, DB2 and so on.

Advantages:

1. Easy to maintain: all use table structure with consistent format

2. Easy to use: SQL language is universal and can be used for complex queries.

3. Complex operation: SQL is supported, which can be used for very complex queries between one table and multiple tables.

Disadvantages:

1. Poor reading and writing performance, especially efficient reading and writing of massive data.

2. Fixed watch structure, less flexibility

3. For the high concurrent read and write requirements, the hard disk Ipicuro is a big bottleneck for the traditional relational database.

2 、 NoSQL

1. All databases except the mainstream relational databases are considered to be non-relational.

2. Including Redis, MongBD, Hbase, CouhDB, Memcache, etc.

3. Strictly speaking, a non-relational database is not a database, but a collection of structured data storage methods, which can be documents or key-value equivalence.

Advantages:

1. Flexible format: data can be stored in key-value (key-value) form, document form, picture form, etc., flexible in use and widely used in a wide range of applications, while relational databases only support basic types

2. High speed: nosql can use hard disk or random access memory as carrier, while relational database can only use hard disk

3. High scalability

4. Low cost: nosql database is easy to deploy and is basically open source software.

Disadvantages:

1. Without sql support, the cost of learning and using is high.

2. Transaction is not supported and fault tolerance rate is low.

3. The data structure is relatively complex, and the complex query is a little short.

3. Background introduction of NoSQL.

(1) High performance: high concurrent read and write requirements for database

(2) Huge Storage: requirements for efficient storage and access of massive data

(3) High Scalability & & High Availability: requirements for high scalability and high availability of the database.

II. Redis

(1) Redis runs based on memory and supports persistence

(2) adopt the storage form of key-value (key-value pair)

(3) advantages:

Has extremely high data reading and writing speed

Support for rich data types

Support data persistence

Atomicity

Support for data backup

(4) comparison between Redis and Memcache:

1. Storage method:

Memecache stores all the data in memory and will hang up when the power is off. The data cannot exceed the memory size.

Some of the Redis is stored on the hard disk, which ensures the persistence of the data.

2. Data support type:

Memcache's support for data types is relatively simple

Redis has complex data types.

3. Using the underlying model is different:

The underlying implementation between them and the application protocol for communicating with the client are different.

Redis directly builds its own VM mechanism, because the general system calls system functions, it will waste a certain amount of time to move and request.

4. Value size:

Redis can achieve a maximum of 1GB, while memcache has only 1MB.

5. What are the benefits of using redis?

1. Fast, because the data is stored in memory. The advantage similar to HashMap,HashMap is that the time complexity of search and operation is O (1).

2. Support rich data types and support string,list,set,sorted set,hash

3, support transactions, operations are atomicity, the so-called atomicity is either all or none of the changes to the data.

4, rich features: can be used for caching, messages, press key to set the expiration time, after expiration will be automatically deleted.

Advantages of redis over memcached:

All values of memcached are simple strings, and redis, as its replacement, supports richer data types.

Redis is much faster than memcached.

Redis can persist its data

Memcache supports structured, but Redis supports structured and unstructured.

3. Install and deploy redis

1. Install the redis environment

Latest installation package: redis-5.0.7.tar.gz

System: Linux

2. Operation steps

Step 1: download the installation package on windows to Linux locally by remote mount.

Step 2: install the compilation tool

Yum install gcc gcc-c++ make-y

Step 3: decompress

Tar zxvf redis-5.0.7.tar.gz-C / opt/

Step 4: install

Cd redis-5.0.7/

Makemake PREFIX=/usr/local/redis install

Step 5: create a soft connection

Ln-s / usr/local/redis/bin/* / usr/local/bin/

Step 5: start the installation script to check the port status

Cd / redis-5.0.7/utils/./install_server.sh # # start the script. Note: enter all the way to the end. Change the last file path to: / usr/local/redis/bin/redis-servernetstat-natp | grep 6379

In addition, you can also use redis_6379 to control its switch on and off:

/ etc/init.d/redis_6379 stop / / close / etc/init.d/redis_6379 start / / enable / etc/init.d/redis_6379 restart / / restart redis-cli remote connection operation:

1. Modify the configuration file and add the IP address of the host.

Vim / etc/redis/6379.confbind 127.0.0.1 192.168.220.131 add the IP address of the host. If you don't add the listening address, you will not be able to connect

As shown in the figure:

2. Restart the service

3. Remote connection:

Redis-cli-h 192.168.220.131-p 6379Universe Mameh: specify host IP address,-p: specify port

IV. Redis configuration file

Configuration parameters (/ etc/redis/6379.conf):

5. Common commands in redis database

1. Common commands

(1) to connect to the local database

/ usr/local/redis/bin/redis-cli

(2) connect to remote database

Redis-cli-h 192.168.220.131-p 6379

(3) storing data: set

(4) obtain data: get

2. Key command

(1) keys: get the list of key values that conform to the rule

> keys * / View all keys in the current database > keys v* / / View keys in the current database that begin with v > keys v? / / View keys in the current database that start with v and contain any character > keys vkeys? / / View keys in the current database that start with v and then contain any two characters

(2) exists: judge whether the key value exists

(3) del: delete the specified key of the current database

(4) type: get the value value type corresponding to key

(5) rename (overwrite) / renamenx (not overwrite): rename the existing key:

(6) dbsize: view the number of key in the current database

(7) redis-benchmark testing tools:

Redis-benchmark-h 192.168.220.131-p 6379-c 100-n 100000amp / send 100 concurrent connections and 100000 requests to the redis server with IP address 192.168.220.131 and port 6379 to test performance redis-benchmark-h 192.168.220.131-p 6379-Q-d 100amp / Test the performance of packets with access size of 100byte

3. Redis multi-data operation

(1) Redis supports multiple databases. By default, 16 databases and 0-15 names are supported.

(2) multiple data are independent and do not interfere with each other.

(3) Common commands for multiple data

1. Select: switching between multiple databases (default is in database 0)

2. Move: moving data between multiple databases

3. Flushdb: clear the data in the database

6. Redis persistence:

(1) Overview of persistence:

Redis runs in memory, and the data in memory will be lost when the power is off.

In order to reuse Redis data, or to prevent system failure, you need to write the data in Redis to disk space, that is, persistence.

(2) persistence classification:

1. RDB method: to obtain copies of all the data in Redis at a certain time by creating snapshots.

2. AOF mode: write the executed write command to the end of the file and record the changes of the data in the way of log.

(3) RDB persistence:

1. It is the default persistence method of Redis, and the default file name is dump.rdb

2. Trigger conditions:

-performs a specified number of writes within a specified time interval (profile control)

-execute save or bgsave (asynchronous) commands

-execute the flushall command to clear all data in the database

-execute the shutdown command to ensure that the server shuts down properly and does not lose any data.

3. Advantages and disadvantages:

Suitable for large-scale data recovery

If the business does not require high data integrity and consistency, RDB is a good choice.

The integrity and consistency of data is not high.

Memory is consumed during backup.

4. Recover data through RDB file:

Copy the dump.rdb file to the bin directory of redis's installation directory and restart the redis service.

5. Configuration file options

Vim / etc/redis/6379.confsave 9001 save 300 10save 60 10000 Compact at least one write within 900 seconds, at least 10 writes within 300 seconds, and at least 10000 writes within 60 seconds. Snapshot operations will be triggered as long as one of them is satisfied Comment all save entries to close RDBdbfilename dump.rdb / / RDB file name dir / var/lib/redis/6379 / / RDB file path rdbcompression yes / / whether to compress or not

(4) AOF persistence:

Redis is not enabled by default

Make up for the deficiency of RDB (data inconsistency)

Record each write in the form of a log and append it to the file

Redis restart will execute the write instruction from front to back according to the contents of the log file to complete the data recovery.

1. Recover data according to AOF file:

Copy the appendonly.aof file to the bin directory of the installation directory of redis, and restart redis access.

2. Configuration file options

Vim / etc/redis/6379.confappendonly yes / / enable AOF persistence appendfilename "appendonly.aof" / / AOF file name # appendfsync always appendfsync everysec#appendfsync no// explanation: always: synchronous persistence, every time the data changes will be immediately written to disk; everysec: recommended by default, asynchronous record once per second (default); no: do not synchronize, let the operating system decide how to synchronize. Aof-load-truncated yes / / ignore the last instruction that may have a problem

3. Rewriting mechanism of AOF:

AOF works by appending write operations to files, resulting in more and more file redundancy.

When the size of the AOF file exceeds the set threshold, Redis will compress the contents of the AOF file

4. The rewriting principle of AOF:

Redis will fork a new process, read the data in memory (not the old file), return to a temporary file, and finally replace the old aof file

5. Rewrite configuration of AOF:

When vim / etc/redis/6379.conf is BGREWRITEAOF in the log, setting it to yes means that the new write operation does not synchronize fsync, but is temporarily stored in the buffer to avoid disk IO operation conflicts, and write after the rewrite is completed. By default, the current AOF file size of nono-appendfsync-on-rewrite no// in redis is twice the size of the AOF file in the last log rewrite, and the BGREWRITEAOF operation auto-aof-rewrite-percentage 100max / the minimum value for the current AOF file to execute the BGREWRITEAOF command occurs to avoid frequent BGREWRITEAOFauto-aof-rewrite-min-size 64mb7 and Redis performance management due to the small file size at the beginning of starting redis.

1. Memory fragmentation rate

(1) calculated by dividing the memory value used_memory_rss allocated by the operating system by the memory value used_memory used by redis

(2) memory fragmentation is caused by inefficient allocation / recovery of physical memory by the operating system.

Discontiguous physical memory allocation

(3) tracking the memory fragmentation rate is very important to understand the resource performance of redis instances.

-it is reasonable for the memory fragmentation rate to be slightly greater than 1, which indicates that the memory fragmentation rate is relatively low.

-the memory fragmentation rate exceeds 1.5, indicating that redis consumes 150% of the actual physical memory, of which 50% is the memory fragmentation rate.

-if the memory fragmentation rate is less than 1, the Redis memory allocation exceeds the physical memory, and the operating system is swapping memory.

2. Memory utilization

(1) the memory utilization of the redis instance exceeds the maximum available memory, and the operating system begins to exchange memory with swqp space.

(2) avoid memory swapping:

-Select for cached data size

-use Hash data structures whenever possible

-set the expiration time of key

3. Recycle key

1. Ensure rational allocation of redis's limited memory resources.

2. When the memory usage reaches the set maximum threshold, you need to choose a recycling policy for key:

-Recycling policy is prohibited by default

-modify the maxmemory-policy attribute value in the redis.conf configuration file

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.