Shulou(Shulou.com)06/02 Report--

Blog outline:

A brief introduction to MemCache

1. Protocol 2, event handling 3, storage mode 4, communication distributed 5, application scenario 6 of memcached, workflow 7 in memcached application, consistent Hash algorithm of memcached

Second, deploy LNMP dynamic and static separation & & memcache cache server 1, environment preparation 2, deploy Nginx server 3, deploy PHP server 4, deploy MySQL database 5, deploy Memcached server 6, deploy memcache client 7, use memcache to achieve session sharing 8, test memcached cache database 1, introduction to MemCache

MemCache is a free, open source, high-performance, distributed distributed memory object caching system, which is used for dynamic Web applications to reduce the load of the database. It reduces the number of times to read the database by caching data and objects in memory, thus improving the speed of website access. MemCaChe is a HashMap that stores key-value pairs. Key-value storage is used in memory for any data (such as strings, objects, etc.). The data can come from database calls, API calls, or the results of page rendering. The design concept of MemCache is small and powerful, and its simple design promotes rapid deployment, easy development and solving many problems facing large-scale data cache. the open API makes MemCache used in most popular programming languages such as Java, C/C++/C#, Perl, Python, PHP, Ruby and so on.

In addition, why are there two names, Memcache and memcached? In fact, Memcache is the name of the project (also the name of its client), and memcached is its server-side main program file name.

Memcached is a key / value system, the system is much simpler than MySQL, although MySQL also has cache, but database SQL parsing will consume performance, query is slower than memcached, in addition, MySQL cache design is more complex, because to consider transactions, logs, storage engine and other modules, its performance is not as good as memcached.

Memcached only does one thing, simple and efficient, and is better than MySQL on cache, which should be easy to understand.

As a high-speed distributed cache server, memcached has the following characteristics:

Simple protocol; event handling based on libevent; built-in memory storage mode; distributed memcached that does not communicate with each other. 1. Agreement

Memcached's server-client communication does not use complex formats such as XML, but uses a simple text-line-based protocol.

Therefore, data can also be saved and obtained on memcached through telnet.

2. Event handling

Libevent is a library that encapsulates event handling functions such as epoll of Linux and kqueue of BSD operating systems into a unified interface. O (1) performance can be achieved even if the number of connections to the server increases. Memcached uses this libevent library, so it can perform its high performance on Linux, BSD, Solaris, and other operating systems.

3. Storage mode

To improve performance, the data saved in memcached is stored in memcached's built-in memory storage space. Because the data exists only in memory, restarting memcached and restarting the operating system will cause all data to disappear. In addition, when the content capacity reaches the specified value, the unused cache is automatically deleted based on the LRU (Least Recently Used) algorithm. Memcached itself is a server designed for caching, so it doesn't think too much about the persistence of the data.

4. Distributed communication

Although memcached is a "distributed" cache server, there is no distributed function on the server side. Memcached does not communicate with each other to share information. So, how to distribute it? It all depends on the implementation of the client.

5. Application scenarios of memcached

1) the front-end cache application of the database: let it share the concurrent pressure of the data. When the data is updated, the program can tell the cache to update.

2) shared storage of session session sharing

6. Workflow in memcached application

It is an in-memory cache, which can read the data cached in memory through API. When users need to read data, they will first access the memcached cache. If there is any data in the cache, it will be directly returned to the front-end application. If not, it will be forwarded to the server at the backend. At this time, in addition to returning the data to the user, the server will update the data to the memcached cache.

If, in the actual production environment, the cache server needs to be restarted (or power is cut off), the data in the cache will be lost, and the concurrent pressure on the replacement server will increase, which may cause the introduced server to shut down as well. Unable to provide services, then our processing flow is as follows:

First stop the WEB application from the load balancer-> make the load balancer stop forwarding data to WEB-- > then start the cache server-> initialize the contents of the database to the cache server through the program-> then enable the web application-> restart the database server

7. Consistent Hash algorithm of memcached

The consistent Hash algorithm implements the Hash mapping from the Key to the cache server through a data structure called the consistent Hash ring. To put it simply, a consistent hash organizes the entire hash space into a virtual ring (this ring is called a consistent Hash ring). For example, assuming that the value space of a space hash function H is 0 ~ 2 ^ 32-1 (that is, the hash value is a 32-bit unsigned integer), the whole hash space is as follows:

Next, each server uses H for a hash calculation. Specifically, you can use the server's IP address or host name as the keyword, so that each machine can determine its position on the above hash ring and arrange it clockwise. Here, we assume that the calculated locations of the three nodes memcache are as follows:

Next, the hash value h of the data is calculated using the same algorithm, and the position of the data on the hash ring is determined. If we have data A, B, C, D, 4 objects, the position after hashing is as follows:

According to the consistent hash algorithm, data An is bound to server01, D is bound to server02, and B and C are clockwise to find the nearest service node on server03.

The hash ring scheduling method thus obtained has high fault tolerance and scalability:

Suppose server03 is down:

You can see that C and B are affected at this time, and the B and C nodes are relocated to Server01. In general, in a consistent hash algorithm, if a server is not available, the data affected is only the data between this server and the previous server in its ring space (that is, the first server encountered walking counterclockwise). The rest will not be affected.

Consider another situation, if we add a server Memcached Server 04 to the system:

At this point, A, D, and C are not affected, only B needs to be relocated to the new Server04. In general, in the consistent hashing algorithm, if you add a server, the data affected is only the data between the new server and the previous server in its ring space (that is, the first server encountered when walking counterclockwise). Nothing else will be affected.

To sum up, the consistent hashing algorithm only needs to relocate a small part of the data in the ring space for the increase or decrease of nodes, so it has good fault tolerance and scalability.

The disadvantage of consistent hash: when there are too few service nodes, it is easy to cause the problem of data skew due to the uneven division of nodes. We can solve the problem by adding virtual nodes.

More importantly, the more cache server nodes in the cluster, the smaller the impact of adding / decreasing nodes, which is easy to understand. In other words, with the increase of the size of the cluster, the probability of continuing to hit the original cached data will increase. Although there is still a small amount of data cached in the server that cannot be read, the proportion is small enough that even if the database is accessed, it will not cause fatal load pressure on the database.

Second, deploy LNMP dynamic and static separation & & memcache cache server 1. Environment preparation

Download the source package I provided, and upload the corresponding source package to each server.

2 、 Deploy Nginx server [root@nginx ~] # yum-y erase httpd # Uninstall the included httpd service [root@nginx ~] # yum-y install openssl-devel pcre-devel # installation depends on [root@nginx ~] # cd / usr/src [root@nginx src] # rz # upload Nginx source code package [root@nginx src] # tar zxf nginx-1.14.0.tar.gz [root@nginx src] # cd nginx-1.14 .0 / [root@nginx nginx-1.14.0] #. / configure-- prefix=/usr/local/nginx-- user=www-- group=www & & make & & make install# to compile and install [root@nginx nginx-1.14.0] # useradd-M-s / sbin/nologin www # create and run user [root@nginx nginx-1.14.0] # ln-sf / usr/local/nginx/sbin/nginx / usr/local/sbin/ # do the command Soft link [root@nginx nginx-1.14.0] # nginx # start the Nginx service [root@nginx nginx-1.14.0] # netstat-anput | grep 80 # make sure that port 80 is listening # below is to configure nginx to associate with the PHP server [root@nginx nginx-1.14.0] # cd / usr/local/nginx/conf/ [root@nginx conf] # vim nginx.conf # Edit the main configuration file # in the server {} field Add the following to location ~\ .php$ {root / var/www/html # specify the web page storage path of the PHP server fastcgi_pass 192.168.20.3 PHP 9000; # specify the PHP server listening port fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME / scripts$fastcgi_script_name; include fastcgi.conf } # after the change is completed Save and exit [root@nginx conf] # nginx-s reload # restart to make the changes take effect 3. Deploy PHP server [root@php ~] # rz # upload source code package libmcrypt and php-5.6.27 [root@php ~] # tar zxf libmcrypt-2.5.7.tar.gz-C / usr/src # unpack [root@php ~] # tar zxf php-5.6.27.tar.gz-C / usr/src # unpack [root@php ~] # yum-y install libxml2-devel openssl-devel bzip2-devel # installation dependency [root@php ~] # cd / usr/src/libmcrypt-2.5.7 # switch to the decompressed path [root@php libmcrypt-2.5.7] # / configure-- prefix=/usr/local/libmcrypt & & make & & make install# edit and install [root@php libmcrypt-2.5.7] # cd.. / php-5.6.27/ # to enter the decompressed path of the PHP source package [root@php php-5 .6.27] # / configure-- prefix=/usr/local/php5.6-- with-mysql=mysqlnd-- with-pdo-mysql=mysqlnd-- with-mysqli=mysqlnd-- with-openssl-- enable-fpm-- enable-sockets-- enable-sysvshm-- enable-mbstring-- with-freetype-dir-- with-jpeg-dir-- with-png-dir-- with-zlib-- with-libxml-dir=/usr-- enable-xml-- with-mhash-- with-mcrypt=/usr / local/libmcrypt-- with-config-file-path=/etc-- with-config-file-scan-dir=/etc/php.d-- with-bz2-- enable-maintainer-zts & & make & & make install# compilation and installation PHP# next to adjust the configuration file of PHP and control the start and stop of the service [root@php php-5.6.27] # cp php.ini-production / etc/php.ini # copy the PHP configuration provided in the source code File [root@php php-5.6.27] # cp sapi/fpm/init.d.php-fpm / etc/init.d/php-fpm # copy its service control script file [root@php php-5.6.27] # chmod + x / etc/init.d/php-fpm # Grant execution permission [root@php php-5.6.27] # chkconfig-- add php-fpm # added as system service To support systemctl management [root@php php-5.6.27] # chkconfig php-fpm on # Open [root@php php-5.6.27] # cp / usr/local/php5.6/etc/php-fpm.conf.default / usr/local/php5.6/etc/php-fpm.conf# copy the default configuration file provided by php-fpm and edit it [root@php php-5.6.27] # vim / usr/local/php5.6/etc / php-fpm.conf # modify it Optimize listen = 192.168.20.3 pm.max_children 9000 # listener address is the local IP9000 port pm.max_children = 50 # maximum number of started processes pm.start_servers = 5 # initial start processes pm.min_spare_servers = 5 # minimum idle process pm.max_spare_servers = 35 # maximum idle process # after modification Save and exit [root@php php-5.6.27] # systemctl start php-fpm # start the PHP service [root@php php-5.6.27] # netstat-anput | grep php-fpm # confirm that its port 9000 has started [root@php ~] # mkdir-p / var/www/html # to create its web page storage directory Must be the same as the web page storage path of the nginx server [root@php php-5.6.27] # vim / var/www/html/index.php # write the PHP test page [root@php php-5.6.27] # vim / var/www/html/index1.php # test script to connect to the database

At this point, you can access port 80 of the Nginx server to view the two web page files defined on the php server (when accessing the script file that connects to the database, you need to deploy the database and create users to connect to):

Visit index.php and see the following page indicating that php is running normally:

Visit index1.php and see the following page, which shows that there is no problem with the coordination between LNMP (I have deployed the MySQL database and created the corresponding user before visiting the address below):

4. Deploy MySQL database

I deploy a simple MySQL database here to provide testing functions.

[root@mysql ~] # rz# upload the mysql.sh script file and mysql-5.7.22-linux provided by me. The source code package [root@mysql ~] # sh mysql.sh # executes the mysql.sh script. After execution, you need to wait a long time for Starting MySQL... SUCCESS! # when this prompt appears, the MySQL deployment is successful mysql: [Warning] Using a password on the command line interface can be insecure.# after installing MySQL successfully, the default database root password is 123 [root@mysql ~] # mysql-uroot-p123 # Log in to MySQL database mysql > create database bbs;mysql > grant all on bbs.* to ljz@ "192.168.20.%" identified by 'pwwd@123' 5 、 Deploy Memcached server [root@mamcache ~] # cd / usr/src # switch to the specified directory [root@mamcache src] # rz # upload the required source code package [root@mamcache src] # ls # that is, upload the following two source code packages: debug kernels libevent-2.0.22-stable.tar.gz memcached-1.4.33.tar.gz [root@mamcache src] # tar zxf libevent-2.0.22- Stable.tar.gz # unpack [root@mamcache src] # tar zxf memcached-1.4.33.tar.gz # unpack [root@mamcache src] # cd libevent-2.0.22-stable/ # change to the extracted directory [root@mamcache libevent-2.0.22-stable] # / configure & & make & & make install # compile and install [root@mamcache libevent-2.0.22-stable] # cd.. / Memcached-1.4.33/ # change to the memcached directory [root@mamcache memcached-1.4.33] # / configure-- prefix=/usr/local/memcached-- with-libevent=/usr/local & & make & & make install# compilation installation [root@mamcache ~] # ls / usr/local/memcached/bin/memcached # confirm that the command is installed / usr/local/memcached/bin/memcached [root@mamcache ~] # ln-sf / usr/local / memcached/bin/memcached / usr/local/bin/ # soft link to the command [root@memcache memcached-1.4.33] # memcached- d-m 1024-l 192.168.20.4-p 11211-u root-c 10240-P / usr/local/memcached/memcached.pid# start the memcached service The above startup parameters are described as follows: the #-d option starts a daemon. #-m the amount of memory allocated to Memcache for use, in MB (default 64MB). #-l the IP address to listen to. (default: INADDR_ANY, all addresses) #-p sets the port on which Memcache's TCP listens, preferably above 1024. #-u the user running Memcache needs to use this parameter to specify the user if it is currently root. The #-c option is the maximum number of concurrent connections running, and the default is 1024. #-P sets the path of the pid file where Memcache is saved. #-M returns an error when memory is exhausted, instead of deleting the item #-f block size growth factor. The default is 1.2 blocks-n minimum allocated space. Key+value+flags defaults to 4 hours-h to display help [root@mamcache ~] # netstat-anput | grep 11211 # make sure that TCP and udp ports are listening. 6. Deploy memcache client (return to PHP server for configuration) [root@php ~] # cd / usr/src [root@php src] # rz # upload memcache-3.0.8.tgz source code package [root@php src] # tar zxf memcache-3.0.8.tgz # unpack [root@php src] # cd memcache-3.0.8/ # enter the unzipped directory [root@php memcache-3.0.8] # / usr/local/php5.6/bin/phpize # generate the command In order to generate the configure file # if you make an error in executing the above command, you need to execute the autoconf package of the "yun-y install autoconf" installation prompt. [root@php memcache-3.0.8] # / configure-- enable-memcache-- with-php-config=/usr/local/php5.6/bin/php-config & & make & & make install# after the above command is executed, the path where the memcache.so is stored will be displayed [root@php memcache-3.0.8] # vim / etc/php.ini # Edit the main configuration file # add the above at the end of the configuration file To specify the storage path of the memcache.so file extension = / usr/local/php5.6/lib/php/extensions/no-debug-zts-20131226/memcache.so [root@php memcache-3.0.8] # systemctl restart php-fpm # restart the php service, and make the changes take effect [root@php memcache-3.0.8] # cd / var/www/html/ [root@php html] # vim test3.php # write the test file # after editing, save and exit This test script displays the version of memcached # and inserts a key-value pair "test=123" with a cache time of 600 seconds, and its ID is "key".

When the client accesses the edited test3.php file, you will see the following:

Install the Telnet command on the memcached server and log in to the cache to see if its key-value pair is available (the following is done on the memcached server):

[root@memcache ~] # yum-y install telnet # install the Telnet command [root@memcache ~] # telnet 192.168.20.4 11211 # Log in to port 11211 of memcached get key # query the key-value pair whose ID is "key", you can see the "test=123" VALUE key 1 66O:8: "stdClass": 2: {SRAV 8: "str_attr"; SJV 4: "test"; SPLV 8: "int_attr"; iRanger 123 } when END# performs the above get verification, it needs to increase the save time value of the key-value pair inserted in the test3.php file by # or revisit it, so as to avoid cache invalidation and query failure.

At this point, LNMP separation & & memcache cache server has been basically deployed. Next, configure PHP to communicate with memcached server to save session session.

7. Use memcache to achieve session sharing (do the following on the PHP server)

Next, the PHP communicates with the memcached server to save the session session.

[root@php ~] # vim / etc/php.ini # add the following at the end of the configuration file: ession.save_handler = memcachesession.save_path = "tcp://192.168.20.4:11211?persistent=1&weight=1&timeout=1&retry_interval=15" # Note the content written above is explained as follows: # session.save_handler: set the storage mode of session to memcache. # session data is accessed by file by default. # session.save_path: set the location of session storage # comma when using multiple memcached server, "separated, # can take extra parameters" persistent "," weight "," timeout "," retry_interval ", and so on, # like this:" tcp://host:port?persistent=1&weight=2,tcp://host2:port2 ". [root@php html] # systemctl restart php-fpm # restart to make the changes take effect [root@php html] # vim / var/www/html/test2.php # write the test file # after writing, save and exit

The client accesses the written test2.php test file as follows:

Similarly, use the Telnet command to query the value of its session_id on the memcached server, as follows:

[root@memcache ~] # telnet 192.168.20.4 11211 # Login cache listening port get naapo2eet2d9s4to4mt7hchnr1 # execute the get command VALUE naapo2eet2d9s4to4mt7hchnr1 0 26session_time | iget naapo2eet2d9s4to4mt7hchnr1 1572450021 is the same as the value accessed by our web page, indicating that it is cached 8. Test the memcached cache database

Create a table for testing on the MySQL database (all operations are on the MySQL database) as follows:

Mysql > create database testdb1; # create a library mysql > use testdb1; # switch to the created library mysql > create table test1 (id int not null auto_increment,name varchar (20) default null,primary key (id)) engine=innodb auto_increment=1 default charset=utf8;# creation table, with two columns: ID number and name mysql > insert into test1 (name) values ('tom1'), (' tom2'), ('tom3'), (' tom4'), ('tom5') # insert test data into the table mysql > select * from test1; # determine the inserted data +-+-+ | id | name | +-+-+ | 1 | tom1 | | 2 | tom2 | | 3 | tom3 | 4 | tom4 | | 5 | tom5 | +-+-+ 5 rows in set (0.00 sec) mysql > desc test1 # you can execute this statement to view the table structure of test1 mysql > grant select on testdb1.* to user@'%' identified by 'pwd@123';# to create a read-only database user on the test1 table for subsequent testing

The next step is to test. Here is a php script to test whether memcache cached data successfully.

Write the following test file on the PHP server:

The client accesses the script file for the test, and the page visited for the first time is as follows:

After the client refreshes, you will see the following page:

Before the cache query expires, you can obtain the corresponding cache data on the memcache through get, as follows (operate on the memcache server):

[root@memcache ~] # telnet 192.168.20.4 11211 # Log in to the cache listening port get d8c961e9895ba4b463841924dbcefc2b # execute the get command VALUE d8c961e9895ba4b463841924dbcefc2b 0 251a:5: {iRAPR 0: "id"; SRAV 1: "1"; SRAV 4: "name"; SRAV 4: "tom1";} iRule 1: "id"; SSR 1: "2"; SER 4: "name"; SSR 4: "tom2" } itom3 2: {id 2: "id"; SJV 1: "3"; SLAR 4: "name"; SLAR 4: "tom3";} iRover 3: "tom5"; SJV 4: "name"; SRAV 4: "tom4";} iRV 4: "id"; SSR 1: "5"; SPLR 4: "name"; SER 4: "tom5";}

-this is the end of this article. Thank you for reading-

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.