Shulou(Shulou.com)06/01 Report--

This article mainly introduces the definition and analysis of mysql-mmm master replication, hoping to supplement and update some knowledge for you. If you have any other questions you need to know, you can continue to follow my updated article in industry information.

Mysql-MMM master master replication

MMM (Master-Master replication manager for MySQL) is a set of script programs that support dual-master failover and dual-master daily management. It mainly manages dual-master replication. In fact, in the application, only one master is responsible for writing operations, the other is standby redundancy, or part of the read operation. You can also separate read and write requests in combination with master-slave replication.

The mmm is divided into the agent side and the monitor side. The agent side is deployed on the database node, and the monitor is deployed on the monitoring and management side. Monitor is the only single point of failure in the entire database cluster, because the monitor side only replicates, monitors and manages agent nodes, so the task is very light, and generally there will not be any failure. Moreover, after the monitor end assigns the vip address to the agent database CVM, even if the monitor service is stopped, the business will not be affected, and only needs to be repaired in time. I really don't trust that I can deploy keepalived for monitor.

Environment

Hostname: system: IP address: installation software: database role m1centos6.5192.168.100.150mysql mysql-server mysql-mmm*masterm2centos6.5192.168.100.151mysql mysql-server mysql-mmm*masterm3centos6.5192.168.100.152mysql mysql-server mysql-mmm*slavem4centos6.5192.168.100.153mysql mysql-server mysql-mmm*slavemonitorcentos6.5192.168.100.154mysql mysql-mmm*monitor monitoring

Database role, and corresponding vip:

Hosts: vip role

M1192.168.100.250

Write is responsible for writing operations m2write does not normally work, standby redundant m3192.168.100.201read read operations m4192.168.100.202read read operations

Modify hostname: modify M1 m2 m3 m4 monitor in turn

Install the mysql service in M1-M4:

[root@m1 ~] # yum-y install mysql mysql-server mysql-devel [root@m2 ~] # yum-y install mysql mysql-server mysql-devel [root@m3 ~] # yum-y install mysql mysql-server mysql-devel [root@m4 ~] # yum-y install mysql mysql-server mysql-devel

Modify the mysql main configuration file for M1:

[root@m1 ~] # vi / etc/my.cnf [mysqld] datadir=/var/lib/mysqlsocket=/var/lib/mysql/mysql.socklog-error=/var/lib/mysql/mysql.errslow_query_log_file=/var/lib/mysql/slow_query_log.loguser=mysqlcharacter-set-server=utf8log-bin=mysql-binserver-id=150binlog-ignore-db=mysql Information_schemalog-slave-updatessync_binlog=1auto_increment_increment=2auto_increment_offset= 1[client] default_character_set= utf8[mysqld _ safe] log-error=/var/log/mysqld.logpid-file=/var/run/mysqld/mysqld.pid

Start the mysql service to see if it is started

[root@m1 ~] # / etc/init.d/mysqld start [root@m1 ~] # netstat-utpln | grep 3306

Tcp 0 0 0.0.0.0 3306 0.0.0.015 * LISTEN 1359/mysqld

Synchronize the main configuration files of mysql to m2, m3, M4 servers

The authenticity of host '192.168.100.151 (192.168.100.151)' can't be established.RSA key fingerprint is 6b:3d:8b:50:dc:45:ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.151' (RSA) to the list of known hosts.root@192.168.100.151's password: my.cnf 100% 465 0.5KB/s 00:00 The authenticity of host' 192.168.100.152 (192.168.100.152) 'can't be established.RSA key fingerprint is 6b:3d:8b:50:dc:45 Ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.152' (RSA) to the list of known hosts.root@192.168.100.152's password: my.cnf 100% 465 0.5KB/s 00:00 The authenticity of host '192.168.100.153 (192.168.100.153)' can't be established.RSA key fingerprint is 6b:3d:8b:50:dc:45 Ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.153' (RSA) to the list of known hosts.root@192.168.100.153's password: my.cnf 100% 465 0.5KB/s 00:00

Log in to the database to view the bin-log file name and offset of the database:

Make a note of the File Position information, which will be used in the m1-m2-m3 database later.

[root@m1 ~] # mysqladmin-uroot password 123123 [root@m1 ~] # mysql-uroot-p123123mysql > show master status +-- + | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +-- -+-- + | mysql-bin.000003 | 249 | | mysql Information_schema | +-- + 1 row in set (0.00 sec)

Authorization allows replication

Mysql > grant replication slave on *. * to 'replication'@'192.168.100.%' identified by' 123123 query OK, 0 rows affected (0.02 sec) # # authorize replication users to have database replication permissions on the network segment 192.168.100.0 mysql > flush privileges; # # refresh permissions Query OK, 0 rows affected (0.00 sec) mysql > quitBye

Change m2 database configuration file

This is the configuration file synchronized by the M1 server. You just need to change the server-id. Server-id is the unique identifier in the mysql cluster and is defined here with the ip address at the end.

[root@m2 ~] # sed-I'/ server-id/s/150/151/g' / etc/my.cnf [root@m2 ~] # grep id/ etc/my.cnf server-id=151pid-file=/var/run/mysqld/mysqld.pid

Start logging into the database

[root@m2 ~] # / etc/init.d/mysqld start [root@m2 ~] # mysqladmin-uroot password 123123 [root@m2 ~] # mysql-uroot-p123123

View the bin-log file on m2 and record the offset, and authorize the user to copy permissions

Mysql > show master status +-- + | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +-- -+-- + | mysql-bin.000003 | 249 | | mysql Information_schema | +-+ 1 row in set (0.00 sec) mysql > grant replication slave on *. * to 'replication'@'192.168.100.%' identified by' 123123' Query OK, 0 rows affected (0.01 sec) mysql > flush privileges;Query OK, 0 rows affected (0.05 sec)

Add another primary m2 synchronization data on M1

[root@m1 ~] # mysql- uroot-p123123mysql > change master to-> ip of master_host='192.168.100.151', # # m2-> authorized user on master_user='replication', # # m2-> password of authorized user on master_password='123123', # # m2-> bin-log file of master_log_file='mysql-bin.000003', # # m2 (just found on m2)-> master_log_pos=249 # # offset of log files Query OK, 0 rows affected (0.08 sec)

Start synchronization after the assignment

Mysql > start slave;Query OK, 0 rows affected (0.00 sec)

View synchronization status information:

Both of these items are considered successful for yes.

Slave_IO_Running: Yes

Slave_SQL_Running: Yes

Mysql > show slave status\ G * * 1. Row * * Slave_IO_State: Waiting for master to send event Master_Host: 192.168.100.151 Master_User: replication Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000003 Read_Master_Log_Pos: 495 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 497 Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 495 Relay_Log_Space: 653 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.00 sec) ERROR: No query specified

Add another primary M1 of m2 to m2:

Mysql > change master to-> master_host='192.168.100.150',-> master_user='replication',-> master_password='123123',-> master_log_file='mysql-bin.000003',-> master_log_pos=249;Query OK, 0 rows affected (0.10 sec) mysql > start slave;Query OK, 0 rows affected (0.00 sec) mysql > show slave status\ G * * 1. Row * * Slave_IO_State: Waiting for master to send event Master_Host: 192.168.100.150 Master_User: replication Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000003 Read_Master_Log_Pos: 741 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 497 Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 741 Relay_Log_Space: 653 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.00 sec) ERROR: No query specified

Verify:

Create libraries on M1 and m2 respectively to verify synchronization:

Created on M1

Mysql > create database M1 sec OK query OK, 1 row affected (0.01 sec) mysql > show databases +-+ | Database | +-+ | information_schema | | m1_test | | mysql | | test | | +-+ 5 rows in set (0.00 sec)

View on m2

Mysql > show databases;+-+ | Database | +-+ | information_schema | | m1_test | | mysql | | test | +-+ 4 rows in set (0.07 sec)

Build a new library on m2

Mysql > create database test_m2;Query OK, 1 row affected (0.04 sec)

View on M1

Mysql > show databases;+-+ | Database | +-+ | information_schema | | m1_test | | mysql | | test | | test_m2 | +-+ 5 rows in set (0.00 sec)

It's synchronizing normally.

Specify the master database on the m3-m4 slave database

[root@m3 ~] # sed-I'/ server-id/s/150/152/g' / etc/my.cnf [root@m3 ~] # grep id/ etc/my.cnf server-id=152pid-file=/var/run/mysqld/mysqld.pid [root@m3 ~] # / etc/init.d/mysqld start [root@m3 ~] # mysqladmin-uroot password 123123 [root@m3] # mysql-uroot-p123123mysql > change master to-> master_host='192.168.100.150' -> master_user='replication',-> master_password='123123',-> master_log_file='mysql-bin.000003',-> master_log_pos=249 Query OK, 0 rows affected (0.08 sec) mysql > start slave;Query OK, 0 rows affected (0.00 sec) mysql > show slave status\ G * * 1. Row * * Slave_IO_State: Waiting for master to send event Master_Host: 192.168.100.150 Master_User: replication Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000003 Read_Master_Log_Pos: 929 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 931 Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 929 Relay_Log_Space: 1087 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.04 sec) ERROR: No query specifiedmysql > show databases-> +-+ | Database | +-+ | information_schema | | m1_test | | mysql | | test | | test_m2 | +-+ 5 rows in set (0.08 sec) mysql > [root@m4 ~] # sed-I'/ server-id/s/150/153/g' / etc/my.cnf [root@m4 ~] # grep id/ etc/my.cnf server-id=153pid-file=/var/run/mysqld/mysqld.pid [root@m4 ~] # / etc/init.d/mysqld start [root@m4 ~] # mysqladmin-uroot password 123123 [root@m4] # mysql-uroot-p123123mysql > change master to-> master_host='192.168.100.150' -> master_user='replication',-> master_password='123123',-> master_log_file='mysql-bin.000003',-> master_log_pos=249 Query OK, 0 rows affected (0.10 sec) mysql > start slave;Query OK, 0 rows affected (0.00 sec) mysql > show slave status\ G * * 1. Row * * Slave_IO_State: Waiting for master to send event Master_Host: 192.168.100.150 Master_User: replication Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000003 Read_Master_Log_Pos: 929 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 931 Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 929 Relay_Log_Space: 1087 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.00 sec) ERROR: No query specifiedmysql > show databases +-+ | Database | +-+ | information_schema | | m1_test | | mysql | | test | | test_m2 | +-+ 5 rows in set (0.00 sec)

Install mysql-mmm:

Download the epel extension yum source: execute on the server used

Wget-O / etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo

Install mmm software: execute on all servers

Yum-y install mysql-mmm*

Authorization: authorization on M1, and other hosts will automatically synchronize permissions

[root@m1] # mysql-uroot-p123123mysql > grant replication client on *. * to 'mmm_monitor'@'192.168.100.%' identified by' monitor';Query OK, 0 rows affected (0.02 sec) mysql > grant super,replication client,process on *. * to 'mmm_agent'@'192.168.100.&' identified by' agent';Query OK, 0 rows affected (0.00 sec) mysql > flush privileges;Query OK, 0 rows affected (0.00 sec)

Modify configuration file: mmm_common.conf (common profile for all nodes)

[root@monitor ~] # vi / etc/mysql-mmm/mmm_common.conf

Active_master_role writer cluster_interface eth0 pid_path / var/run/mysql-mmm/mmm_agentd.pid bin_path / usr/libexec/mysql-mmm/ replication_user replication # # user name of replication synchronization replication_password 123123 # # password of synchronization agent_user mmm_agent # # use of Agent Account name agent_password agent # # proxy password ip 192.168.100.150 mode master peer db2 ip 192.168.100.151 mode master peer db1 ip 192.168.100.152 mode slave ip 192.168.100.153 mode slave hosts db1 Db2 ips 192.168.100.250 # # vip mode exclusive # # exclusive mode hosts db3, db4 ips 192.168.100.201, 192.168.100.202 # read vip mode balanced # # balanced mode

Synchronize the configuration file to M1 m2 m3 m4:

[root@monitor] # for i in 150 151 152 153 X do scp / etc/mysql-mmm/mmm_common.conf root@192.168.100.$i:/etc/mysql-mmm/;doneThe authenticity of host '192.168.100.150 (192.168.100.150)' can't be established.RSA key fingerprint is 6b:3d:8b:50:dc:45:ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.150' (RSA) to the list of known hosts.root@192.168.100.150's password: mmm_common.conf 100% 851 0.8KB/s 00:00 The authenticity of host' 192.168.100.151 (192.168.100.151) 'can't be established.RSA key fingerprint is 6b:3d:8b:50:dc 45:ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.151' (RSA) to the list of known hosts.root@192.168.100.151's password: mmm_common.conf 100% 851 0.8KB/s 00:00 The authenticity of host' 192.168.100.152 (192.168.100.152) 'can't be established.RSA key fingerprint is 6b:3d:8b:50:dc 45:ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.152' (RSA) to the list of known hosts.root@192.168.100.152's password: mmm_common.conf 100% 851 0.8KB/s 00:00 The authenticity of host '192.168.100.153 (192.168.100.153)' can't be established.RSA key fingerprint is 6b:3d:8b:50:dc 45:ea:58:38:6e:5d:0d:9b:8f:04:f2.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '192.168.100.153' (RSA) to the list of known hosts.root@192.168.100.153's password: mmm_common.conf 100851 0.8KB/s 00:00

Modify the database mysql-m1 to mysql-m4:

[root@m1 ~] # vi / etc/mysql-mmm/mmm_agent.conf [root@m1 ~] # cat / etc/mysql-mmm/mmm_agent.confinclude mmm_common.confthis db1 [root@m2 ~] # vi / etc/mysql-mmm/mmm_agent.conf [root@m2] # cat / etc/mysql-mmm/mmm_agent.confinclude mmm_common.confthis db2 [root@m3 ~] # vi / etc/mysql-mmm/mmm_agent.conf [root@m3 ~] # cat / etc/mysql-mmm/mmm_agent.confinclude mmm_common.confthis db3 [root@m4 ~] # vi / etc/mysql-mmm/mmm_agent.conf [root@m4 ~] # cat / etc/mysql-mmm/mmm_agent.confinclude mmm_common.confthis db4 [root@monitor ~] # vi / etc/mysql-mmm/mmm_mon.conf [root@monitor ~] # cat / etc/mysql-mmm/mmm_mon.confinclude mmm_common.conf ip 127 .0.0.1 pid_path / var/run/mysql-mmm/mmm_mond.pid bin_path / usr/libexec/mysql-mmm status_path / var/lib/mysql-mmm/mmm_mond.status ping_ips 192.168.100.150 192.168.100.151,192.168.100.152,192.168.100.153 # # Monitoring each node database: modify to the real ip address of each server auto_set_online 60 # # automatic online time Monitor_user mmm_monitor # # username of the monitoring service monitor_password monitor # # password of the monitoring service. These two entries are debug 0 authorized on M1.

Startup service: mmm-agent service of m1-m4; mmm-monitor service of monitoring side

[root@m1 ~] # / etc/init.d/mysql-mmm-agent startStarting MMM Agent Daemon: [OK] [root@m1] # [root@m2 ~] # [root@m2 ~] # / etc/init.d/mysql-mmm-agent startStarting MMM Agent Daemon: [OK] [root@m3 ~] # / etc/init. D/mysql-mmm-agent startStarting MMM Agent Daemon: [OK] [root@m4 ~] # / etc/init.d/mysql-mmm-agent startStarting MMM Agent Daemon: [OK]

[root@monitor ~] # / etc/init.d/mysql-mmm-monitor startStarting MMM Monitor Daemon: [OK]

View the status of each agent database

[root@monitor] # mmm_control show db1 (192.168.100.150) master/ONLINE. Roles: writer (192.168.100.250) db2 (192.168.100.151) master/ONLINE. Roles: db3 (192.168.100.152) slave/ONLINE. Roles: reader (192.168.100.201) db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.202)

# # find that the request for writing is sent to the vip of db1, and the request for reading is sent to db3 db4 vip

Log in to the database using vip:

First authorize on M1: because synchronization is set, only one database is authorized, and other databases will synchronize permissions.

[root@m1 ~] # mysql-uroot-p123123-smysql > grant all on *. * to 'root'@192.168.100.154 identified by' 123123 transactions politics MySQL > flush privileges; [root@monitor ~] # mysql-uroot-p123123-h 192.168.100.250-smysql > show databases;Databaseinformation_schemam1_testmysqltesttest_m2

Log in to read the database:

[root@monitor] # mysql-uroot-p123123-h 192.168.100.201-smysql > show databases;Databaseinformation_schemam1_testmysqltesttest_m2mysql >

In a production environment, you only need to specify the vip address for writing data and the vip address pool for reading data on the application server.

Test:

Simulate the failure of the main server M1, stop the mysql, and check the status:

[root@m1 ~] # / etc/init.d/mysqld stop stop mysqld: [OK] [root@m1 ~] # [root@monitor ~] # mmm_control show db1 (192.168.100.150) master/HARD_OFFLINE. Roles: # # shows as offline db2 (192.168.100.151) master/ONLINE. Roles: writer (192.168.100.250) # # vip transferred to db2 db3 (192.168.100.152) slave/ONLINE. Roles: reader (192.168.100.201) db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.202)

Then start M1.

[root@m1] # / etc/init.d/mysqld start is starting mysqld: [OK] [root@m1 ~] # [root@monitor ~] # mmm_control show db1 (192.168.100.150) master/AWAITING_RECOVERY. Roles: # # restore status db2 (192.168.100.151) master/ONLINE. Roles: writer (192.168.100.250) db3 (192.168.100.152) slave/ONLINE. Roles: reader (192.168.100.201) db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.202) [root@monitor] # mmm_control show db1 (192.168.100.150) master/ONLINE. Roles: # # presence db2 (192.168.100.151) master/ONLINE. Roles: writer (192.168.100.250) db3 (192.168.100.152) slave/ONLINE. Roles: reader (192.168.100.201) db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.202)

Simulate the failure of the slave database m3, stop the database mysql to check the status, and then start to check the status.

[root@m3 ~] # / etc/init.d/mysqld stop stop mysqld: [OK]

At this point, the vip reading database db3 will be transferred to db4;db4, which is temporarily responsible for the read operation.

[root@monitor] # mmm_control show db1 (192.168.100.150) master/ONLINE. Roles: db2 (192.168.100.151) master/ONLINE. Roles: writer (192.168.100.250) db3 (192.168.100.152) slave/HARD_OFFLINE. Roles: db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.201), reader (192.168.100.202) [root@m3 ~] # / etc/init.d/mysqld start is starting mysqld: [OK] [root@m3 ~] #

After the db3 database returns to normal, vip will transfer back and accept the read operation from the new job.

[root@monitor] # mmm_control show db1 (192.168.100.150) master/ONLINE. Roles: db2 (192.168.100.151) master/ONLINE. Roles: writer (192.168.100.250) db3 (192.168.100.152) slave/ONLINE. Roles: reader (192.168.100.201) db4 (192.168.100.153) slave/ONLINE. Roles: reader (192.168.100.202)

Read the above definition and analysis of mysql-mmm master replication, I hope it can bring some help to everyone in practical application. Due to the limited space in this article, it is inevitable that there will be deficiencies and need to be supplemented. If you need more professional answers, you can contact us on the official website for 24-hour pre-sales and after-sales to help you answer questions at any time.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.