Codis 3.2Cluster stand-alone environment installation

Codis 3.2Cluster stand-alone environment installation

I. Software environment

System: centos 6.5

Jdk:jdk-8u111-linux-x64.gz

Go: go1.7.3.linux-amd64.tar.gz

Zookeeper: zookeeper-3.4.6.tar.gz

Native IP:172.16.40.131

Software download address:

Codis: https://github.com/CodisLabs/codis

Zookeeper: https://zookeeper.apache.org/

Go: http://golangtc.com/download

2. Deploy zookeeper clusters

1. Install related dependency packages

Yum install-y gcc makegcc-c++ automake lrzsz openssl-devel zlib-* bzip2-* readline* git nmap unzipwget lsof xz net-tools mercurial

two。 Install java

Tar zxvfjdk-8u111-linux-x64.gz-C / usr/local/

Cd / usr/local/

Ln-sv jdk1.8.0_111 java

Modify environment variabl

Vi / etc/profile

JAVA_HOME=/usr/local/java

PATH=$JAVA_HOME/bin:$PATH

CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

Export JAVA_HOME

Export PATH

Export CLASSPATH

Let the environment variable take effect

Source / etc/profile

View JAVA version

Java-version

Java version "1.8.0,111"

Java (TM) SE Runtime Environment (build1.8.0_111-b14)

Java HotSpot (TM) 64-Bit Server VM (build25.111-b14, mixed mode)

3. Install zookeeper

Mkdir / {app,appdata}

Cd / app

Mkdir-pv. / {zk1,zk2,zk3} / {data,log}

Tar zxvf zookeeper-3.4.6.tar.gz-C / app/zk1/

Tar zxvf zookeeper-3.4.6.tar.gz-C / app/zk2/

Tar zxvf zookeeper-3.4.6.tar.gz-C / app/zk3/

Ln-sv / app/zk1/zookeeper-3.4.6 / app/zk1/zookeeper

Ln-sv / app/zk2/zookeeper-3.4.6 / app/zk2/zookeeper

Ln-sv / app/zk3/zookeeper-3.4.6 / app/zk3/zookeeper

Cp / app/zk1/zookeeper/conf/zoo_sample.cfg/app/zk1/zookeeper/conf/zoo.cfg

Modify zoo.cfg configuration file

Vi / app/zk1/zookeeper/conf/zoo.cfg

# The number of milliseconds of each tick

TickTime=2000

# The number of ticks that the initial

# synchronization phase can take

InitLimit=10

# The number of ticks that can pass between

# sending a request and getting anacknowledgement

SyncLimit=5

# the directory where the snapshot isstored.

# do not use / tmp for storage, / tmp here isjust

# example sakes.

DataDir=/app/zk1/data

# the port at which the clients willconnect

ClientPort=2181

# the maximum number of client connections.

# increase this if you need to handle moreclients

# maxClientCnxns=60

#

# Be sure to read the maintenance sectionof the

# administrator guide before turning onautopurge.

#

# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain indataDir

# autopurge.snapRetainCount=3

# Purge task interval in hours

# Set to "0" to disable autopurge feature

# autopurge.purgeInterval=1

Server.1=172.16.40.131:2881:3888

Server.2=172.16.40.131:2882:3888

Server.3=172.16.40.131:2883:3888

Generate myid

Echo "1" > / app/zk1/data/myid

Configure zk2, zk3:

Cp / app/zk1/zookeeper/conf/zoo.cfg / app/zk2/zookeeper/conf/

Cp / app/zk1/zookeeper/conf/zoo.cfg / app/zk3/zookeeper/conf/

Sed-I's Universe zk1 *

Sed-I's Universe zk1 *

Sed-I's app/zk2/zookeeper/conf/zoo.cfg 2181 / app/zk2/zookeeper/conf/zoo.cfg 2181 / 2182

Sed-I's Compact 2181 / app/zk3/zookeeper/conf/zoo.cfg 2181 / app/zk3/zookeeper/conf/zoo.cfg

Echo "2" > / app/zk2/data/myid

Echo "3" > / app/zk3/data/myid

Start the zookeeper service

/ app/zk1/zookeeper/bin/zkServer.sh start

/ app/zk2/zookeeper/bin/zkServer.sh start

/ app/zk3/zookeeper/bin/zkServer.sh start

View zookeeper status

/ app/zk1/zookeeper/bin/zkServer.sh status

JMX enabled by default

Using config:/app/zk1/zookeeper/bin/../conf/zoo.cfg

Mode: leader

/ app/zk2/zookeeper/bin/zkServer.sh status

JMX enabled by default

Using config:/app/zk2/zookeeper/bin/../conf/zoo.cfg

/ app/zk3/zookeeper/bin/zkServer.sh status

JMX enabled by default

Using config:/app/zk3/zookeeper/bin/../conf/zoo.cfg

Mode: follower

II. Deploy codis

Deployment of 1.go environment

Cd / app

Tar zxvf go1.7.3.linux-amd64.tar.gz-C / app

Mkdir gopkg

Configure GOROOT, GOPATH

Vi / etc/profile

Export GOROOT=/app/go

Export GOPATH=/app/gopkg

Export PATH=$PATH:$GOROOT/bin

Effective environment variable

Source / etc/profile

View go version

Go version

Go version go1.7.3 linux/amd64

2.codis compilation

Create a codis compilation directory

Mkdir-pv / app/gopkg/src/github.com/CodisLabs/

Download the codis package

Cd / app/gopkg/src/github.com/CodisLabs/

Git clone https://github.com/CodisLabs/codis.git-b release3.2

Compile

Cd codis/

Make

The compilation is correct as follows. No error indicates that the compilation is complete.

=

Go build-I-o bin/codis-dashboard./cmd/dashboard

Go build-I-tags "cgo_jemalloc"-o bin/codis-proxy. / cmd/proxy

Go build-I-o bin/codis-admin. / cmd/admin

Go build-I-o bin/codis-fe. / cmd/fe

View the bin directory generation file

Cd / app/gopkg/src/github.com/CodisLabs/codis/bin/

[root@codis-02 bin] # ll

Total 84488

Drwxr-xr-x. 4 root root 4096 Jun 13 18:12 assets

-rwxr-xr-x. 1 root root 15465215 Jun 1318:12 codis-admin

-rwxr-xr-x. 1 root root 17085915 Jun 1318:12 codis-dashboard

-rwxr-xr-x. 1 root root 15358965 Jun 1318:12 codis-fe

-rwxr-xr-x. 1 root root 19311099 Jun 1318:12 codis-proxy

-rwxr-xr-x. 1 root root 7982986 Jun 13 18:11 codis-server

-rwxr-xr-x. 1 root root 5580519 Jun 13 18:11 redis-benchmark

-rwxr-xr-x. 1 root root 5712403 Jun 13 18:11 redis-cli

-rw-r--r--. 1 root root 168 Jun 13 18:11 version

3.codis instance

Deploy redis instance with one master and two slaves

Directory required to create a codis instance

Mkdir-pv / app/codis/redis/ {7001, 7002, 7003, 7004, 7005, 7006}

Cp-r / app/gopkg/src/github.com/CodisLabs/codis/bin/app/codis/

The redis configuration is as follows

Vi / app/codis/redis/7001/redis.conf

Bind 0.0.0.0

Protected-mode no

Port 7001

Tcp-backlog 511

Timeout 60

Tcp-keepalive 300

Daemonize yes

Supervised no

Pidfile "/ tmp/redis_7001.pid"

Loglevel notice

Logfile "/ app/codis/redis/7001/redis_7001.log"

Databases 16

Save 900 1

Save 300 10

Save 60 10000

Stop-writes-on-bgsave-error yes

Rdbcompression yes

Rdbchecksum yes

Dbfilename "dump_7001.rdb"

Dir "/ app/codis/redis/7001"

Slave-serve-stale-data yes

Slave-read-only yes

Repl-diskless-sync no

Repl-diskless-sync-delay 5

Repl-disable-tcp-nodelay no

Slave-priority 100

Appendonly yes

Appendfilename "appendonly.aof"

Appendfsync everysec

No-appendfsync-on-rewrite no

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64mb

Aof-load-truncated yes

Lua-time-limit 5000

Slowlog-log-slower-than 10000

Slowlog-max-len 128

Latency-monitor-threshold 0

Notify-keyspace-events ""

Hash-max-ziplist-entries 512

Hash-max-ziplist-value 64

List-max-ziplist-size-2

List-compress-depth 0

Set-max-intset-entries 512

Zset-max-ziplist-entries 128

Zset-max-ziplist-value 64

Hll-sparse-max-bytes 3000

Activerehashing yes

Client-output-buffer-limit normal 0 0 0

Client-output-buffer-limit slave 256mb 64mb60

Client-output-buffer-limit pubsub 32mb 8mb60

Hz 10

Aof-rewrite-incremental-fsync yes

Configure 7002, 7003, 7004, 7005, 7006

Cp / app/codis/redis/7001/redis.conf / app/codis/redis/7002/

Cp / app/codis/redis/7001/redis.conf / app/codis/redis/7003/

Cp / app/codis/redis/7001/redis.conf / app/codis/redis/7004/

Cp / app/codis/redis/7001/redis.conf / app/codis/redis/7005/

Cp / app/codis/redis/7001/redis.conf / app/codis/redis/7006/

Sed-I's app/codis/redis/7002/redis.conf 7001 / app/codis/redis/7002/redis.conf 7001 / 7002

Sed-I's Universe 7001Universe 7003 Universe G' / app/codis/redis/7003/redis.conf

Sed-I's Universe 7001Universe 7004Universe G' / app/codis/redis/7004/redis.conf

Sed-I's Universe 7001Universe 7005Universe g'/ app/codis/redis/7005/redis.conf

Sed-I's app/codis/redis/7006/redis.conf 7001 / app/codis/redis/7006/redis.conf 7001 / 7006

Start the codis-server service

/ app/codis/bin/codis-server / app/codis/redis/7001/redis.conf

/ app/codis/bin/codis-server / app/codis/redis/7002/redis.conf

/ app/codis/bin/codis-server / app/codis/redis/7003/redis.conf

/ app/codis/bin/codis-server / app/codis/redis/7004/redis.conf

/ app/codis/bin/codis-server / app/codis/redis/7005/redis.conf

/ app/codis/bin/codis-server / app/codis/redis/7006/redis.conf

3. Deploy codis-proxy

1. Generate codis-proxy configuration file

Cd / app/codis/bin/

/ app/codis/bin/codis-proxy-- default-config | tee proxy.toml > > proxy.toml

Vi proxy.tom

# # #

# #

# Codis-Proxy #

# #

# # #

# Set Codis Product Name/Auth.

Product_name = "codis-demo"

Product_auth = ""

# Set auth for client session

# 1. Product_auth is used for auth validation among codis-dashboard

# codis-proxy and codis-server.

# 2. Session_auth is different from product_auth, it requires clients

# to issue AUTH before processing any other commands.

Session_auth = ""

# Set bind address for admin (rpc), tcponly.

Admin_addr = "0.0.0.0 11080"

# Set bind address for proxy, proto_typecan be "tcp", "tcp4", "tcp6", "unix" or "unixpacket".

Proto_type = "tcp4"

Proxy_addr = "0.0.0.0 19000"

# Set jodis address & session timeout

Jodis_name is short for jodis_coordinator_name, only accept "zookeeper" & "etcd".

# 2. Jodis_addr is short for jodis_coordinator_addr

# 3. Proxy will be registered as node:

# if jodis_compatible = true (not suggested):

# / zk/codis/db_ {PRODUCT_NAME} / proxy- {HASHID} (compatible with Codis2.0)

# or else

# / jodis/ {PRODUCT_NAME} / proxy- {HASHID}

Jodis_name = "zookeeper"

Jodis_addr = "172.16.40.131VOR 2881172.16.40.131VOF 2882172.16.40.131VOL2883"

Jodis_timeout = "20s"

Jodis_compatible = false

# Set datacenter of proxy.

Proxy_datacenter = ""

# Set max number of alive sessions.

Proxy_max_clients = 1000

# Set max offheap memory size. (0 todisable)

Proxy_max_offheap_size = "1024mb"

# Set heap placeholder to reduce GCfrequency.

Proxy_heap_placeholder = "256mb"

# Proxy will ping backend redis (and clear'MASTERDOWN' state) in a predefined interval. (0 to disable)

Backend_ping_period = "5s"

# Set backend recv buffer size & timeout.

Backend_recv_bufsize = "128kb"

Backend_recv_timeout = "30s"

# Set backend send buffer & timeout.

Backend_send_bufsize = "128kb"

Backend_send_timeout = "30s"

# Set backend pipeline buffer size.

Backend_max_pipeline = 20480

# Set backend never read replica groups,default is false

Backend_primary_only = false

# Set backend parallel connections perserver

Backend_primary_parallel = 1

Backend_replica_parallel = 1

# Set backend tcp keepalive period. (0 todisable)

Backend_keepalive_period = "75s"

# Set number of databases of backend.

Backend_number_databases = 16

# If there is no request from client for along time, the connection will be closed. (0 to disable)

# Set session recv buffer size & timeout.

Session_recv_bufsize = "128kb"

Session_recv_timeout = "30m"

# Set session send buffer size & timeout.

Session_send_bufsize = "64kb"

Session_send_timeout = "30s"

# Make sure this is higher than the maxnumber of requests for each pipeline request, or your client may be blocked.

# Set session pipeline buffer size.

Session_max_pipeline = 10000

# Set session tcp keepalive period. (0 todisable)

Session_keepalive_period = "75s"

# Set session to be sensitive to failures.Default is false, instead of closing socket, proxy will send an error responseto client.

Session_break_on_failure = false

# Set metrics server (such as http://localhost:28000), proxy will report json formatted metrics to specifiedserver in a predefined period.

Metrics_report_server = ""

Metrics_report_period = "1s"

# Set influxdb server (such as http://localhost:8086), proxy will report metrics to influxdb.

Metrics_report_influxdb_server = ""

Metrics_report_influxdb_period = "1s"

Metrics_report_influxdb_username = ""

Metrics_report_influxdb_password = ""

Metrics_report_influxdb_database = ""

# Set statsd server (such aslocalhost:8125), proxy will report metrics to statsd.

Metrics_report_statsd_server = ""

Metrics_report_statsd_period = "1s"

Metrics_report_statsd_prefix = ""

Parameter description:

Product_name cluster name. Refer to the dashboard parameter description.

Product_auth cluster password. Default is empty.

Admin_addr RESTfulAPI port

Proto_type Redis port type, accepting tcp/tcp4/tcp6/unix/unixpacket

Proxy_addr Redis port address or path

Jodis_addr Jodis registered zookeeper address

Jodis_timeout Jodis registration sessiontimeout time (in second)

The path where jodis_compatible Jodis registers zookeeper

Backend_ping_period and codis-server snooping cycle, which is prohibited in second,0.

The maximum read timeout for the connection between session_max_timeout and client is prohibited in second,0.

Session_max_bufsize and client connection read-write buffer size (in byte)

Maximum pipeline size of session_max_pipeline to client connection

Tcpkeepalive cycle of session_keepalive_period and client. Only tcp is valid. 0 means prohibited.

two。 Start codis-proxy

Nohup / app/codis/bin/codis-proxy-- ncpu=4--config=/app/codis/bin/proxy.toml-- log=proxy.log-- log-level=WARN &

IV. Deploy codis-dashboard

1. Generate codis-proxy configuration file

/ app/codis/bin/codis-dashboard--default-config | tee dashboard.toml > > dashboard.toml

Modify codis-proxy configuration

Vidashboard.toml

# # #

# #

# Codis-Dashboard #

# #

# # #

# SetCoordinator, only accept "zookeeper" & "etcd" & "filesystem".

# QuickStart

# coordinator_name= "filesystem"

# coordinator_addr= "/ tmp/codis"

Coordinator_name = "zookeeper"

Coordinator_addr = "172.16.40.131VOR 2181172.16.40.131RU 2182172.16.40.131VOL2183"

# SetCodis Product Name/Auth.

Product_name = "codis-demo"

Product_auth = ""

# Setbind address for admin (rpc), tcp only.

Admin_addr = "0.0.0.0 18080"

# Setarguments for data migration (only accept 'sync' &' semi-async').

Migration_method= "semi-async"

Migration_parallel_slots= 100

Migration_async_maxbulks= 200

Migration_async_maxbytes= "32mb"

Migration_async_numkeys= 500

Migration_timeout= "30s"

# Setconfigs for redis sentinel.

Sentinel_quorum= 2

Sentinel_parallel_syncs= 1

Sentinel_down_after= "30s"

Sentinel_failover_timeout= "5m"

Sentinel_notification_script= ""

Sentinel_client_reconfig_script= ""

Parameter description:

Coordinator_name external storage type, which accepts zookeeper/etcd

Coordinator_addr external storage address

Product_name cluster name, which satisfies the rule\ w [\ w\.\ -] *

Product_auth cluster password. Default is empty.

Admin_addrRESTful API port

Start codis-dashboard

Nohup/app/codis/bin/codis-dashboard-ncpu=4-config=/app/codis/bin/dashboard.toml-log=dashboard.log-log-level=WARN &

5. Deploy codis-fe

5.1 generate codis-fe configuration file

/ app/codis/bin/codis-admin-- dashboard-list-- zookeeper=172.16.40.131:2182 | tee. / codis.json > > codis.json

[root@codis-02bin] # cat codis.json

[

{

"name": "codis-demo"

"dashboard": "172.16.40.131VR 18080"

}

]

Start the codis-fe service

Nohup/app/codis/bin/codis-fe-ncpu=4-log=fe.log-log-level=WARN-dashboard-list=/app/codis/bin/codis.json-listen=0.0.0.0:18090 &

View relevant service ports

[root@codis-02bin] # ss-tunlp | grep codis

Tcp LISTEN 0128 *: 19000 *: * users: ("codis-proxy", 10106, 7)

Tcp LISTEN 0128 *: 7001 *: * users: ("codis-server", 10063 pr. 4)

Tcp LISTEN 0128 *: 7002 *: * users: ("codis-server", 10068pr. 4)

Tcp LISTEN 0128 *: 7003 *: * users: ("codis-server", 10073pr. 4)

Tcp LISTEN 0128 *: 7004 *: * users: (("codis-server", 10078)

Tcp LISTEN 0 128 *: 7005 *: * users: ("codis-server", 10083 pr. 4)

Tcp LISTEN 0128 *: 7006 *: * users: ("codis-server", 10088pr. 4)

Tcp LISTEN 0 128: 18080: * users: ("codis-dashboard", 10150

Tcp LISTEN 0 128: 11080: * users: ("codis-proxy", 10106, 8)

Tcp LISTEN 0 128: 18090: * users: ("codis-fe", 10221

VI. Redis sentinel deployment

Create a redis-sentinel directory

Mkdir-pv / app/codis/sentinel/ {27001pr 27002}

Configure the sentinel file

Vi / app/codis/sentinel/27001/sentinel-27001.conf

Bind 0.0.0.0

Protected-mode no

Port 27001

Dir / app/codis/sentinel/27001/

Vi / app/codis/sentinel/27002/sentinel-27002.conf

Bind 0.0.0.0

Protected-mode no

Port 27002

Dir / app/codis/sentinel/27002/

Start the sentinel service

Cp/app/gopkg/src/github.com/CodisLabs/codis/extern/redis-3.2.8/src/redis-sentinel/app/codis/bin/

/ app/codis/bin/redis-sentinel / app/codis/sentinel/27001/sentinel-27001.conf &

/ app/codis/bin/redis-sentinel / app/codis/sentinel/27002/sentinel-27002.conf &

7. Use codis-fe to manage codis cluster environment

1.codis-fe management interface

Http://172.16.40.131:18090

Add codis-proxy:

Use commands to add

[root@codis-02 bin] #. / codis-admin-- dashboard=127.0.0.1:18080-- create-proxy-x 127.0.0.1 purl 11080

Interface addition can also be used.

Add codis-server

NEW GROUP first

Add one master and two slaves

Add two groups

The above can also be added with commands.

Add redis-sentinel

Command increment

[root@codis-02bin] # / codis-admin-- dashboard=172.16.40.131:18080-- sentinel-add-- addr=172.16.40.131:27002

Assign slot

Slot: a data slot, a unit for slicing data between codis-server; it is recommended that data slots be evenly divided into multiple group

Add redis-sentinel to the codis-fe management interface and synchronize. After synchronization, the redis-sentinel configuration file will be updated.

Root@codis-02bin] # cat / app/codis/sentinel/27001/sentinel-27001.conf

Bind 0.0.0.0

Protected-mode no

Port 27001

Dir "/ app/codis/sentinel/27001"

# Generated by CONFIG REWRITE

Sentinelmyid b445aae57d1283b4d89d9ed23fd96982dbc344c3

Sentinelmonitor codis-demo-2 172.16.40.131 7004 2

Sentinelfailover-timeout codis-demo-2 300000

Sentinelconfig-epoch codis-demo-2 0

Sentinelleader-epoch codis-demo-2 0

Sentinelknown-slave codis-demo-2 172.16.40.131 7005

Sentinelknown-slave codis-demo-2 172.16.40.131 7006

Sentinelknown-sentinel codis-demo-2 172.16.40.131 27002b236624dce5df7317e5c3e5f30401899c4961094

Sentinelmonitor codis-demo-1 172.16.40.131 7001 2

Sentinelfailover-timeout codis-demo-1 300000

Sentinelconfig-epoch codis-demo-1 0

Sentinelleader-epoch codis-demo-1 0

Sentinelknown-slave codis-demo-1 172.16.40.131 7002

Sentinelknown-slave codis-demo-1 172.16.40.131 7003

Sentinelknown-sentinel codis-demo-1 172.16.40.131 27002 b236624dce5df7317e5c3e5f30401899c4961094

Sentinelcurrent-epoch 0