Shulou(Shulou.com)06/02 Report--

Redis cluster installation uses codis, which is open source by pea pods. Compared with twemproxy, there are many advantages. Refer to http://blog.csdn.net/hunci/article/details/51799468.

Don't talk nonsense, do it.

The following installation document copied the stir-fried meat, and the link is as follows

Https://www.kissni.com/2017/04/06/codis-redis/

For the construction documents of the official website, see https://github.com/CodisLabs/codis/blob/release3.2/doc/tutorial_zh.md readme.

But there are also some problems in the deployment.

1, an error occurred during codis make

Go build-I-o bin/codis-dashboard. / cmd/dashboard

Go build github.com/CodisLabs/codis/vendor/github.com/ugorji/go/codec: / opt/local/go/pkg/tool/linux_amd64/compile: signal: killed

Make: * * [codis-dashboard] error 1

Reason: I use vps, the memory is very small, only 500m, check the log / var/log/message

May 16 09:30:06 vultr kernel: Out of memory: Kill process 10020 (compile) score 460 or sacrifice child

May 16 09:30:06 vultr kernel: Killed process 10020, UID 0, (compile) total-vm:383560kB, anon-rss:371776kB, file-rss:120kB

Because the linux system memory invocation mechanism is triggered when the application service is started, oom kills the compilation process in order to ensure the normal operation of the system.

Solution: two ways, adjust the kernel, not recommended, will hang up (please ignore the big god)

The second way is to add more memory.

2. Error is reported during compilation

Zmalloc.h:50:31: error: jemalloc/jemalloc.h: No such file or directory

Zmalloc.h:55:2: error: # error "Newer version of jemalloc required"

Make [1]: * * [adlist.o] Error 1

Undefined reference to `dladdr'

Two mistakes

Resolve:

In fact, both mistakes are due to the failure to establish the correct location of the link library.

Just add two parameters when compiling

Make MALLOC=libc LDFLAGS+=-ldl

3. When you add codis-proxy to the dashboard panel, the server configuration file of dashboard and the configuration file in codis-proxy

Product_name = "codis-demo" product_auth = ""

This authentication must be done all the time, because both of them go to zookeeper, or dashboard gives codis-proxy through zk.

To convey a message.

Architecture diagram

Environment configuration installation gityum install-y git autoconf installation golang (using version 1.5.x) wget https://storage.googleapis.com/golang/go1.5.4.linux-amd64.tar.gztar zxvf go1.5.4.linux-amd64.tar.gzmkdir / opt/localmv go / opt/local/ configuration environment variable vi / etc/profile# is added as follows: # golang ENVexport GOROOT=/opt/local/goexport PATH=$PATH:$GOROOT/binexport GOPATH=/opt/local/codis# effective immediately source / etc/profile# check version go versiongo Version go1.5.4 linux/amd64echo $GOPATH/opt/local/codis install Codis download Codis source code mkdir-p $GOPATH/src/github.com/CodisLabscd $GOPATH/src/github.com/CodisLabsgit clone https://github.com/CodisLabs/codis.git-b release3.1 compile Codis source code cd $GOPATH/src/github.com/CodisLabs/codismakemake-j-C extern/redis-3.2.4/# enter the following information: go build-I-o bin/codis-dashboard. / cmd/dashboardgo Build-I-o bin/codis-proxy. / cmd/proxygo build-I-o bin/codis-admin. / cmd/admingo build-I-o bin/codis-ha. / cmd/hago build-I-o bin/codis-fe. / cmd/fe# View the installed version cat bin/versionversion = 2017-03-08 14:07:13 + 0800 @ b1919d11593dfd1f47a2461837233dfc8fc78002 @ 3.1.5-26-gb1919d1compile = 2017-04-05 18:13:46 + 0800 by go version go1.5.4 linux/amd64# copy file Easy to manage mkdir-p / opt/local/codis/ {bin,logs,data} / cp-rf $GOPATH/src/github.com/CodisLabs/codis/bin/* / opt/local/codis/bincp-rf $GOPATH/src/github.com/CodisLabs/codis/config / opt/local/codis/ configuration CodisCodis Dashboard

It is a background service, and the display page is displayed by fe.

Cluster management tools, support codis-proxy, codis-server add, delete, and data migration and other operations. When the state of the cluster changes, codis-dashboard maintains the consistency of the state of all codis-proxy under the cluster.

For the same business cluster, there can only be 0 or 1 codis-dashboard at the same time.

All modifications to the cluster must be done through codis-dashboard.

Cd / opt/local/codis/config/vim dashboard.toml# modify configuration file # Codis-Dashboard # # # Set Coordinator Only accept "zookeeper" & "etcd" & "filesystem" .coordinator _ name = "zookeeper" coordinator_addr = "127.0.0.1 Vol 2181" # Set Codis Product Name/Auth.product_name = "codis-demo" product_auth = "" # Set bind address for admin (rpc) Tcp only.admin_addr = "0.0.0.0 sentinel_failover_timeout 18080" # Set configs for redis sentinel.sentinel_quorum = 2sentinel_parallel_syncs = 1sentinel_down_after = "30s" sentinel_failover_timeout = "5m" sentinel_notification_script = "" sentinel_client_reconfig_script = "" # launch Dashboardnohup / opt/local/codis/bin/codis-dashboard-- ncpu=4-- config=/opt/local/codis/config/dashboard.toml\-- log=/opt/local/codis/logs / dashboard.log-- log-level=WARN & Codis Proxycd / opt/local/codis/config/vim proxy.toml# modify configuration file # Codis -Proxy # Set Codis Product Name/Auth.product_name = "codis-demo" product_auth = "" # Set bind address for admin (rpc) Tcp only.admin_addr = "0.0.0.0VOV 11080" # the comment is obvious. This is the address where proxy is added to dashboard. It is the admin command console. Control the address of proxy # Set bind address for proxy, proto_type can be "tcp", "tcp4", "tcp6", "unix" or "unixpacket" .proto _ type = "tcp4" proxy_addr = "0.0.0.0proto_type can be 19000" # Set jodis address & session timeout Only accept "zookeeper" & "etcd" .jodis _ name = "zookeeper" jodis_addr = "127.0.0.1 etcd 2181" jodis_timeout = "20s" jodis_compatible = false# Set datacenter of proxy.proxy_datacenter = "" # Set max number of alive sessions.proxy_max_clients = 100mm Set max offheap memory size. (0 to disable) proxy_max_offheap_size = "1024mb" # Set heap placeholder to reduce GC frequency.proxy_heap_placeholder = "256mb" # Proxy will ping backend redis (and clear 'MASTERDOWN' state) in a predefined interval. (0 to disable) backend_ping_period = "5s" # Set backend recv buffer size & timeout.backend_recv_bufsize = "128kb" backend_recv_timeout = "30s" # Set backend send buffer & timeout.backend_send_bufsize = "128kb" backend_send_timeout = "30s" # Set backend pipeline buffer size.backend_max_pipeline = 102 Set backend never read replica groups, default is falsebackend_primary_only = false# Set backend parallel connections per serverbackend_primary_parallel = 1backend_replica_parallel = Set backend tcp keepalive period. (0 to disable) backend_keepalive_period = "75s" # If there is no request from client for a long time, the connection will be closed. (0 to disable) # Set session recv buffer size & timeout.session_recv_bufsize = "128kb" session_recv_timeout = "30m" # Set session send buffer size & timeout.session_send_bufsize = "64kb" session_send_timeout = "30s" # Make sure this is higher than the max number of requests for each pipeline request, or your client may be blocked.# Set session pipeline buffer size.session_max_pipeline = 102 million Set session tcp keepalive period. (0 to disable) session_keepalive_period = "75s" # Set session to be sensitive to failures. Default is false, instead of closing socket, proxy will send an error response to client.session_break_on_failure = false# Set metrics server (such as http://localhost:28000), proxy will report json formatted metrics to specified server in a predefined period.metrics_report_server = "" metrics_report_period = "1s" # Set influxdb server (such as http://localhost:8086), Proxy will report metrics to influxdb.metrics_report_influxdb_server = "" metrics_report_influxdb_period = "1s" metrics_report_influxdb_username = "" metrics_report_influxdb_password = "" metrics_report_influxdb_database = "" # launch codis-proxynohup / opt/local/codis/bin/codis-proxy-- ncpu=4-- config=/opt/local/codis/config/proxy.toml\-- log=/opt/local/codis/logs/proxy.log-- log-level=WARN & The log output is as follows: proxy waiting online. # must be added to the cluster It's normal. # you can use the Codis-fe interface to add or use Codis-admin to add Codis-Server# to start server in the same way as starting redis. Nohup / opt/local/codis/bin/codis-server / opt/local/codis/config/redis.conf & # after starting Codis-Server, you need to use Codis-fe or Codis-admin to add to the cluster # configuration file bind 127.0.0.1protected-mode yesport 6379tcp-backlog 2048timeout 0tcp-keepalive 300daemonize nosupervised nopidfile / Var/run/redis_6379.pidloglevel noticelogfile "/ opt/local/codis/logs/redis.log" maxmemory 10gbdatabases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yesdbfilename redis_dump.rdbdir / opt/local/codis/dataslave-serve-stale-data yesslave-read-only yesrepl-diskless-sync norepl-diskless-sync-delay 5repl-disable-tcp-nodelay noslave-priority 100appendonly noappendfilename "appendonly.aof" appendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100autoMaia of Mimin- Size 64mbaof-load-truncated yeslua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128latency-monitor-threshold 0notify-keyspace-events "" hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-size-2list-compress-depth 0set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64hll-sparse-max-bytes 3000activerehashing yesclient-output-buffer-limit normal 00 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yesCodis-FE

It should be noted here that FE is the real display page, so the listening address should be written in the local address, so that you can access the control panel of the cluster in the browser. While codis-dashboard is a background service, you can't figure it out at first.

# launch codis-fenohup / opt/local/codis/bin/codis-fe-- ncpu=4-- log=/opt/local/codis/logs/fe.log-- log-level=WARN-- zookeeper=127.0.0.1:2181-- listen=127.0.0.1:8080 & Codis-admin

In fact, it is the command management tool of the cluster.

# add codis-proxy/opt/local/codis/bin/codis-admin-- dashboard=127.0.0.1:18080-- create-proxy-x 127.0.0.1 create-proxy 1108log input as follows: jodis create node / jodis/codis-demo/proxy-15fba3007f3c6ee0887749681cb82307proxy is working... # repair abnormal exit of Codis-dashboard # dashboard Or use / opt/local/codis/bin/codis-admin-- remove-lock-- product=codis-demo-- zookeeper=127.0.0.1:2181# to repair the abnormal exit of Codis-proxy# proxy when kill-9, or / opt/local/bin/codis-admin-- dashboard=127.0.0.1:18080-- remove-proxy-- addr=127.0.0.1:11080-- force when kill-9

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.