How to install and configure Twemproxy under Linux

This article is about how to install Twemproxy on Linux. Xiaobian thinks it is quite practical, so share it with everyone for reference. Let's follow Xiaobian and have a look.

Twemproxy is a proxy sharding mechanism open sourced by Twitter. Twemproxy, as a proxy, can accept access from multiple programs, forward it to each Redis server in the background according to routing rules, and then return to the original route. This solution solves the problem of single Redis instance carrying capacity very well.

First you need to install autoconf, automake, libtool tools, such as ubuntu can be installed using the following command

yum install autoconf automake yum install libtool Download Twemproxy and install cd /usr/servers wget https://github.com/twitter/twemproxy/archive/v0.4.0.tar.gz tar -xvf v0.4.0.tar.gz cd twemproxy-0.4.0/ autoreconf -fvi ./ configure && make

Note here that the above installation method may cause Twemproxy to crash on some servers when there are a large number of msets, such as CFLAGS="-O1"./ configure && make or CFLAGS="-O3 -fno-strict-aliasing" ./ configure && make installation.

configured

vim /usr/servers/twemproxy-0.4.0/conf/nutcracker.ymlserver1: listen: 127.0.0.1:1111 hash: fnv1a_64 distribution: ketama redis: true servers: - 127.0.0.1:6379:1

Start Twemproxy

/usr/servers/twemproxy-0.4.0/src/nutcracker -d -c /usr/servers/twemproxy-0.4.0/conf/nutcracker.yml

-d specifies background startup-c specifies configuration file; here we specify proxy port 1111

Check if the startup is successful

ps -aux | grep nutcracker

Go to Redis Client

/usr/servers/redis-2.8.19/src/redis-cli -p 1111

execute the following command

127.0.0.1: 1111> set i 1 OK 127.0.0.1: 1111> get i "1" Settings

basic configuration

It uses YML syntax such as:

server1: listen: 127.0.0.1:1111 hash: fnv1a_64 distribution: ketama timeout:1000 redis: true servers: - 127.0.0.1:6660:1 - 127.0.0.1:6661:1 server1: Name of the current fragment configuration, a configuration file can have multiple fragment configurations;listen: IP and port to listen to;hash: hash algorithm;distribution: fragmentation algorithm, such as consistency Hash/modulo;timeout: timeout for connecting to backend Redis or receiving responses;redis: whether it is a redis proxy, if false, it is a memcached proxy;servers: list of proxy servers, which will be fragmented using the fragmentation algorithm configured by distribution; Fragmentation algorithm #hash algorithm: one_at_a_time md5 crc16 crc32 (crc32 implementation compatible with libmemcached) crc32a (correct crc32 implementation as per the spec) fnv1_64 fnv1a_64 fnv1_32 fnv1a_32 hsieh murmur jenkins #fragmentation algorithm: ketama(consistent Hash algorithm) modula random(random algorithm) server list servers: - ip:port:weight alias such as servers: - 127.0.0.1:6660:1 - 127.0.0.1:6661:1 or servers: - 127.0.0.1:6660:1 server1 - 127.0.0.1:6661:1 server2HashTag

For example, a commodity includes: basic commodity information (p?), Product description (d?), Color Size (c?) If we do not use HashTag when storing, this data will not be stored in one shard, but scattered into multiple shards. In this way, we will need to obtain data from multiple shards for merging, and mget cannot be performed. If we have HashTag, we can use the data in the middle of "::" to do shard logic, so that the same id will be divided into one shard.

nutcracker.yml is configured as follows:

server1: listen: 127.0.0.1:1111 hash: fnv1a_64 distribution: ketama redis: true hash_tag: "::" servers: - 127.0.0.1:6660:1 server1 - 127.0.0.1:6661:1 server2 Connect Twemproxy/usr/servers/redis-2.8.19/src/redis-cli -p 1111

I can connect to port 6660 on my server

/usr/servers/redis-2.8.19/src/redis-cli -p 6660 127.0.0.1: 6660> get p:12: "1" 127.0.0.1: 6660> get d:12: "1" 127.0.0.1: 6660> get c:12: "1" Thank you for reading! About "How to install and configure Twemproxy under Linux" This article is shared here, I hope the above content can be of some help to everyone, so that everyone can learn more knowledge, if you think the article is good, you can share it to let more people see it!