Shulou(Shulou.com)06/02 Report--

Editor to share with you the example analysis of redis multiplexing technology, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Redis is a single-threaded but very good in-memory database, which is mainly used as a caching system. Redis uses network IO multiplexing technology to ensure the high throughput of the system when there are multiple connections.

Principle of LINUX IO Multiplexing

Redis multiplexing, provides select, epoll, evport, kqueue several options, in the compilation time to choose one.

Select is provided by POSIX and is supported by general operating systems; epoll is supported by LINUX kernel; evport is supported by Solaris kernel; kqueue is supported by Mac system

The servers we usually run are on LINUX systems, and I don't know much about Solaris and Mac systems. Here we focus on comparing the differences of select, poll and epoll multiplexing.

Select: the maximum number of connections that can be opened by a single process is defined by the FD_SETSIZE macro, with a size of 1024 or 2048. A sharp increase in the number of FD will cause performance problems. Copy data is required for messaging from the kernel to and to user space.

Performance issues:

(1) every time you call select, you need to copy the fd collection from the user mode to the kernel state. This overhead will be very large in many cases of fd.

(2) at the same time, each call to select requires traversing all the fd passed in the kernel, which is also very expensive in many cases when fd is called.

Poll: basically like select, the difference is that there is no limit on the number of FD, because the underlying implementation is not an array, but a linked list

Epoll: although the number of FD connections is limited, it can be considered unlimited; the implementation in the epoll kernel is based on the callback function on each fd, and only active socket will actively call callback, so when there is less active socket, there is no linear degradation in the performance of using epoll; the kernel and users pass messages through shared memory; LINUX IO multiplexing interfaces

The interface of select in LINUX:

# include

/ * According to earlier standards * /

# include

# include

# include

Int select (int nfds, fd_set * readfds, fd_set * writefds

Fd_set * exceptfds, struct timeval * timeout)

Void FD_CLR (int fd, fd_set * set)

Int FD_ISSET (int fd, fd_set * set)

Void FD_SET (int fd, fd_set * set)

Void FD_ZERO (fd_set * set)

The parameters of the select function:

-the number of FD of nfds:fd_set, which records the FD status of fd_set collection by bitmap

-readfds: fd_set collection to monitor which read operations are not block, and if readable, select

-writefds:fd_set collection to monitor which writes are not block

-exceptfds: fd_set collection to monitor which except operations are not block

-timeout: the minimum time for FD z to be block. If both field of timeout are 0, it will be returned immediately. If this parameter is NULL, it will always block.

Select returns a value greater than 1 if there are one or more read operations, write operations, and except operations that are not block; if there is no FD that is not block, but some FD block times out, return 0; if an error occurs, return-1

The FD_XXX function is a tool function for adding, deleting, emptying, and judging fd_set.

Pseudo code for select:

While (1) {

Int ret = select (streams [])

If (ret > 0) {

For i in streams [] {

If i has data {

Read or write streams [i]

}

}

} else if (ret = = 0) {

Handle timeout FDs

} else {

Handle error

}

}

The API of epoll's LINUX:

# include

/ / predefined EVENT

Enum EPOLL_EVENTS

{

EPOLLIN = 0x001

# define EPOLLIN EPOLLIN

EPOLLPRI = 0x002

# define EPOLLPRI EPOLLPRI

EPOLLOUT = 0x004

# define EPOLLOUT EPOLLOUT

EPOLLRDNORM = 0x040

# define EPOLLRDNORM EPOLLRDNORM

EPOLLRDBAND = 0x080

# define EPOLLRDBAND EPOLLRDBAND

EPOLLWRNORM = 0x100

# define EPOLLWRNORM EPOLLWRNORM

EPOLLWRBAND = 0x200

# define EPOLLWRBAND EPOLLWRBAND

EPOLLMSG = 0x400

# define EPOLLMSG EPOLLMSG

EPOLLERR = 0x008

# define EPOLLERR EPOLLERR

EPOLLHUP = 0x010

# define EPOLLHUP EPOLLHUP

EPOLLRDHUP = 0x2000

# define EPOLLRDHUP EPOLLRDHUP

EPOLLWAKEUP = 1U stop) {

If (eventLoop- > beforesleep! = NULL)

EventLoop- > beforesleep (eventLoop)

AeProcessEvents (eventLoop, AE_ALL_EVENTS | AE_CALL_AFTER_SLEEP)

}

}

* The function returns the number of events processed. , /

Int aeProcessEvents (aeEventLoop * eventLoop, int flags)

{

* some event fires. , /

Numevents = aeApiPoll (eventLoop, tvp)

/ * After sleep callback. , /

If (eventLoop- > aftersleep! = NULL & & flags & AE_CALL_AFTER_SLEEP)

EventLoop- > aftersleep (eventLoop)

For (j = 0; j)

< numevents; j++) { aeFileEvent *fe = &eventLoop->

Events [eventLoop-> fired.fd]

Int mask = eventLoop- > Fired [j] .mask

Int fd = eventLoop- > Fired [j] .fd

Int rfired = 0

/ * note the fe- > mask & mask &... Code: maybe an already processed

* event removed an element that fired and we still didn't

* processed, so we check if the event is still valid. , /

If (fe- > mask & mask & AE_READABLE) {

Rfired = 1

Fe- > rfileProc (eventLoop,fd,fe- > clientData,mask)

}

If (fe- > mask & mask & AE_WRITABLE) {

If (! rfired | | fe- > wfileProc! = fe- > rfileProc)

Fe- > wfileProc (eventLoop,fd,fe- > clientData,mask)

}

Processed++

}

}

/ * Check time events * /

If (flags & AE_TIME_EVENTS)

Processed + = processTimeEvents (eventLoop)

Return processed; / * return the number of processed file/time events * /

}

Static void aeApiDelEvent (aeEventLoop * eventLoop, int fd, int delmask) {

AeApiState * state = eventLoop- > apidata

Struct epoll_event ee = {0}; / * avoid valgrind warning * /

Int mask = eventLoop- > events [FD] .mask & (~ delmask)

Ee.events = 0

If (mask & AE_READABLE) ee.events | = EPOLLIN

If (mask & AE_WRITABLE) ee.events | = EPOLLOUT

Ee.data.fd = fd

If (mask! = AE_NONE) {

Epoll_ctl (state- > epfd,EPOLL_CTL_MOD,fd,&ee)

} else {

/ * Note, Kernel

< 2.6.9 requires a non null event pointer even for * EPOLL_CTL_DEL. */ epoll_ctl(state->

Epfd,EPOLL_CTL_DEL,fd,&ee)

}

}

The above is all the content of the example analysis of redis multiplexing technology, thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.