Shulou(Shulou.com)05/31 Report--

This article mainly talks about "what are the skills of Redis in memory allocation and usage statistics". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what are the skills of Redis in memory allocation and usage statistics?"

Specifically, it is:

If the TC_ MALLOC library of Google exists in the system, the tc_malloc family of functions is used instead of the original malloc family of functions.

If the current system is a Mac system, the memory allocation function in is used.

In other cases, in front of each segment of allocated space, an additional fixed-length field is assigned to record the size of the allocated space.

The source code is in config.h and zmalloc.c, respectively:

/ * config.h * /

# if defined (USE_TCMALLOC)

# include

# if TC_VERSION_MAJOR > = 1 & & TC_VERSION_MINOR > = 6

# define HAVE_MALLOC_SIZE 1

# define redis_malloc_size (p) tc_malloc_size (p)

# endif

# elif defined (_ _ APPLE__)

# include

# define HAVE_MALLOC_SIZE 1

# define redis_malloc_size (p) malloc_size (p)

# endif

/ * zmalloc.c * /

# ifdef HAVE_MALLOC_SIZE

# define PREFIX_SIZE (0)

# else

# if defined (_ _ sun)

# define PREFIX_SIZE (sizeof (long long))

# else

# define PREFIX_SIZE (sizeof (size_t))

# endif

# endif

Because the malloc function family on tc_malloc and Mac platforms provides functions for calculating the size of allocated space (tc_malloc_size and malloc_size, respectively), there is no need to allocate a separate space record size. For linux and sun platforms, record the size of the allocated space. For linux, use sizeof (size_t) fixed-length field records, and for sun os, use sizeof (long long) fixed-length field records. That is, the PREFIX_SIZE macro in the above source code.

So what's the use of this record? The answer is to count how much memory the current process takes up. In zmalloc.c, there is a static variable:

Static size_t used_memory = 0

It records the total amount of memory currently occupied by the process. This variable is updated whenever memory is allocated or freed. Because when allocating memory, you can clearly know how much memory to allocate. But when you free memory (for platforms that don't provide the malloc_size function), you can't know how much space will be freed just by pointing to the pointer to free memory. At this point, the PREFIX_SIZE fixed-length field mentioned above works, and the size of the space can be obtained from the contents recorded in it. The zmalloc function is as follows (remove the irrelevant code):

Void * zmalloc (size_t size) {

Void * ptr = malloc (size+PREFIX_SIZE)

If (! ptr) zmalloc_oom (size)

* ((size_t*) ptr) = size

Update_zmalloc_stat_alloc (size+PREFIX_SIZE,size)

Return (char*) ptr+PREFIX_SIZE

# endif

}

What are the skills of Redis in memory allocation and usage statistics

See that when space is allocated, the space size is size+PREFIX_SIZE. For mac systems or in the case of using tc_malloc, PREFIX_SIZE is 0. The size of the allocated space is then recorded in the size_t before the space where the ptr pointer is pointed. The last thing that is returned is the pointer over the record area. The zfree function is similar (get rid of irrelevant code):

Void zfree (void * ptr) {

Void * realptr

Size_t oldsize

If (ptr = = NULL) return

Realptr = (char*) ptr-PREFIX_SIZE

Oldsize = * (size_t*) realptr)

Update_zmalloc_stat_free (oldsize+PREFIX_SIZE)

Free (realptr)

# endif

}

Move the pointer forward PREFIX_SIZE first, and then take out the length of the space saved when the space is allocated. Finally, free the whole space.

The macros update_zmalloc_stat_alloc (_ n) and update_zmalloc_stat_free (_ n) are responsible for updating the used_memory variable when memory is allocated or freed. The definition of macro is mainly based on efficiency considerations. Restore it to a function, which looks like this:

Void update_zmalloc_stat_alloc (_ _ n)

{

Do {

Size_t _ n = (_ n)

Size_t _ stat_slot = (_ _ size)

< ZMALLOC_MAX_ALLOC_STAT) ? __size : ZMALLOC_MAX_ALLOC_STAT; 　　if (_n&(sizeof(long)-1)) _n += sizeof(long)-(_n&(sizeof(long)-1)); 　　if (zmalloc_thread_safe) { 　　pthread_mutex_lock(&used_memory_mutex); 　　used_memory += _n; 　　zmalloc_allocations[_stat_slot]++; 　　pthread_mutex_unlock(&used_memory_mutex); 　　} else { 　　used_memory += _n; 　　zmalloc_allocations[_stat_slot]++; 　　} 　　} while(0) 　　} 　　void update_zmalloc_stat_free(__n) 　　{ 　　do { 　　size_t _n = (__n); 　　if (_n&(sizeof(long)-1)) _n += sizeof(long)-(_n&(sizeof(long)-1)); 　　if (zmalloc_thread_safe) { 　　pthread_mutex_lock(&used_memory_mutex); 　　used_memory -= _n; 　　pthread_mutex_unlock(&used_memory_mutex); 　　} else { 　　used_memory -= _n; 　　} 　　} while(0) 　　} 　　代码中除了更新used_memory变量外，还有几个要关注的地方： 　　先对_n的低位向上取整，最后_n变为sizeof(long)的倍数，比如对于32位系统，sizeof(long) == 100(二进制)，_n向上取整之后，低两位都变为0。 　　如果进程中有多个线程存在，则在更新变量的时候要加锁。 　　在zmalloc函数中还有一个统计量要更新：zmalloc_allocations[]。 　　在 zmalloc.c 中，zmalloc_allocations是这样定义的： 　　size_t zmalloc_allocations[ZMALLOC_MAX_ALLOC_STAT+1]; 　　其作用是统计程序分配内存时，对不同大小空间的请求次数。统计的空间范围从1字节到256字节，大于256字节的算为256。统计结果通过调用 zmalloc_allocations_for_size 函数返回： 　　size_t zmalloc_allocations_for_size(size_t size) { 　　if (size >

ZMALLOC_MAX_ALLOC_STAT) return 0

Return zmalloc_ allocations[size]

}

Another statistic of memory usage is returned by calling the zmalloc_used_memory function:

Size_t zmalloc_used_memory (void) {

Size_t um

If (zmalloc_thread_safe) pthread_mutex_lock (& used_memory_mutex)

Um = used_memory

If (zmalloc_thread_safe) pthread_mutex_unlock (& used_memory_mutex)

Return um

}

In addition, the zmalloc_get_rss function is implemented for different systems in zmalloc.c. In the linux system, the memory usage of the system statistics is obtained by reading / proc/$pid/stat files.

At this point, I believe you have a deeper understanding of "what are the skills of Redis in memory allocation and usage statistics". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.