Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "Netty distributed ByteBuf using subPage-level memory allocation method". The editor shows you the operation process through an actual case. The operation method is simple, fast and practical. I hope this article "Netty distributed ByteBuf using subPage-level memory allocation method" can help you solve the problem.

SubPage level memory allocation

For simplicity, we only take 16 bytes as an example to explain its allocation logic.

Before analyzing its logic, let's first look at an attribute of PoolArean:

Private final PoolSubpage [] tinySubpagePools

This attribute is an array of PoolSubpage, which is similar to a cache of subpage. After we create a subpage, we will associate the created subpage with each of the attributes. Next time, we can find the associated subpage directly through the elements of the attribute when assigning it.

We initialize it in the constructor. Look at the initialization code tinySubpagePools = newSubpagePoolArray (numTinySubpagePools) in the constructor.

Here, numTinySubpagePools is 32.

Follow to the newSubpagePoolArray (numTinySubpagePools) method:

Private PoolSubpage [] newSubpagePoolArray (int size) {return new PoolSubpage [size];}

Here you directly create a PoolSubpage array with a length of 32

After it is created in the constructor, it is assigned a value of for (int I = 0; I) through a loop

< tinySubpagePools.length; i ++) { tinySubpagePools[i] = newSubpagePoolHead(pageSize);} 我们跟到newSubpagePoolHead中: private PoolSubpage newSubpagePoolHead(int pageSize) { PoolSubpage head = new PoolSubpage(pageSize); head.prev = head; head.next = head; return head;} 这里创建了一个PoolSubpage对象head head.prev = head;head.next = head; 这种写法我们知道Subpage其实也是个双向链表, 这里的将head的上一个节点和下一个节点都设置为自身, 有关PoolSubpage的关联关系, 我们稍后会看到 这样通过循环创建PoolSubpage, 总共会创建出32个subpage, 其中每个subpage实际代表一块内存大小:

5-8-1

Here is the structure of the cached array tinySubPageDirectCaches in the previous section.

Knowing the tinySubpagePools property, let's look at the allocate method of PoolArean, that is, the entry method of the buffer:

Private void allocate (PoolThreadCache cache, PooledByteBuf buf, final int reqCapacity) {/ / normalize final int normCapacity = normalizeCapacity (reqCapacity); if (isTinyOrSmall (normCapacity)) {int tableIdx; PoolSubpage [] table; / / determine whether it is tinty boolean tiny = isTiny (normCapacity); if (tiny) {/ /

< 512 //缓存分配 if (cache.allocateTiny(this, buf, reqCapacity, normCapacity)) { return; } //通过tinyIdx拿到tableIdx tableIdx = tinyIdx(normCapacity); //subpage的数组 table = tinySubpagePools; } else { if (cache.allocateSmall(this, buf, reqCapacity, normCapacity)) { return; } tableIdx = smallIdx(normCapacity); table = smallSubpagePools; } //拿到对应的节点 final PoolSubpage head = table[tableIdx]; synchronized (head) { final PoolSubpage s = head.next; //默认情况下, head的next也是自身 if (s != head) { assert s.doNotDestroy && s.elemSize == normCapacity; long handle = s.allocate(); assert handle >

= 0; s.chunk.initBufWithSubpage (buf, handle, reqCapacity); if (tiny) {allocationsTiny.increment ();} else {allocationsSmall.increment ();} return;}} allocateNormal (buf, reqCapacity, normCapacity); return } if (normCapacity > > 4;}

Here we divide normCapacity by 16, which is actually 1.

Let's go back to PoolArena's allocate method and continue to look at:

Table = tinySubpagePools

Here you assign tinySubpagePools to the local variable table and move on

Final PoolSubpage head = table [tableIdx]

In this step, we get a PoolSubpage through the subscript, because we take 16 bytes as an example, so we get the PoolSubpage with subscript 1, which corresponds to 16B of memory.

Let's take a look at the step final PoolSubpage s = head.next. With the tinySubpagePools attribute we just learned, head.next is itself by default, so if (s! = head) returns false. Let's move on:

Next, you will go to the allocateNormal (buf, reqCapacity, normCapacity) method:

Private synchronized void allocateNormal (PooledByteBuf buf, int reqCapacity, int normCapacity) {/ / first allocate memory on the original chunk (1) if (q050.allocate (buf, reqCapacity, normCapacity) | | q025.allocate (buf, reqCapacity, normCapacity) | | q000.allocate (buf, reqCapacity, normCapacity) | | qInit.allocate (buf, reqCapacity, normCapacity) | | q075.allocate (buf, reqCapacity, normCapacity)) {+ + allocationsNormal; return } / / create chunk for memory allocation (2) PoolChunk c = newChunk (pageSize, maxOrder, pageShifts, chunkSize); long handle = c.allocate (normCapacity); + + allocationsNormal; assert handle > 0; / / initialize byteBuf (3) c.initBuf (buf, handle, reqCapacity); qInit.add (c);}

The logic here has been analyzed in the previous section. First, it is allocated in the original chunk. If the allocation is not successful, a chunk will be created for allocation.

Let's look at this step.

Long handle = c.allocate (normCapacity)

Follow long allocate (int normCapacity) {if ((normCapacity & subpageOverflowMask)! = 0) {return allocateRun (normCapacity);} else {return allocateSubpage (normCapacity);}} in the allocate (normCapacity) method

When we analyzed the page level allocation in the previous section, we analyzed the allocateRun (normCapacity) method

Since we are taking 16 bytes as an example, this time we analyze the allocateSubpage (normCapacity) method, which allocates memory at the subpage level.

Private long allocateSubpage (int normCapacity) {PoolSubpage head = arena.findSubpagePoolHead (normCapacity); synchronized (head) {int d = maxOrder; / / indicates that the distribution node int id = allocateNode (d) at layer 11; if (id

< 0) { return id; } //获取初始化的subpage final PoolSubpage[] subpages = this.subpages; final int pageSize = this.pageSize; freeBytes -= pageSize; //表示第几个subpageIdx int subpageIdx = subpageIdx(id); PoolSubpage subpage = subpages[subpageIdx]; if (subpage == null) { //如果subpage为空 subpage = new PoolSubpage(head, this, id, runOffset(id), pageSize, normCapacity); //则将当前的下标赋值为subpage subpages[subpageIdx] = subpage; } else { subpage.init(head, normCapacity); } //取出一个子page return subpage.allocate(); }} 首先, 通过 PoolSubpage head = arena.findSubpagePoolHead(normCapacity) 这种方式找到head节点, 实际上这里head, 就是我们刚才分析的tinySubpagePools属性的第一个节点, 也就是对应16B的那个节点 int d = maxOrder 是将11赋值给d, 也就是在内存树的第11层取节点, 这部分上一小节剖析过了, 可以回顾图5-8-5部分 int id = allocateNode(d) 这里获取的是上一小节我们分析过的, 字节数组memoryMap的下标, 这里指向一个page, 如果第一次分配, 指向的是0-8k的那个page, 上一小节对此进行详细的剖析这里不再赘述 final PoolSubpage[] subpages = this.subpages 这一步, 是拿到PoolChunk中成员变量subpages的值, 也是个PoolSubpage的数组, 在PoolChunk进行初始化的时候, 也会初始化该数组, 长度为2048 也就是说每个chunk都维护着一个subpage的列表, 如果每一个page级别的内存都需要被切分成子page, 则会将这个这个page放入该列表中, 专门用于分配子page, 所以这个列表中的subpage, 其实就是一个用于切分的page 5-8-2 int subpageIdx = subpageIdx(id) 这一步是通过id拿到这个PoolSubpage数组的下标, 如果id对应的page是0-8k的节点, 这里拿到的下标就是0 在 if (subpage == null) 中, 因为默认subpages只是创建一个数组, 并没有往数组中赋值, 所以第一次走到这里会返回true, 跟到if块中: subpage = new PoolSubpage(head, this, id, runOffset(id), pageSize, normCapacity); 这里通过new PoolSubpage创建一个新的subpage之后, 通过 subpages[subpageIdx] = subpage 这种方式将新创建的subpage根据下标赋值到subpages中的元素中 在new PoolSubpage的构造方法中, 传入head, 就是我们刚才提到过的tinySubpagePools属性中的节点, 如果我们分配的16字节的缓冲区, 则这里对应的就是第一个节点 我们跟到PoolSubpage的构造方法中PoolSubpage(PoolSubpage head, PoolChunk chunk, int memoryMapIdx, int runOffset, int pageSize, int elemSize) { this.chunk = chunk; this.memoryMapIdx = memoryMapIdx; this.runOffset = runOffset; this.pageSize = pageSize; bitmap = new long[pageSize >

> > 10]; init (head, elemSize);}

Here we focus on the attribute bitmap, which is an array of type long with an initial size of 8, which is only the size of the initialization. The real size depends on how many blocks the page is divided into.

Here you assign the property, and we follow it in the init method:

Void init (PoolSubpage head, int elemSize) {doNotDestroy = true; this.elemSize = elemSize; if (elemSize! = 0) {maxNumElems = numAvail = pageSize / elemSize; nextAvail = 0; bitmapLength = maxNumElems > 6; if ((maxNumElems & 63)! = 0) {bitmapLength + +;} for (int I = 0; I

< bitmapLength; i ++) { //bitmap标识哪个子page被分配 //0标识未分配, 1表示已分配 bitmap [i] = 0; } } //加到arena里面 addToPool(head);} this.elemSize = elemSize 表示保存当前分配的缓冲区大小, 这里我们以16字节举例, 所以这里是16 maxNumElems = numAvail = pageSize / elemSize 这里初始化了两个属性maxNumElems, numAvail, 值都为pageSize / elemSize, 表示一个page大小除以分配的缓冲区大小, 也就是表示当前page被划分了多少分 numAvail则表示剩余可用的块数, 由于第一次分配都是可用的, 所以 numAvail=maxNumElems bitmapLength表示bitmap的实际大小, 刚才我们分析过, bitmap初始化的大小为8, 但实际上并不一定需要8个元素, 元素个数要根据page切分的子块而定, 这里的大小是所切分的子块数除以64 再往下看, if ((maxNumElems & 63) != 0) 判断maxNumElems也就是当前配置所切分的子块是不是64的倍数, 如果不是, 则bitmapLength加1, 最后通过循环, 将其分配的大小中的元素赋值为0 这里详细介绍一下有关bitmap, 这里是个long类型的数组, long数组中的每一个值, 也就是long类型的数字, 其中的每一个比特位, 都标记着page中每一个子块的内存是否已分配, 如果比特位是1, 表示该子块已分配, 如果比特位是0, 表示该子块未分配, 标记顺序是其二进制数从低位到高位进行排列 这里, 我们应该知道为什么bitmap大小要设置为子块数量除以, 64, 因为long类型的数字是64位, 每一个元素能记录64个子块的数量, 这样就可以通过子page个数除以64的方式决定bitmap中元素的数量 如果子块不能整除64, 则通过元素数量+1方式, 除以64之后剩余的子块通过long中比特位由低到高进行排列记录 这里的逻辑结构如下所示:

5-8-3

We follow private void addToPool (PoolSubpage head) {assert prev = = null & & next = = null; prev = head; next = head.next; next.prev = this; head.next = this;} in addToPool (head)

The head here, as we just talked about, is an element in the array tinySubpagePools in Arena. Through the above logic, the newly created Subpage will be associated with the elements in tinySubpagePools through a bi-directional linked list. We take 16 bytes as an example, and the association relationship is shown in the figure:

5-8-4

In this way, the next time you need to allocate 16 bytes of memory, you can find the subpage associated with its element through tinySubpagePools to allocate it.

Let's go back to the allocateSubpage method of PoolChunk:

Private long allocateSubpage (int normCapacity) {PoolSubpage head = arena.findSubpagePoolHead (normCapacity); synchronized (head) {int d = maxOrder; / / indicates that the distribution node int id = allocateNode (d) at layer 11; if (id

< 0) { return id; } //获取初始化的subpage final PoolSubpage[] subpages = this.subpages; final int pageSize = this.pageSize; freeBytes -= pageSize; //表示第几个subpageIdx int subpageIdx = subpageIdx(id); PoolSubpage subpage = subpages[subpageIdx]; if (subpage == null) { //如果subpage为空 subpage = new PoolSubpage(head, this, id, runOffset(id), pageSize, normCapacity); //则将当前的下标赋值为subpage subpages[subpageIdx] = subpage; } else { subpage.init(head, normCapacity); } //取出一个子page return subpage.allocate(); }} 创建完了一个subpage, 我们就可以通过subpage.allocate()方法进行内存分配了 我们跟到allocate()方法中long allocate() { if (elemSize == 0) { return toHandle(0); } if (numAvail == 0 || !doNotDestroy) { return -1; } //取一个bitmap中可用的id(绝对id) final int bitmapIdx = getNextAvail(); //除以64(bitmap的相对下标) int q = bitmapIdx >

> > 6; / / divided by 64, is actually the offset of the current absolute id int r = bitmapIdx & 63; assert (bitmap [Q] > > r & 1) = 0 / / the current bit is marked with 1 bitmap [Q] | = 1L = 0) {/ / after a sub-page is released, the location of the bitmapIdx of the current sub-page is recorded. The next allocation can be directly obtained through bitmapIdx to get a sub-page this.nextAvail =-1; return nextAvail;} return findNextAvail ();}

Here nextAvail represents the next available bitmapIdx, which will be marked upon release, and the subblock of the tag released corresponds to the subscript of bitmapIdx, if > = 1;} return-1;}

The search starts with the first bit of the current element until a bit marked 0 is found and the subscript of the current bit is returned, as shown in the following figure:

5-8-5

Let's go back to the allocate () method long allocate () {if (elemSize = = 0) {return toHandle (0);} if (numAvail = = 0 |! doNotDestroy) {return-1;} / / take an id (absolute id) final int bitmapIdx = getNextAvail () available in bitmap; / / divide by 64 (relative subscript of bitmap) int Q = bitmapIdx > 6 / / dividing by 64 to take the remainder is actually the offset of the current absolute id int r = bitmapIdx & 63; assert (bitmap [Q] > > r & 1) = = 0; / / the current bit tag is 1 bitmap [Q] | = 1L > > 6 to obtain the array subscript of the element to which the bitmapIdx belongs in the bitmap

Int r = bitmapIdx & 63 indicates that the position where the bitmapIdx is obtained is the number of bits starting from the lowest bit of the current element

Bitmap [Q] | = 1L

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.