Shulou(Shulou.com)11/24 Report--

This article comes from the official account of Wechat: programming Technology Universe (ID:xuanyuancoding), author: Xuanyuan Wind O

I am a network monitoring software, and my mission is to monitor all traffic coming in and out of the network.

All the time, my work has been very good, but as the network I monitor is getting larger and larger, and the traffic in the network becomes more and more, I begin to be a little too busy, and the phenomenon of packet loss has gradually occurred. And recently, this phenomenon has become more and more serious.

One night, my brother, a programmer, called me off my hard drive.

"what time is it? why aren't you off work?" Oh, I asked my little brother.

"well, the product manager said that I had to support the analysis of 10 gigabytes of network traffic next month. I was under a lot of pressure and had no choice but to work overtime." With that, he arranged his increasingly thinning hair.

"10 trillion? 10Gbps? are you kidding me? this is the rhythm that kills me."

"that's right, but I'm so worried. Please tell me that you have been working for so long, and you can tell me if you are unhappy or feel there is room for improvement." the little brother looked at me sincerely.

I thought for a moment and said, "if you don't like it, there is!" it's just that I spend too much time copying packets from kernel space to user-state space. It used to be a small amount of data, but now the network traffic is so heavy, it really kills me. "

The little brother sighed, "Oh, this can't be changed. The data packet is obtained through the API of the operating system, and the operating system is read from the network card. We are programs working in user space, and we have to copy them again. There is no way. Can you think of something else?"

I also sighed, "well, there's another slot. Can you give it to me directly after receiving the data packet? don't give it to the system's protocol stack and netfilter framework to deal with it. Anyway, I have to re-analyze it after I bring it. I go through it from them every time. They are inefficient, which doesn't drag me down."

The little brother frowned, blinked and said, "Big Brother, we can't change this. My level is limited. I don't have the ability to transform you to bypass the operating system so that you can deal with the network card directly." why don't you say another one? Hey, hey. "

"well, I won't embarrass you. There's a simple question that you need to change."

"what's the problem? tell me about it."

"that is, I now spend a lot of time on thread switching, and when I get the scheduling execution again, I often find that I have changed a CPU core, resulting in the invalidation of the previous cache, so I have to re-establish the cache, which is another big waste! can you let my worker thread monopolize the core of CPU, so I can certainly improve a lot of productivity!"

The little brother thought for a moment and said, "No problem, you can have this!" it can be done with thread affinity. I'll draw a few cores for you and don't let them participate in the thread scheduling allocation of the system. I'll leave it to you. "

A few days after the interruption problem, the programmer brother upgraded me so that several of my worker threads could monopolize the CPU core, and the work efficiency was improved a lot.

However, it is still a long way from the 10-gigabit traffic analysis index required by the product manager.

One night, the little brother of the programmer talked to me again.

"now the analytical ability has indeed improved, but it is still far from the goal. Please tell me quickly. Do you have any suggestions for improvement?"

"Yes, there is, but I guess you will still say you can't change it." I rolled my eyes.

"tell me about it first!"

"now this packet is notified to be read in the form of an interrupt. Can I get it myself without interruption? you don't know that every interrupt has to save the context and switch from user mode to kernel mode. With so much traffic, it's too expensive!" Oh, I said excitedly.

The little brother was silent after hearing this.

"see, I told you you couldn't change it! forget it. Tell the product manager that this requirement can't be done as soon as possible, and we're both at ease."

"I can't do that. This project is very important to me. I'm counting on you to get a promotion and a raise and get to the top of my life." Oh, the little brother said very firmly.

"I really can't. Then find a few more machines and send me a few copies of copy. If the software doesn't work, it depends on the hardware to pile up the performance." Oh, I winked at him.

"needless to say, the boss won't agree."

"then there's nothing I can do. To tell you the truth, if I want to be able to handle 10 gigabytes of network traffic, I have to bypass the operating system. I have to read the data packet from the network card myself. Go and study it carefully. If you want to get a promotion and a raise, how can you be afraid of it?" Oh, I cheered my little brother up.

The little brother nodded. "you said, I can do it. Give me some time."

DPDK has been like this for more than a week, and my little brother programmer has never come to me again, and I don't know how his research is going.

After a few more days, he finally came again.

"come on out! I've found a way. I'll start to transform you tomorrow!"

As soon as I heard it, I was interested, "what way? how are you going to transform me?"

"this new solution can solve all the problems you raised before, allowing you to deal directly with the network card, without interrupting to notify the reading of data packets, and no longer having to hand over data packets to the system protocol stack and netfilter framework for processing, instead of frequently switching between user mode and kernel mode." Oh, the little brother is getting more and more excited!

"you are so awesome that you can solve all these problems! how do you do this and how does it work?" Oh, I asked curiously.

The little brother was a little embarrassed. "how can I do that?" in fact, this is a technology developed by others. I just use it. "

"well, have you figured out how it works? don't fool me at that time!" Oh, I'm a little worried.

"you can rest assured that this technology is called DPDK, which is developed by other people's Intel. It is reliable!"

Next, the little brother programmer introduced to me the principle of this technology called DPDK.

With DPDK, UIO is driven by the user mode of the operating system, and I can read the packets of the network card in the user mode by polling, without interrupting any more!

Read directly in the user state, there is no need to move the packet kernel state space and user state space back and forth. After reading it, I can analyze it directly, without having to go to the system protocol stack and netfilter to waste time, it's perfect!

"it's more than that! it also supports large-page memory technology," the little brother said proudly.

"large pages of memory? what is this?"

"doesn't the system manage memory pages in 4KB size by default? this unit is too small, there will be a large number of memory pages for our server memory, and in order to manage these pages, there will be a large number of page table items. The size of the cache TLB for memory address translation in CPU is limited, and too many page table items will fail frequently, reducing the speed of memory address translation!"

Hearing this, I suddenly understood: "I know, if you enlarge this unit, there will be fewer memory pages to manage, fewer page table items, and TLB will not fail easily. Is it right that address translation can be faster?"

"that's right, can you guess how big it is?" The little brother pretended to be mysterious.

"double it, 8KB?" When I saw my little brother shaking his head, I guessed, "is it 16KB?"

"it's too conservative to support both 2MB and 1GB sizes!"

"it's so big, so awesome!"

The second day of the idling problem, the little brother of the programmer began to restructure me thoroughly.

After the upgrade, I tried to run and found a problem: if there are not many or no data packets, my polling is basically a waste of time and has been idling, because I have an exclusive core. The occupancy rate of this core has always been 100%, and many other programs complain that I am * * no * *.

So my little brother programmer upgraded me to Interrupt DPDK mode: when there was no packet processing, I went to sleep and changed to interrupt notification. You can also share the CPU core with other threads, which is no longer exclusive, but DPDK threads have a higher scheduling priority, and once there are more packets, I return to polling mode, which allows me to switch flexibly.

The programmer brother worked overtime for two weeks in a row, and after some optimization and upgrading, my packet analysis and processing ability has been greatly improved.

However, it is a pity that after several rounds of testing, when faced with 10Gbps traffic, I am still a little bit inadequate, or a little worse.

The little brother was a little discouraged. "I don't know what to do. Do you think there is anything else that can be improved?"

"I am basically working at full capacity now, and there should be no room for improvement. Now the only place where I have time to catch my breath is when the data is in competition. When the data is locked, the thread switches and takes a break."

The little brother thought for a few seconds, then suddenly his eyes lit up and said happily, "got it!"

Before I could ask, I was shut down and went off work.

What exactly is the programmer's little brother going to do to me?

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.