Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to understand the soft interrupt of the linux kernel". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to understand the soft interrupt of the linux kernel".

Introduction of soft interrupt

The processing that can be delayed is separated from the hard interrupt handler so that the processing can be run in the case of open interrupts, which is called soft interrupts. It can be seen that this separation of soft interrupts can greatly shorten the response time of hard interrupts, which is very important for many real-time applications.

We only talk about soft interrupts in this article, but we'll talk about tasklet, workqueue and so on later. When we talk about the soft interrupt process (see linux kernel 4.0), we will try to understand the details in depth and share our own understanding (if not, please point out, thank you).

Definition of soft interrupt data structure

There are currently 10 soft interrupts (defined by NR_SOFTIRQS), which are managed through the softirq_ vector [NR _ SOFTIRQS] array, all of which are shared by cpu.

Registration of soft interrupts

Bind the specific soft interrupt handler function to the soft interrupt number through open_softirq (). For example, the network system registers the soft interrupt handling function of sending and receiving packets:

The code is as follows:

Open_softirq (NET_TX_SOFTIRQ, net_tx_action)

Open_softirq (NET_RX_SOFTIRQ, net_rx_action)

Activation of soft interrupt

Each cpu has a bitmap of 32bit (that is, _ _ softirq_pending) to maintain whether the soft interrupt on this cpu is active.

The code is as follows:

Typedef struct {

Unsigned int _ _ softirq_pending

# ifdef CONFIG_SMP

Unsigned int ipi_ irqs[NR _ IPI]

# endif

} _ cacheline_aligned irq_cpustat_

Irq_cpustat_t irq_ stat[NR _ CPUS] _ cacheline_aligned

One of the activation opportunities for soft interrupts: irq_exit

Soft interrupts may be activated in the irq_exit function, and the activation conditions are:

Not in the hard interrupt and not in the soft interrupt and there are settings in the _ _ softirq_pending of this cpu.

The code is as follows:

If (! in_interrupt () & & local_softirq_pending ())

Invoke_softirq ()

From this condition, we can know that soft interrupts and hard interrupts are treated equally here (in in_interrupt), reflecting the essence of interrupt handling. Conditions that cannot be in hard interrupts indicate that priority must be given and that soft interrupts must be fully processed before soft interrupts are considered; conditions that cannot be in soft interrupts indicate that nesting of soft interrupts is shielded.

The invoke_softirq function handles either (wake up ksoftirqd first) and leave the soft interrupt to the ksoftirqd specialized thread, or call _ _ do_softirq immediate processing directly (of course, instant processing distinguishes which stack it is on: the current stack or a separate soft interrupt stack).

Let's take a look at the real-time process. The hard interrupt bit in the preempt_count is definitely cleared before the local_softirq_pending, and can be preempted if there is no soft interrupt bit in the preempt_count (turn off the hard interrupt immediately). Preemption must be prohibited during the entry into _ _ do_softirq to handle various soft interrupts. Preemption in the context of hard (soft) interrupts is notoriously not allowed: it makes the execution time of the interrupted process uncertain and unfair (that is, there is no intention of scheduling departure in the processing of hard and soft interrupts).

The second time to activate soft interrupt: raise_softirq

The packet receiving mode of network card has evolved from non-NAPI to NAPI mode, which fully demonstrates the advantages of soft interrupt: handing over the receiving task to soft interrupt processing to the maximum extent and simplifying the hard interrupt processing to the greatest extent. We'll talk about this kind of evolution later.

The raise_softirq function calls the _ _ raise_softirq_irqoff function to set the corresponding soft interrupt on the _ _ softirq_pending bitmap of the specified cpu. The difference between the raise_softirq_irqoff function and the raise_softirq function is whether the operation of the interrupt has been completed. Setting bitmap is a competitive operation, which can be done in all hard interrupts, so it has to be done in the case of off interruption.

The third activation of soft interrupt: ksoftirqd

Each cpu has a ksoftirqd thread that specifically handles soft interrupts when the number of soft interrupts is large:

The code is as follows:

DEFINE_PER_CPU (struct task_struct *, ksoftirqd)

The (loop) processing of run_ksoftirqd, the core function of the ksoftirqd thread, is to turn off the interrupt to see the setting of the _ _ softirq_pending of this cpu, execute _ _ do_softirqd () if there is any, and execute the open interrupt. This execution is smooth because it is on the thread's own stack and there is no problem affecting the user's process.

Here is a question, here used to be off preemption protection, now it is off interrupt protection (see patch 3e339b Use hotplugthread infrastructure softirq: 2012)? Our understanding is that the preemptive protection mode will allow more subsequent soft interrupts to be handled by ksoftirqd, which is not in line with the auxiliary status of ksoftirqd. As far as the status of dealing with soft interrupts is concerned, it should be dominated by irq_exit and supplemented by ksoftirqd. )

You can also see in ksoftirqd that it can be preempted before a soft interrupt is executed, but once execution is started, it cannot be preempted (consistent with the idea described in one of the above schedules: irq_exit). That is to say, the processing idea of soft and hard interrupts is the same: scheduling is not allowed during execution!

The reason why it cannot be preempted is actually a similar transactional principle: once it starts, it cannot be stopped. Another reason is that the user-defined hard (soft) interrupt program is executed, and the operation is uncertain, and if scheduling is possible during these operations, it will be out of the control of the kernel.

Activation of soft interrupts IV: elsewhere

For example, netif_rx_ni (), which is preemptive before performing a do_softirq, cannot be scheduled during a soft interrupt.

Activation of soft interrupts part 5: local_bh_enable

The code is as follows:

If (unlikely (! in_interrupt () & & local_softirq_pending ()

Do_softirq ()

Come to think of it, if exceptions and soft interrupts have shared data, exception handling needs to turn off soft interrupts when they reach the critical area of this shared data, but not hard interrupts. So when you finish the critical area, you need to turn on a soft interrupt, and this is the time to activate (see preempt_count, in fact, it may also be a preemptive time).

The reason for using "activate" instead of "invoke" is that the peripheral processing only modifies the _ _ softirq_pending bitmap of this cpu, which is actually handled by core mechanisms (such as ksoftirqd, soft interrupt handling that can pass in_interrupt inspection), and this is the idea of soft interrupts: make hard interrupts (or others) execute faster, so they don't use direct calls.

The principle of "activation" is that whoever activates and handles the soft interrupt on which cpu will handle the soft interrupt caused by the hard interrupt (in other words, the home cpu is the soft interrupt followed by the hard interrupt). In this way, give full play to the advantages of smp, balanced to each cpu. As for the relationship between hard interrupts and cpu, we'll talk about it later when we talk about hard interrupts. Each cpu maintains its own soft interrupt mechanism, and each cpu is not related to each other. Note that there is a correlation: when each cpu processes the same type of soft interrupt in parallel, this type of soft interrupt processing needs to protect the shared data, which is the price of soft interrupt reentrant.

Do_softirq for soft interrupt Core function processing

Do_softirq first checks the soft interrupt reentry condition: it must not be in the hard interrupt and does not exist in the soft interrupt. After meeting the conditions, you can start to do the following soft interrupt processing:

The code is as follows:

Pending = local_softirq_pending ()

If (pending)

_ _ do_softirq ()

This processing is done under the protection of off interrupts, after all, soft interrupts and hard interrupts are essentially the same, both of which are interrupt systems (of course, entering the hard / soft interrupt is another matter). You can also see that the local variable pending is not passed into _ _ do_softirq, so it is only judged here, not used. There may be differences between the judged value and the internal value, and there will be fewer bits in the bitmap.

Let's take a closer look at this inspection condition. Our understanding is:

This condition achieves two effects: soft interrupts on the same cpu are not nested, and soft interrupts are not handled in nested hard interrupts. For the same cpu, the execution of the _ _ do_softirq function is serial and non-reentrant (the do_softirq function can be said to be reentrant); for multiple cpu, the _ _ do_softirq function is reentrant, even if it is the same type of soft interrupt. In other words, the soft interrupt can serialize the soft interrupt processing on this cpu through this check condition, of course, the multi-cpu is parallel, so the same type of soft interrupt processing still needs to protect its own related shared data structures.

_ _ do_softirq of soft interrupt kernel function processing

The _ _ do_softirq function processing is as far as possible (although it may not be finished) to perform all activated soft interrupts (identified by the _ _ softirq_pending bitmap on this cpu). We analyze it in three stages.

Prepare for processing: turn off soft interrupts (the effect is to make the above-mentioned check condition true, thus achieving the purpose of prohibiting soft interrupt nesting on this cpu).

Core processing stage: turn off the hard interrupt, obtain the _ _ softirq_pending bitmap of this cpu and store it, clear the bitmap, open the hard interrupt (only turn off the hard interrupt when reading and writing the bitmap to prevent other hard interrupts from operating at the same time). Execute all the soft interrupts of this cpu (obtained from the stored bitmap). This core process is a loop, up to 10 times (MAX_SOFTIRQ_RESTART). After all, it uses the stack of the user process and cannot be borrowed for too long. The condition for exiting the loop is that the total time is exceeded or preempted (if the interrupt is opened, it will be preempted) or up to 10 times.

End processing stage: turn off hard interrupt, turn on soft interrupt.

In addition, if the soft interrupts can not be solved for 10 cycles, it means that there are a lot of hard interrupts and a lot of additional soft interrupts. Then it does not continue to affect the borrowed user process stack and passes it directly to a dedicated ksoftirqd kernel thread for processing. This also explains the meaning of the loop: new hard interrupts will be entered during soft interrupts, bringing in new soft interrupts (of course, only setting on the _ _ softirq_pending of this cpu, there will be no actual processing), so it needs to be processed repeatedly (the goal of processing is very clear, that is, to clear the _ _ softirq_pending bitmap on this cpu).

And look at the process that prevents soft interrupts from nesting. If the current kernel path An is interrupted by another kernel path B before the atomic operation, then B executes the hard interrupt and soft interrupt and returns to A here, and A then executes the atomic operation, and the subsequent soft interrupt processing should be idle, because it must have been finished by B. If B interrupts after the atomic operation, B executes the hard interrupt, does not execute its own soft interrupt, but exits directly (because the soft interrupt is nested), returns to A here, and A then executes. This time A will handle B's soft interrupt in addition to its own soft interrupt.

As you can see from the above, the soft interrupt bit in preempt_count has two functions: to prevent soft interrupts from being nested on a single cpu, and to ensure that they are not preempted during the execution of soft interrupts.

Finally, I have to repeat: the _ _ do_softirq functions mentioned here are all processed on one cpu, and parallelism on multiple cpu is not controlled at all.

Thank you for your reading. the above is the content of "how to understand the soft interrupt of the linux kernel". After the study of this article, I believe you have a deeper understanding of how to understand the soft interrupt of the linux kernel, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.