Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about what the Linux kernel refers to, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

The kernel belongs to the core part of the operating system, which has the basic functions of the operating system, and is mainly responsible for managing the memory, processes, device drivers, file system and network interface of the system. Therefore, the performance and stability of the operating system are determined by the kernel.

1. Memory management

The use of memory by processes

All programs to be executed in the computer must occupy a certain amount of memory, which is mainly used to store the program code stored from the disk, but also to store the data input by the user.

Linux operating system uses virtual memory management technology, so that each process has its own non-interference process address storage space. The space is a linear virtual space with a block size of 4GB. What the user sees and touches is a virtual address, but can not see the actual physical memory address. Therefore, the use of virtual address can not only protect the operating system, but also facilitate user programs to use address space larger than the actual physical memory.

An ordinary process consists of five different data segments: code segment, data segment, BSS segment, stack and stack:

Code snippet: an operation instruction mainly used to store executable files. The code snippet only allows read operations, not modify operations. It is to prevent illegal modification at run time.

Data segment: the data segment is used to store the initialized global variables in the executable file, that is, the statically assigned variables and global variables of the program

BSS segment: the BSS segment contains uninitialized global variables in the program, and all BSS segments are set to "0" in memory.

Heap: a heap is used to store memory segments that are dynamically allocated while a process is running. its size is not fixed and can be expanded or shrunk dynamically. When the process calls the "malloc" function to allocate memory, the newly allocated memory is dynamically added to the heap (the heap is expanded); when the memory is freed using the "free" function, the freed memory is removed from the heap (the heap is reduced)

Stack: a stack is a local variable temporarily created by the user to store the program. The stack has the feature of first-in, first-out, so the stack can be used to save or restore the call site. Therefore, you can think of the stack as a memory area for storing and exchanging temporary data.

Note that segments, BSS, and heaps are usually stored contiguously, that is, contiguous in memory locations.

Physical memory

Physical memory is the real size of memory provided by system hardware. In addition to physical memory, there is also a virtual memory in the Linux system. Virtual memory exists to meet the shortage of physical memory. It is a piece of logical memory virtualized by disk space. The disk space used as virtual memory is called swap space.

For computer systems, the computer's memory and other resources are fixed and limited. In order to make the limited physical memory meet the large memory demand of applications, the Linux system adopts a memory management method called "virtual memory". Although the object that the application operates on is virtual memory mapped to physical memory, the processor manipulates physical memory directly. For example, when an application accesses a virtual address, it must first convert the virtual address to a physical address before the processor can resolve the address access request.

As an extension of the physical memory, the Linux system will use the virtual memory of the swap partition when the physical memory is insufficient. In general, the kernel will store the temporarily unused memory block information into the swap space, so that the physical memory can be released, and this memory can be used for other purposes. When the original content is needed, the information will be re-read from the swap space to the physical memory.

The Linux system divides memory into "memory pages" that are easy to handle. Linux includes ways to manage available memory, as well as hardware mechanisms used for physical and virtual mapping. But memory management does not just manage 4KB buffers. Linux provides abstractions for 4KB buffers, such as the slab allocator. This memory management model uses 4KB buffers as the cardinality, allocates structures from them, and tracks the usage of memory pages, such as which pages are full, which pages are not fully used, and which pages are empty. This allows the mode to dynamically adjust memory usage according to system needs.

Note that the memory management of the Linux kernel adopts a paging access mechanism. In order to ensure that the physical memory can be fully utilized, the kernel will automatically exchange the data block information that is not frequently used in the physical memory into the virtual memory at an appropriate time, and keep the frequently used information to the physical memory.

In order to support multiple users to use memory, it sometimes occurs that the available memory is completely consumed. For this reason, pages can be moved out of memory and placed on disk, a process called swapping, in which pages are swapped from memory to the hard disk.

Linux system is conditional for page exchange. Not all pages are swapped into virtual memory when they are not in use. According to the most frequently used algorithm recently, the Linux kernel only needs to swap some infrequently used page files into virtual memory. For example, when a process that takes up a lot of memory needs to consume a lot of memory resources, some uncommonly used page files will be swapped into virtual memory. When the process that takes up a lot of memory resources ends and releases a lot of memory, the page files that have just been swapped out will not be automatically swapped into physical memory, and there will be a lot of free physical memory in the system. Swap space is also used.

The pages of swap space will first be swapped to physical memory when in use. If there is not enough physical memory to accommodate these pages, they will be swapped out immediately. As a result, there may not be enough space in virtual memory to store these swap pages, resulting in fake panic and service exception problems in the Linux system, although it can recover itself within a period of time. But the restored system is basically unusable.

Therefore, it is very important to plan and design Linux memory usage reasonably.

2. Process management

What is the process?

A process is a separate program that runs in its own virtual address space. From the operating system's point of view, everything running on the system can be called a process. In a Linux system, multiple processes can be run at the same time, and Linux can realize "multitasking" by running these processes in turn in a short interval of time. This short time interval is called "time slice", the method of letting the process run in turn is called "process scheduling", and the program that completes the scheduling is called the scheduler.

The difference between a program and a process is that although a process is generated by a program, it is not a program. A program is a collection of process instructions, which can enable one or more processes. At the same time, the program only occupies disk space, not the system running resources, while the process only takes up the system memory space, which is dynamic and changeable. If the process shuts down, the memory resources it occupies will be released.

Process scheduling controls the process's access to CPU. When you need to select the next process to run, it is up to the scheduler to select the process that is most worth running. A running process is a process that is waiting for CPU resources, and if a process is waiting for other resources, it is not runnable. Linux uses a relatively simple priority-based process scheduling algorithm to select new processes.

Through the multitasking mechanism, each process can occupy the computer separately, thus simplifying the programming. Each process has its own address space and can only be accessed by this process, so that the operating system avoids mutual interference between processes and the possible harm caused by "malicious attack" programs to the system. In order to accomplish a particular task, it is sometimes necessary to combine the functions of two programs, such as one program outputs text and the other program sorts the text. To this end, the operating system also provides a communication mechanism between processes to help accomplish such tasks. The common inter-process communication mechanisms in Linux systems are signals, pipes, shared memory, semaphores, sockets and so on.

The kernel provides an application programming interface (API) through SCI to create a new process (fork, exec, or Portable Operating System Interface [POS Ⅸ] function), stop processes (kill, exit), and communicate and synchronize between them (signal or POS Ⅸ mechanism).

Classification of processes

According to the functions of processes and the classification of running programs, processes can be divided into two main categories:

System process: management tasks such as memory resource allocation and process switching can be performed; moreover, the operation of this process is not interfered by users, and even root users cannot interfere with the operation of system processes

User process: a process generated by executing a user program, application, or system program outside the kernel that can be run or shut down under the control of the user.

For user processes, they can be divided into interactive processes, batch processes and daemons:

Interactive process: a process started by a Shell terminal that needs to interact with the user during execution, either in the foreground or in the background

Batch process: this process is a collection of processes that are responsible for starting other processes sequentially

Daemon: a daemon is a process that runs all the time, often starts when the Linux system starts and terminates when the system shuts down. They are independent of the control terminal and periodically perform certain tasks or wait for certain events to be processed.

Status of the process

After the process starts, it does not start immediately, and usually has the following five states:

Runnable status: running or preparing to run

Interruptible waiting state: in the blocking state, if a certain condition is reached, it will become a running state. At the same time, the process in this state will be awakened in advance due to receiving the signal, thus entering the running state.

Uninterrupted wait state: similar to the meaning of "interruptible wait state", except that the process in this state does not respond to the signal

Zombie state: also known as zombie process, each process will be in a zombie state after the end, waiting for the parent process to call and then release resources, the process in this state has ended, but its parent process has not yet released its system resources

Paused status: indicates that the process is temporarily stopped at this time to receive some special processing.

3. File system

The Linux operating system supports various file systems through a component called VFS (Virtual File System), that is, the virtual file system.

The virtual file system hides the specific details of various hardware, separates the file system operation from the specific implementation details of different file systems, and provides a unified interface for all devices. the virtual file system provides dozens of different file systems. Virtual file system can be divided into logical file system and device driver. Logical file system refers to the file system supported by Linux, such as ext 3, fat, etc., and the device driver refers to the device driver module written for each hardware controller.

The virtual file system is a very useful aspect of the Linux kernel because it provides a general interface abstraction for the file system, that is, VFS provides an exchange layer between the user and the file system.

Note that when the Linux operating system starts, the first thing that must be mounted is the root file system; if the system cannot mount the root file system from the specified device, the system will make an error and exit boot. Other file systems can then be mounted automatically or manually. Therefore, different file systems can exist in a system at the same time.

This section begins with a brief introduction to the file system, which will be explained in more detail in subsequent chapters.

4. Device driver

Device driver is the main part of the Linux kernel, it is a software layer between the application and the actual hardware, the same hardware, loading different drivers may provide different functions.

Like other parts of the operating system, device drivers run in a highly privileged processor environment, so they can operate directly on the hardware, but because of this, any error in the device driver can lead to a crash of the operating system. The device driver actually controls the interaction between the operating system and hardware devices.

The device driver provides a set of abstract interfaces that can be understood by the operating system to complete the interaction with the operating system, while the specific operation details related to the hardware are completed by the device driver. Generally speaking, the device driver is related to the control chip of the device. For example, if your computer's hard disk is a SCSI hard drive, you need to use a SCSI driver instead of an IDE driver.

The equipment is classified as follows:

Character equipment

A character device is a device that can be accessed like a file, which is implemented by a character device driver. Character device drivers usually implement at least open, close, read, and write system calls.

Character devices can be accessed through file system nodes. the only difference between these device files and ordinary files is that the access to ordinary files can be moved back and forth, while most character devices are a data channel that can only be accessed sequentially. A character device is a byte stream device, access to the device can only be accessed sequentially, byte access, not random access, the character device does not have a request buffer, all access requests are executed sequentially.

Block equipment

Block devices are also accessed through device nodes. The file system can be held on the block device.

In most UNIX systems, block devices can only transfer one or more complete blocks at a time, each containing 513 bytes. The Linux kernel allows applications to read and write block devices like character devices, allowing any number of bytes of data to be passed at a time. Therefore, the difference between block devices and character devices lies only in the way the kernel manages data, that is, the software interface between the kernel and the driver, and these differences are transparent to the user.

Storage devices generally belong to block devices, which have request buffers and support random access without having to access data sequentially. The disk devices under the Linux kernel are all block devices. Although there are block device nodes under the Linux kernel, applications generally access the block devices through the file system and its cache, rather than directly reading and writing data on the block devices through the device nodes.

Network equipment

Unlike character devices and block devices, network devices are message-oriented rather than stream-oriented. They do not support random access and do not have request buffers. Because it is not a stream-oriented device, it is difficult to map network interfaces to nodes in the file system.

The communication between kernel and network device driver is completely different from that between kernel and character and block driver. The kernel calls a set of functions related to packet transmission instead of read and write. The network interface does not have the same device number as character devices and block devices, only a unique name, such as eth0, eth2, etc., and this name does not need to correspond to the device file node.

The difference between character devices and block devices is roughly in the following three aspects:

Character devices are stream-oriented with a minimum access unit of bytes, while block devices are block-oriented with a minimum access unit of 513 bytes

Character devices can only be accessed sequentially by bytes, while block devices can be accessed randomly

The block device can hold the file system, in the form of access, the character device can be accessed through the device node, and although the block device can also be accessed through the device node, the data is generally accessed through the file system.

5. Network interface

Network interface can be divided into network protocol and network driver. Network protocols are responsible for implementing every possible network transport protocol. As we all know, TCP/IP is not only the standard protocol of Internet, but also the de facto industrial standard.

Naming of network interfaces

There is no clear specification for the naming of network interfaces, but the definition of network interface names is generally meaningful. For example:

Abbreviation for lo:local, which generally refers to the local interface

Abbreviation for eth0:ethernet, commonly used on Ethernet interfaces

Wifi0:wifi is a wireless local area network, which generally refers to the wireless network interface.

The work of network interface

The network interface is the basic device used to send and receive data packets. All the network interfaces in the system form a chain structure, and the programs in the application layer are called by name when they use the network interface. Each network interface corresponds to a struct net_device structure in the Linux system, including name, mac, mask and other information. A hardware network card corresponds to a network interface, and its work is completely controlled by the corresponding driver.

Virtual network interface

Virtual network interface has been widely used. "lo" (local interface) is one of the most common interfaces, almost every Linux system has this interface. The virtual network interface does not really receive and send packets from the outside world, but receives and sends packets within the system, so the virtual network interface does not need a driver.

Note that the virtual network interface is consistent with the real network interface in use.

Creation of network interface

The network interface of the hardware network card is created by the driver. The virtual network interface is created by the system or through the application layer program. There are two functions to create a network interface in the driver:

Register_netdev (struct net_device *)

Or

Register_netdevice (struct net_device *)

The difference between these two functions is that the former automatically generates an interface that starts with "eth", while the latter needs to specify the name of the interface in advance.

Note that register_netdev (struct net_device *) is also implemented by calling register_netdevice (struct net_device *).

What is Linux system Linux is a free-to-use and free-spread UNIX-like operating system, is a POSIX-based multi-user, multi-task, multi-threaded and multi-CPU operating system, using Linux can run major Unix tools, applications and network protocols.

After reading the above, do you have any further understanding of what the Linux kernel means? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.