Shulou(Shulou.com)06/02 Report--

Learn more about how the Linux configuration / build system works.

Since the migration of Linux kernel code to Git, the Linux kernel configuration / build system (also known as Kconfig/kBuild) has been around for a long time. However, as a supporting infrastructure, it receives little attention; even kernel developers who use it in their day-to-day work never really consider it.

To explore how the Linux kernel is compiled, this article delves into Kconfig/kBuild internal processes, explains how .config files and vmlinux/bzImage files are generated, and introduces an intelligent technique for dependency tracking.

Kconfig

The first step in building a kernel is always configuration. Kconfig helps make the Linux kernel highly modular and customizable. Kconfig provides users with a number of configuration goals:

Config uses line-oriented programs to update current configuration nconfig uses ncurses menu-based programs to update current configuration menuconfig uses menu-based programs to update current configuration xconfig updates current configuration with qt-based front-end update current configuration gconfig uses GTK+-based front-end update current configuration oldconfig uses provided .config as a basis to update current configuration localmodconfig update unloaded current configuration disable module localyesconfig updates current configuration Convert local MODS to core defconfig A new configuration that gets the default configuration from the Defconfig provided by Arch Savedefconfig saves the current configuration as. / defconfig (minimum configuration) allnoconfig uses "no" to answer all options of the new configuration allyesconfig new configuration, in this configuration All options are accepted as "yes" accept allmodconfig if possible select a new configuration module alldefconfig set all symbols as default new configuration randconfig has random answers to all options listnewconfig lists new options olddefconfig is the same as oldconfig But without prompting, set the new symbol to the default value kvmconfig for kvm client kernel support enable other options xenconfig enable Xen dom0 and guest kernel support other options tinyconfig configure the smallest possible kernel

I think menuconfig is the most popular of these goals. The target is handled by different host programs, which are provided by the kernel and generated during the kernel build process. Some goals have a GUI (for the user's convenience), while most don't. The tools and source code related to kconfig are mainly located in the scripts/kconfig/ kernel source code. We can from scripts/kconfig/makefile, there are several host programs, including CONF, mconf, and nconf. In addition to CONF, each of them is responsible for one of the GUI-based configuration goals, so CONF deals with most of them.

Logically, the infrastructure of Kconfig has two parts: one implements the new language to define configuration items (see the Kconfig file under the kernel source code), while the other configuration items parse the Kconfig language and handle configuration operations.

The internal processes for most configuration goals are roughly the same (as shown below):

Note that all configuration items have a default value.

The first step is to read the Kconfig file under the source root to build the initial configuration database, and then to update the initial database by reading the existing configuration file according to this priority:

.config / lib/Module/$ (shell,uname-r) / .config / etc/kernel-config/boot/config-$ (shell,uname-r) ARCH_DEFCONFIGARCH/$ (ARCH) / Defconfig

If you are doing a GUI-based configuration, the database will be updated according to your customization through menuconfig or command-line-based configuration oldconfig. Finally, dump the configuration database into the .config file.

But the .config file is not the final material for kernel construction; that's why the syncconfig target exists. Syncconfig used to be called silentoldconfig, but it was not like the old name said, so it was renamed. In addition, because it is for internal use (not for users), it is removed from the list.

Here is an example of the syncconfig function:

Syncconfig accepts .config as input and exports many other files, which fall into three categories:

Auto.conf & tristate.conf is used to generate file text processing. For example, you might see a statement like this in the makefile of a component:

Obj-$ (CONFIG_GENERIC_CALIBRATE_DELAY) + = calibrate.o

Autoconf.h is used in C language source files.

The empty file include/config/ is used for configuration dependency tracking during kbuild, which is explained below.

After configuration, we will know which files and code snippets are not compiled.

KBuild

Component architecture, called recursive production, is a common way of GNU. Make it possible to manage a large project. KBuild is a good example of recursive make. By dividing the source file into different modules / components, each component is managed by its own Makefile. When you start building, the top-level Makefile calls the makefile of each component in the correct order, builds the components, and collects them into the final executor.

KBuild refers to different types of makefile:

Makefile is located at the top makefile in the source root. .config is the kernel configuration file. ARCH/$ (ARCH) / Makefile is an arched Makefile, which complements the top makefile. Scripts/Makefile* describes the common rules for all kbuild makefile. Finally, there are about 500 Kbuildmakefiles.

The top makefile contains archmakefile, read the .config file, enter the subdirectory, call production, and make the makefile of each component with the help of the routines defined in. Scripts/Makefile*, builds each intermediate object and links all intermediate objects to the vmlinux. The core file Documentation/kbuild/makefiles.txt describes all aspects of these production documents.

For example, let's look at how to generate vmlinux on x86-64:

(the illustration is based on the blog of Richard Y. Steven (Richard Y.Steven). It is updated and used with the permission of the author.

All .o files that enter vmlinux first enter their own built-in.a, which is represented by variables. KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_Main, KBUILD_VMLINUX_LIBS, and then collected into the vmlinux file.

Take a look at how to implement recursive make in the Linux kernel, with simplified Makefile code:

# In top Makefilevmlinux: scripts/link-vmlinux.sh $(vmlinux-deps) + $(call if_changed Link-vmlinux) # Variable assignmentsvmlinux-deps: = $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS) export KBUILD_VMLINUX_INIT: = $(head-y) $(init-y) export KBUILD_VMLINUX_MAIN: = $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y) export KBUILD_VMLINUX_LIBS: = $(libs-y1) export KBUILD_LDS: = arch/$ (SRCARCH) / kernel/vmlinux.ldsinit-y: = init/drivers-y: = drivers/ sound/ firmware/net-y: = net/libs-y: = lib/core-y: = usr/virt-y: = virt/# Transform to corresponding built-in.ainit-y: = $(patsubst% / % / built-in.a, $(init-y) core-y: = $(patsubst% /,% / built-in.a, $(core-y)) drivers-y: = $(patsubst% /,% / built-in.a, $(drivers-y)) net-y: = $(patsubst% /,% / built-in.a, $(net-y)) libs-y1: = $(patsubst% /,% / lib.a) $(libs-y)) libs-y2: = $(patsubst% /,% / built-in.a, $(filter-out% .a, $(libs-y)) virt-y: = $(patsubst% /,% / built-in.a, $(virt-y)) # Setup the dependency Vmlinux-deps are all intermediate objects, vmlinux-dirs# are phony targets, so every time comes to this rule, the recipe of vmlinux-dirs# will be executed. Refer "4.6Phony Targets" of `info make` $(sort $(vmlinux-deps)): $(vmlinux-dirs) # Variable vmlinux-dirs is the directory part of each built-in.avmlinux-dirs: = $(patsubst% /,%, $(filter% /) $(init-y) $(init-m)\ $(core-y) $(core-m) $(drivers-y) $(drivers-m)\ $(net-y) $(net-m) $(libs-y) $(libs-m) $(virt-y)) # The entry of recursive make$ (vmlinux-dirs): $(Q) $(MAKE) $(build) = $@ need-builtin=1

Recursive recipe extensions, such as:

Make-f scripts/Makefile.build obj=init need-builtin=1

This means that make will enter scripts/Makefile.build and continue to build every built-in.a. In. The scripts/link-vmlinux.sh,vmlinux file is finally located under the source root with the help of.

Understand vmlinux and bzImage

Many Linux kernel developers may not be aware of the relationship between vmlinux and bzImage. For example, here is their relationship in x86-64:

The source root vmlinux is stripped, compressed, put into the piggy.S, and then linked to the arch/x86/boot/compressed/vmlinux by other peers. At the same time, a file named setup.bin arch/x86/boot is generated below. There may be an optional third file that contains relocation information, depending on config_x86_RELOCS.

One named build is provided by the kernel and builds these two (or three) parts into the final bzImage file.

Dependency tracking

KBuild tracks three kinds of dependencies:

All prerequisite files (* .c and * .h) the options used by CONFIG_ in all prerequisite files are used to compile the target's command line dependencies.

The first one is easy to understand, but what about the second and the third? Kernel developers often see code snippets like this:

# ifdef CONFIG_SMP__boot_cpu_id = cpu;#endif

When the CONFIG_SMP changes, this code should be recompiled. The command line that compiles the source file is also important because different command lines may result in different object files.

When the .C file passes the # include directive, you need to write rules like this:

Main.o: defs.hrecipe...

When managing a large project, you need a lot of these rules; all of them can be boring. Fortunately, most modern C compilers can do this by looking at the lines in the # compiler source file. For the GNU compiler collection (GCC), just add one command line argument:-MD depfile

# In scripts/Makefile.libc_flags =-Wp,-MD,$ (depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE)\-include $(srctree) / include/linux/compiler_types.h\ $(_ _ c_flags) $(modkern_cflags)\ $(basename_flags) $(modname_flags)

This will generate a .D file with the following contents:

Init_task.o: init/init_task.c include/linux/kconfig.h\ include/generated/autoconf.h include/linux/init_task.h\ include/linux/rcupdate.h include/linux/types.h\...

The host program fixdep then handles the other two dependencies by getting the other two. The depfile command line is used as input, and then the .cmd file is output in makefile syntax, which records the target's command line and all prerequisites (including configuration). It looks like this:

# The command line used to compile the targetcmd_init/init_task.o: = gcc-Wp,-MD,init/.init_task.o.d-nostdinc. # The dependency filesdeps_init/init_task.o: =\ $(wildcard include/config/posix/timers.h)\ $(wildcard include/config/arch/task/struct/on/stack.h)\ $(wildcard include/config/thread/info/in/task.h)\ Include/uapi/linux/types.h\ arch/x86/include/uapi/asm/types.h\ include/uapi/asm-generic/types.h.

A .cmd file is included in the recursive build process, providing all dependency information and helping to decide whether to rebuild the target.

The secret behind this is that Fixdep parses depfile (.d files), then parses all dependent files in it, searches for all config_string text, converts them to corresponding empty files, and adds them to the target's prerequisites. Each time the configuration changes, the corresponding empty file is also updated, so kbuild can detect the change and rebuild the targets that depend on it. Because the command line is also recorded, it is easy to compare the final compilation parameters with the current compilation parameters.

look into the future

Kconfig/kbuild remained unchanged for a long time until the new maintainer, Shoichiro Yamada (Masahiro Yamada), joined in early 2017, and now KBuild is actively developing again. Don't be surprised if you soon see something different from this article.

Summary

The above is the whole content of this article. I hope the content of this article has a certain reference and learning value for everyone's study or work. Thank you for your support. If you want to know more about it, please see the relevant links below.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.