Shulou(Shulou.com)06/02 Report--

This article is to share with you what the debugging tools commonly used in Linux background development are, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Linux entry compilation phase nm acquires symbol information contained in binaries strings acquires string constants contained in binaries strip removes symbols contained in binaries readelf displays target file details objdump disassembles source code as far as possible addr2line looks up lines of code according to address run phase gdb powerful debugging tool ldd displays the dynamic libraries that the program needs and the dynamic libraries actually used strace trace The program's current system call ltrace tracker's current library function time to view the program execution time, User mode time, kernel state time gprof display user mode function execution time valgrind check memory errors mtrace check memory errors other proc file system log 02 compilation phase nm (get symbols contained in binaries)

Symbols: functions, variabl

Parameters:

-C converts the C++ function signature into readable form-A lists the symbol name and shows which file it comes from. -a lists all symbols (this will also list debug symbols. Debug symbols are not listed by default)-l lists the corresponding line number of the symbol in the source code (after specifying this parameter, nm will use the debug information to find the file name and the line number of the symbol. For a defined symbol, the line number defined by the symbol is found, and for undefined symbols, it is displayed as blank)-n is sorted by the address of the symbol (by default, by alphabetical order of the symbol name)-u lists only the undefined symbol strings (get string constants in the binary file)

Features:

Get the string constant in the binary file

Purpose:

It is more important to check for KEY leaks

Eg:strings | grep'^.\ {16\} $'find if there is a line with 16 characters in it and display it.

Options:

-a not only scans the target file initialization and load segments, but scans the entire file. -f displays the file name before displaying the string. -n min-len prints a string at least min-len characters long. The default is 4. # strings / lib/tls/libc.so.6 | grep GLIBCGLIBC_2.0GLIBC_2.1GLIBC_2.1.1.

So you can see the versions supported by glibc.

Strip (remove the symbols contained in the binary file)

Purpose:

Executable programs to lose weight (usually only on build modules that have been debugged and tested, because they can't be debugged)

Decompilation, anti-tracking

Readelf (shows target file details)

The nm program can be used to enumerate symbols and their types and values, but to take a closer look at the contents of these named segments in the target file, you need to use more powerful tools. Two of the powerful tools are objdump and readelf.

Readelf tool A GNU tool used to display information about one or more files in ELF format. Use different parameters to view different information in the ELF file.

Readelf

-a display information for all ELF files-h display ELF file header-l display program header (program-header) and program segment (segment) and sections below sections-S show more detailed section information (section)-s display symbol information,-n display identification information (if any)-r display relocation information (if any)-u display expansion function information (if any)-d display dynamic section information Generally, it is the information of dynamic library.

Objdump (disassemble source code as much as possible) objdump-S

Disassemble the source code as much as possible, especially when the-g parameter is specified when compiling.

Addr2line (look up lines of code by address)

When a process crashes, additional information is given in the log file (/ var/log/messages), including the cause of program termination, the fault address, and a brief register dump containing program status words (PSW), general registers, and access registers.

Eg:Mar 31 11:34:28 l02 kernel: failing address: 0

If the executable includes debug symbols (compiled with-g), using addr2line, you can determine which line of code is causing the problem.

Eg:addr2line-e exe addr

In fact, gdb also has this feature, but the advantage of addr2line is that very often, bug is very difficult to reproduce, we only have a copy of crash log. In this way, you can find the corresponding lines of code using addr2line, which is very convenient.

Note:

The executable program is compiled with-g to carry debugging information. If the crash is in a so, the addr2line cannot directly give the code line.

Parameters:

-a displays the address before displaying the function name or file line number-b specifies the binary file format-C parses the C++ symbol as a user-level name, you can specify the parsing style-e specifies the binary file-f also displays the function name-s displays only the base name of the file Instead of the full path-I expand the inline function-j reads the offset from the specified section instead of the absolute address-p each location shows the 03 run phase on one line

Common steps to debug a program:

1. Determine whether the running time is mainly spent in user mode or kernel mode (a way to compare soil: the program temporarily shields daemon () calls. After hardcode receives n requests, exit (0), time the program. ).

2. If it is in user mode, gprof is used for performance analysis.

3. If it is in kernel state, strace is used for performance analysis, and other tools (such as ltrace, etc.) can be used to assist.

Ldd (showing the dynamic libraries needed by the program and the dynamic libraries actually used) # ldd / bin/lslinux-gate.so.1 = > (0xbfffe000) librt.so.1 = > / lib/librt.so.1 (0xb7f0a000) libacl.so.1 = > / lib/libacl.so.1 (0xb7f04000) libc.so.6 = > / lib/libc.so.6 (0xb7dc3000) libpthread.so.0 = > / lib/libpthread.so.0 (0xb7dab000) / lib/ld-linux.so.2 ( 0xb7f1d000) libattr.so.1 = > / lib/libattr.so.1 (0xb7da6000)

The first column: which library needs to be used; the second column: which library file is actually used; and the third column: the loading address of the library file.

If the dynamic library is missing, there will be no second column.

Strace (tracks the current system call)

The result is output to 2 by default.

-p "attach to a process-c finally counts the invocation of each system call-T prints the time of the system call call-t/-tt/-ttt time format-f Maqure F tracks the child process-o" generated by the fork/vfork call, and directs the output of the strace to the file.

Such as: strace-f-o ~ /

-e expr specifies an expression that controls how to trace, in the following format:-e open is equivalent to-e trace=open, indicating that only open calls are tracked

It is convenient to use strace-e open. / prg to see which configuration files or log files the program uses.

-e trace= "tracks only specified system calls

For example:-e trace=open,close,rean,write means that only these four system calls are tracked.

-e trace=file tracks only system calls related to file operations-e trace=process tracks only system calls related to process control-e trace=network tracks all system calls related to the network-e strace=signal tracks all system calls related to system signals-e trace=ipc tracks all system calls related to process communication (tracks current library functions)

The parameters are very close to strace.

Time (View program execution time, user mode time, kernel state time) # time ps aux | grep 'hi'1020 21804 0.0 1888 664 pts/6 S+ 17:46 0:00 grep hireal 0m0.009suser 0m0.000ssys 0m0.004s

Note:

Time only tracks the parent process, so you cannot fork

Gprof (shows the execution time of each function in user mode)

Gprof principle:

When compiling and linking programs (using the-pg compile and link option), gcc adds a function named mcount (or "_ mcount", or "_ _ mcount") to each function of your application, that is, every function in the application compiled by-pg calls mcount, and mcount keeps a function call graph in memory. And find the address of the child function and the parent function in the form of the function call stack. This call graph also holds all the information about the time of the call, the number of calls, and so on.

Steps to use:

1. Use-pg to compile and link the application

Gcc-pg-o exec exec.c

If you need library function calls:

Gcc-lc_p-gp-o exec exec.c

2. Execute the application to generate data gmon.out for gprof analysis

3. Use gprof program to analyze the data generated by the application.

Gprof exec gmon.out > profile.txt

Note:

The program must exit through the normal way (exit (), main returned), kill is not valid. Debugging of background resident programs-my comparison method is to shield daemon () calls and exit (0) after the program hardcode receives n requests.

Sometimes it's not accurate.

Despite the user state time consumption, there is no kernel state consumption in the tube.

Gdb core exec (gdb views the core file) is ready to generate core:

Before starting the program, ulimit-c unlimited, set the core file without limiting the size. (in contrast, ulimit-c 0 can prevent the generation of core files)

By default, a file named core is generated in the path of the executable, and the new core will overwrite the old one.

Set the core file name:

/ proc/sys/kernel/core_uses_pid can control whether pid is added as an extension to the filename of the resulting core file, 1 is an extension, otherwise it is 0.

Proc/sys/kernel/core_pattern can set the location or file name of the formatted core file. For example, if the original file content is core, you can change it to:

Echo "/ data/core/core-%e-%p-%t" > core_pattern

The following is a list of parameters:

% p-insert pid into filename add pid%u-insert current uid into filename add current uid%g-insert current gid into filename add current gid%s-insert signal that caused the coredump into the filename add signal resulting in core t-insert UNIX time that the coredump occurred into filename add unix time at the time of core file generation h-insert hostname where the coredump happened into filename add hostname% e-insert coredumping executable name into filename add command name

Use gdb to view the core:

Gdb

Opprofile (see where CPU is consumed)

Common command

To use oprofile for cpu usage detection, you need to initialize, start detection, export test data, view test results and other steps. The following are commonly used oprofile commands.

Initialization

Opcontrol-no-vmlinux: instructs oprofile not to record kernel module, kernel code related statistics opcontrol-init: load oprofile module, oprofile driver after starting detection

Detection and control

Opcontrol-start: instructs oprofile to start detection opcontrol-dump: instructs to write data detected by oprofile to file opcontrol-reset: data record detected before emptying opcontrol-h: close the oprofile process

View test results

Opreport: display test results from the perspective of image. Processes, dynamic libraries and kernel modules belong to the mirror category opreport-l: display test results opreport-l test from a functional point of view: display test results for test processes opannotate-s test: display test results for test processes from a code point of view opannotate-s / lib64/libc-2.4.so: from a code point of view Display test results for libc-2.4.so library linux # opreportCPU: Core 2 Speed 2128.07 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000CPU_CLK_UNHALT. | samples |% |-31645719 87.6453 no-vmlinux 4361113 10.3592 libend.so 7683 0.1367 Libpython2.4.so.1.0 7046 0.1253 op_test

Valgrind (check for memory errors)

Steps to use:

1. Download and install valgrind on the official website.

2.-g compiled programs can be used.

The sample code test.c of the official website

# include void f (void) {int* x = malloc (10 * sizeof (int)); x [10] = 0; / / problem 1: heap block overrun} / / problem 2: memory leak-- x not freedint main (void) {f (); return 0;}

Compiler gcc-Wall-g-o test test.c

3. Valgrind starts the program and outputs the results on the screen.

Valgrind-tool=memcheck-leak-check=full. / test

Note:

Valgrind can only find access errors in heap memory, but there is nothing to do with objects and static objects on the stack.

Valgrind can affect process performance, which is said to be 20 times slower, so when performance requirements are high, you can only use a lightweight tool like mtrace (but mtrace can only recognize simple memory errors).

If the stack of the core generated by the program is messy, then it is basically stackoverflow. This can be detected by adding-fstack-protector-all and-D_FORTIFY_SOURCE=2 at compile time. Stack-protector-all adds stack-protected code to each function and leaves fingerprints on the stack. (for the record, never used)

Because valgrind cannot check the memory access of stacks and static objects out of bounds, such problems can be detected by using gcc's-fmudflap-lmudflap. (for the record, never used)

There is still a better way to solve the problem of inconsistent types of global variables, which shows that the global object is not a good design from another aspect, which brings trouble to debugging.

Mtrace (check for memory errors)

Mtrace is a tool provided in glibc, the principle is very simple, that is, all the positions of malloc () and free () in your program come down, and the last two cars are paired, and the one that does not match is memory leak.

The steps to use are as follows:

1. Add mtrace () to the code

# include # include int main (void) {int * p; int iTinct Ifdef DEBUG setenv ("MALLOC_TRACE", ". / memleak.log", 1); mtrace (); # endif p = (int *) malloc (1000); return 0;}

This code malloc a space, but there is no free drop. Let's add a mtrace call on lines 9-12.

2. Compile gcc-g-DDEBUG-o test1 test1.c

3. Execute. / test1 and you will find. / memleak.log in the directory.

4. Use mtrace memleak.log to view information.

# mtrace test1 memleak.log- 0x0804a008 Free 3 was never alloc'd 0xb7e31cbe-0x0804a100 Free 4 was never alloc'd 0xb7ec3e3f-0x0804a120 Free 5 was never alloc'd 0xb7ec3e47Memory not freed:-Address Size Caller0x0804a4a8 0x3e8 at / home/illidanliu/test1.c:14

You can see that test1.c does not have a corresponding free ().

04 other proc file systems

The window of the kernel.

The proc file system is a pseudo file system that is stored in memory and does not take up out-of-memory space.

Users and applications can get information about the system through proc and can change some parameters of the kernel.

Proc/ directory structure (part):

Cmdline kernel command line cpuinfo about Cpu information devices available to devices (block devices / character devices) filesystems-supported file system interrupts interrupts using ioports I go O port use kcore kernel kernel image kmsg kernel messages meminfo memory information mounts-loaded file system stat comprehensive statistical state table swaps swap space utilization version kernel version uptime system uptime net network information sys writable It can be used to access or modify kernel parameters.

Proc// directory structure (part):

Cmdline command line parameter environ environment variable value fd A directory containing all file descriptors the memory of the mem process is utilized stat process status status Process status in human readable formcwd current working directory link exe Link to the executable of this processmaps memory image statm process memory status information root link to this process's root directory system log

Log files under / var/log/:

/ var/log/messages overall system information, which also contains logs during system startup. In addition, mail, cron, daemon, kern and auth are also recorded in the var/log/messages log. / var/log/auth.log system authorization information, including user login and permission mechanism. / var/log/boot.log log when the system starts. / var/log/daemon.log various system background daemon log information. / var/log/lastlog records the most recent information for all users. This is not an ASCII file, so you need to view the contents with the lastlog command. / var/log/user.log logs user information at all levels. / var/log/cron whenever the cron process starts a job, it records the relevant information in this file. / var/log/wtmp or utmp login information. / var/log/faillog user login failure message. In addition, incorrect login commands are also recorded in this file. What is Linux system Linux is a free-to-use and free-spread UNIX-like operating system, is a POSIX-based multi-user, multi-task, multi-threaded and multi-CPU operating system, using Linux can run major Unix tools, applications and network protocols.

These are the debugging tools commonly used in Linux background development, and the editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.