Shulou(Shulou.com)06/02 Report--

What this article shares with you is about how Linux modifies ELF to solve glibc compatibility problems. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Linux glibc problem

It is believed that many Linux users have encountered glibc compatibility problems when running programs released by a third party (not the system's own software source). This is generally due to the older version of the GNU C library (glibc) on the current Linux system. For example, I will report when I run a third-party closed source software on a CentOS 664-bit system:

[root@liangxu ~] # ldd tester./tester: / lib64/libc.so.6: version `GLIBC_2.17' not found (required by. / tester). / tester: / lib64/libc.so.6: version `GLIBC_2.14' not found (required by. / tester) linux-vdso.so.1 = > (0x00007ffe795fe000) libpthread.so.0 = > / lib64/libpthread.so.0 (0x00007fc7d4c73000) libOpenCL.so.1 = > / usr/lib64 / libOpenCL.so.1 (0x00007fc7d4a55000) libdl.so.2 = > / lib64/libdl.so.2 (0x00007fc7d4851000) libm.so.6 = > / lib64/libm.so.6 (0x00007fc7d45cd000) libgcc_s.so.1 = > / lib64/libgcc_s.so.1 (0x00007fc7d43b7000) libc.so.6 = > / lib64/libc.so.6 (0x00007fc7d4023000) / lib64/ld-linux-x86-64.so.2 (0x00007fc7d4e90000)

The glibc that comes with CentOS 6 is still very old version 2.12, while downloaded third-party programs rely on glibc version 2.17, in which case you can either recompile the program yourself or upgrade the glibc version of the system. As the program I use is written by a third party and the closed-source software cannot compile by itself, upgrading glibc may solve the problem, but glibc as the core basic library, after all, the direct upgrade action in the production environment is still too big, so I hope there is still a better solution.

Analysis of problems

First, we can check which symbols of the new version of glibc are used by the program, and use the objdump command to view the dynamic symbol information of the ELF file:

[root@liangxu ~] # objdump-T tester | grep GLIBC_2.1.*0000000000000000 DF * UND* 00000000000000 GLIBC_2.14 memcpy0000000000000000 DF * UND* 00000000000000 GLIBC_2.17 clock_gettime

From the above output, you can see that the program uses the memcpy function of glibc 2.14 and the clock_gettime function of glibc 2.17. These two commonly used functions should have been supported by glibc for a long time. We can confirm the symbol version provided by the current system glibc:

[root@liangxu ~] # objdump-T / lib64/libc.so.6 | grep memcpy0000000000091300 w DF .text 0000000000000009 GLIBC_2.2.5 wmemcpy0000000000101070 g DF .text 0000000000001b GLIBC_2.4 _ _ wmemcpy_chk00000000000896b0 g DF .text 00000000000465 GLIBC_2.2.5 memcpy00000000000896a0 g DF .text 00000000000009 GLIBC_2.3.4 _ _ memcpy_ CHK [root @ liangxu ~] # objdump-T / lib64/libc.so.6 | grep clock_gettime000000000038f800 g DO .bss 000000000008 GLIBC_PRIVATE _ vdso_clock_gettime

Here you can see that the memcpy implementation provided by the glibc library of CentOS 6 is version 2.2.5. In addition, libc does not directly implement the clock_gettime function, because in the old version of glibc, clock_gettime is supported by clock_gettime provided by the librt library, and it is also version 2.2.5:

[root@liangxu ~] # objdump-T / lib64/librt.so.1 | grep clock_gettime0000000000000000 DO * UND* 00000000000000 GLIBC_PRIVATE _ _ vdso_clock_gettime0000000000003e70 g DF .text 0000000000008b GLIBC_2.2.5 clock_gettime

After reading here, you can basically understand that the developers of third-party programs are compiled on the Linux system with a new version of glibc, and the implementation of memcpy and clock_gettime uses the latest version provided by glibc on this system by default, so it cannot run normally in the lower version of glibc system.

Solution method

Although we cannot recompile third-party programs, we should be able to avoid upgrading glibc if we can modify the ELF file to force the LD library to use older versions of the memcpy and clock_gettime implementation when loading the program.

Analyze ELF

First, use the readelf command to view the symbol table of ELF. Because there is a lot of output from this command, only the information we care about is posted here:

[root@liangxu] # readelf-sV testerSymbol table '.dynsym' contains 4583 entries: Num: Value Size Type Bind Vis Ndx Name 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND. 11: 00000000000000 0 FUNC GLOBAL DEFAULT UND memcpy@GLIBC_2.14 (5). 67: 00000000000000 0 FUNC GLOBAL DEFAULT UND clock_gettime@GLIBC_2.17 (16). 4582: 0000000000794260 70 FUNC WEAK DEFAULT 12 _ ZNSt15basic_streambufIwSVersion symbols section '.gnu.version' contains 4583 entries: Addr: 000000000045b508 Offset: 0x05b508 Link: 4 (.dynsym) 000: 0 (* local*) 0 (* local*) 2 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 004: 3 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 008: 4 (GLIBC_2.3.2) 3 (GLIBC_2.2.5) 0 (* local*) 5 (GLIBC_2.14). 040: 2 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 3 (GLIBC_2.2.5) 10 (GLIBC_2.17). 11e0: 1 (* global*) 1 (* global*) 11e4: 1 (* global*) 1 (* global*) 1 (* global*) Version needs section '.gnu.version_r' contains 6 entries: Addr: 0x000000000045d8d8 Offset: 0x05d8d8 Link: 5 (.dynstr) 000000: Version: 1 File: ld-linux-x86-64.so.2 Cnt: 1 0x0010: Name: GLIBC _ 2.3 Flags: none Version: 17 0x0020: Version: 1 File: libgcc_s.so.1 Cnt: 3 0x0030: Name: GCC_3.0 Flags: none Version: 13 0x0040: Name: GCC_3.3 Flags: none Version: 11 0x0050: Name: GCC_4.2.0 Flags: none Version: 10 0x0060: Version: 1 File: libm.so.6 Cnt: 1 0x0070: Name: GLIBC_2.2.5 Flags: none Version: 8 0x0080: Version: 1 File: libpthread.so . 0 Cnt: 2 0x0090: Name: GLIBC_2.3.2 Flags: none Version: 15 0x00a0: Name: GLIBC_2.2.5 Flags: none Version: 7 0x00b0: Version: 1 File: libc.so.6 Cnt: 10 0x00c0: Name: GLIBC_2.8 Flags: none Version: 19 0x00d0: Name: GLIBC_2.9 Flags: none Version: 18 0x00e0: Name: GLIBC_2.17 Flags: none Version: 16 0x00f0: Name: GLIBC_2.4 Flags: none Version: 14 0x0100: Name: GLIBC_2.3.4 Flags: none Version: 12 0x0110: Name: GLIBC_2.3 Flags: none Version: 9 0x0120: Name: GLIBC_2.7 Flags: none Version: 6 0x0130: Name: GLIBC_2.14 Flags: none Version: 5 0x0140: Name: GLIBC_2.3.2 Flags: none Version: 4 0x0150: Name: GLIBC_2.2.5 Flags: none Version: 3 0x0160: Version: 1 File: libdl.so.2 Cnt: 1 0x0170: Name: GLIBC_2.2.5 Flags: none Version: 2

We can see all the import and export symbols used by the program for dynamic linking in ELF's .dynsym dynamic symbol table. The numbers in parentheses after memcpy and clock_gettime are decimal version numbers (5 and 16, respectively), and we need to pay special attention to the following .gnu.version and .gnu.version _ r symbol version information sections.

The .gnu.version table contains version information for all dynamic symbols, and each symbol in the .gnu.version table can see the corresponding entry in .gnu.version (a total of 4583 symbols in .dynsym are exactly equal to the ending position 0x11e7 of .gnu.version).

As you can see from the output above, the .gnu.version table starts with the 0x05b508 offset, and we can look at the hexadecimal data for the corresponding offset:

Offset (h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F0005B500 00 00 00 03 00 .0005B510 03 00 03 00 03 00 03 00 03 00 00 0005 00 .0005B520 03 00 03 00 06 00 03 00 07 00 08 00 08 00. .0005B530 03 00 09 00 03 00 03 00 0A 00 07 00 03 00 00 00 .0005B540 03 00 03 00 0B 00 07 00 03 00 03 00 00 00 07 00 .0005B550 00 00 03 00 03 00 03 00 03 00 0C 00 09 00 00 00 .0005B560 07 00 03 00 03 00 07 00 03 00 07 00 0C 00 00 00 .0005B570 0D 00 03 00 07 00 07 00 0E 00 0F 00 03 00 0D 00 .0005B580 03 00 03 00 03 00 03 00 02 00 03 00 03 00 10 00 .0005B590 03 00 00 00 03 00 07 00 08 00 07 00 07 00 03 00 .0005B5A0 03 00 0D 00 03 00 00 00 03 00 03 00 03 00 00 00.

Each entry in .gnu.version takes up two bytes, and its value is the version of the symbol, so you can see that the offset of the 0x0b symbol (that is, the memcpy@GLIBC_2.14 symbol in the .dynsym table) is 0x05b51e (0x05b508 + 0x0b x 2), and the value 0x0005 of this offset corresponds to the value in the .dynsym table. Of course, the same is true of the offset 0x05b58e value of the clock_gettime symbol.

The following key .gnu.version _ r represents the version of the library file that the binary actually depends on. You can also see from the output that the .gnu.version _ r table is segmented according to different library files. Each entry takes up 16 bytes of 0x10. This table starts with the 0x05d8d8 offset. Let's take a look at GLIBC_2.17, which is the hexadecimal data at 0x05d9b8:

Offset (h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F0005D9B0 B0 70 03 00 10 00 97 91 96 06 00 10 00 °p.-'-.0005D9C0 BA 70 03 00 100 00 00 00 14 69 69 0D 00 00 00 E 00 °p.ii.0005D9D0 C5 70 03 00 10 00 00 74 19 69 09 00 00 C 00 p.t.i.0005D9E0 CF 70 03 00 10 00 00 13 69 69 0D 00 00 09 00 p.ii.0005D9F0 6A 70 03 00 10 00 00 17 69 69 0D 00 00 06 00 jp.ii.0005DA00 DB 70 03 00 10 00 00 00 94 91 96 06 00 0005 00 p. ". 0005DA10 E5 70 03 00 10 000 72 19 69 09 00 00 04 00 å p.r.i. 0005DA20 9A 70 03 00 10 00 00 00 75 1A 69 09 00 00 03 00 p.u.i.0005DA30 8E 70 03 00 00 00 01 00 D8 03 00 00.

Each entry in the .gnu.version _ r table is a 16-byte Elfxx_Vernaux structure declared as follows (2 bytes for Elfxx_Half and 4 bytes for Elfxx_Word):

Typedef struct {Elfxx_Word vna_hash; Elfxx_Half vna_flags; Elfxx_Half vna_other; Elfxx_Word vna_name; Elfxx_Word vna_next;} Elfxx_Vernaux

Vna_hash is the hash value of the 4-byte library name (that is, the GLIBC_2.17 string above), vna_other is the version value of the symbol in the corresponding .gnu.version table, vna_name refers to the offset to the library name string (which can also be found in the ELF header), and vna_next is the location of the next entry (usually fixed to 0x00000010).

From the output above, we can see that the first 4 bytes of vna_hash hash at the 0x05d9b8 corresponding to GLIBC_2.17 is 0x06969197, and the value of vna_other 0x0010 (Version: 16 in the output) is the same as the value of the clock_gettime symbol in .gnu.version. Similarly, GLIBC_2.14 matches the value of the memcpy symbol.

Modify ELF symbol table

Because the LD library in the Linux system (that is, the / lib64/ld-linux-x86-64.so.2 library) checks the symbols in the .gnu.version _ r table when loading ELF, we can use any hexadecimal editor to modify the symbol values in the .gnu.version _ r table to force the use of the older version of the function implementation.

First of all, we find that there are 10 entries under the libc.so.6 section of .gnu.version _ r, and the last one is the symbol of the GLIBC_2.2.5 version we need. (from the hexadecimal output above, we can see that the offset of the symbol is 0x05da28, the value of the hash is 0x09691A75, the value of the version is 0x0003, the name of the string points to the address 0003708E). Because in this way, we can directly point the other high-version entries under the libc.so.6 section to the values of the older version entries without changing the size of the ELF file.

For example, for the 0x05d9b8 offset corresponding to GLIBC_2.17, we can directly change the vna_hash value to the 0x09691A75 value of GLIBC_2.2.5 and the vna_other value to 0003708E. In order to be consistent with the version value in the .gnu.version table, we will not modify the vna_other value here.

We also modify the GLIBC_2.14 offset to the same value, and then check the saved ELF file with the readelf command to see the change (only the modified .gnu.version-r table is listed):

[root@liangxu ~] # readelf-sV tester.Version needs section '.gnu.version_r' contains 6 entries: Addr: 0x000000000045d8d8 Offset: 0x05e8d8 Link: 2 (.dynstr) 000000: Version: 1 File: ld-linux-x86-64.so.2 Cnt: 1 0x0010: Name: GLIBC_2.3 Flags: none Version: 17 0x0020: Version: 1 File: libgcc_s.so.1 Cnt: 3 0x0030: Name: GCC_3.0 Flags: none Version: 13 0x0040: Name : GCC_3.3 Flags: none Version: 11 0x0050: Name: GCC_4.2.0 Flags: none Version: 10 0x0060: Version: 1 File: libm.so.6 Cnt: 1 0x0070: Name: GLIBC_2.2.5 Flags: none Version: 8 0x0080: Version: 1 File: libpthread.so.0 Cnt: 2 0x0090: Name: GLIBC_2.3.2 Flags: none Version: 15 0x00a0: Name: GLIBC_2.2.5 Flags: none Version: 7 0x00b0: Version: 1 File : libc.so.6 Cnt: 10 0x00c0: Name: GLIBC_2.8 Flags: none Version: 19 0x00d0: Name: GLIBC_2.9 Flags: none Version: 18 0x00e0: Name: GLIBC_2.2.5 Flags: none Version: 16 0x00f0: Name: GLIBC_2.4 Flags: none Version: 14 0x0100: Name: GLIBC_2.3.4 Flags: none Version: 12 0x0110: Name: GLIBC_2.3 Flags: none Version: 9 0x0120: Name: GLIBC_2.7 Flags : none Version: 6 0x0130: Name: GLIBC_2.2.5 Flags: none Version: 5 0x0140: Name: GLIBC_2.3.2 Flags: none Version: 4 0x0150: Name: GLIBC_2.2.5 Flags: none Version: 3 0x0160: Version: 1 File: libdl.so.2 Cnt: 1 0x0170: Name: GLIBC_2.2.5 Flags: none Version: 2

Patchelf modifies the ELF file

If only a higher version of memcpy is used in a general program, it can be run after modification. Unfortunately, the third-party program I use also uses clock_gettime in the higher version of glibc, but if it is modified in this way, the runtime will still report an error because the libc 2.12 library of CentOS 6 does not provide clock_gettime.

At this time, we need to come up with PatchELF, a gadget developed by the NixOS team that can directly add, delete, and replace library files that depend on ELF files, and is very easy to use.

Check out the PatchELF source code and follow the steps described on the GitHub repository to compile and install it (the general distribution comes with an older version of the patchelf tool that does not support some new features).

Although the libc library of CentOS 6 does not provide clock_gettime implementation, fortunately, it is provided in the librt library that comes with glibc, so we can use the patchelf tool to modify the original program file and let the program load the librt library first, so that the program can load clock_gettime symbols correctly:

[root@liangxu] # patchelf-- add-needed librt.so.1 tester

Then modify the .gnu.version _ r table of the newly generated ELF file with a hexadecimal editor as described above (because the symbol table of the new ELF file is different from the previous one after patchelf runs), change GLIBC_2.17 and GLIBC_2.14 to GLIBC_2.2.5 symbols, and you can see the effect after saving the ELF file:

[root@liangxu ~] # ldd tester linux-vdso.so.1 = > (0x00007fffc17ee000) librt.so.1 = > / lib64/librt.so.1 (0x00007f7f84dca000) libpthread.so.0 = > / lib64/libpthread.so.0 (0x00007f7f84bad000) libOpenCL.so.1 = > / usr/lib64/libOpenCL.so.1 (0x00007f7f8498f000) libdl.so.2 = > / lib64/libdl.so.2 (0x00007f7f8478b000) libm.so.6 = > / lib64/libm.so.6 (0x00007f7f84507000) libgcc_s.so.1 = > / lib64/libgcc_s.so.1 (0x00007f7f842f1000) libc.so.6 = > / lib64/libc.so.6 (0x00007f7f83f5d000) / lib64/ld-linux-x86-64.so.2 (0x00007f7f84fd2000)

From the output of the ldd command, you can see that the modified program will load the librt library, and there is no error in the glibc version, and there is no problem with running the test program.

What is Linux system Linux is a free-to-use and free-spread UNIX-like operating system, is a POSIX-based multi-user, multi-task, multi-threaded and multi-CPU operating system, using Linux can run major Unix tools, applications and network protocols.

The above is how Linux modifies ELF to solve glibc compatibility problems. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.