Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to speed up the Rust compiler, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

Incremental compilation

# 68914: incremental compilation uses the "SipHasher128" hash algorithm to determine what code has changed since the last compiler call. This PR greatly improves the process of extracting bytes from the input byte stream (repeated to ensure that it works on both big-endian and little-endian platforms), and in most cases can increase compilation speed by up to 13%.

In this PR, Nicholas uses a simple shift algorithm to replace the previous slow algorithm, resulting in a smaller amount of code, eliminating a lot of unsafe code, and improved performance. In the Review process of the code, the influence of large and small end byte order on the hash algorithm is also discussed. Rust's CI runs tests on ARM, x86, and WASM, without a big-endian platform. But generally speaking, for different CPU architectures, Rust stores shaping data in the corresponding host byte order by default, which improves performance. So, the final result of the discussion is that the default implementation is correct in small-end order, and then a comment is left behind, and when the large-end order calls the relevant function, the caller transformation byte order is required.

# 69050: LEB 69050 encoding is widely used for storing metadata (metadata) in crate of Rust. However, the speed of Rustc encoding and decoding is not fast enough, this PR is to reduce the number of cycles in the coding and decoding process, thus improving the performance. It also eliminates the use of a Unsafe.

For this PR, by using Callgrind for performance analysis, the author finds that clap-rs-Check-CleanIncr is the benchmark + run + build combination that is most affected by LEB128 coding. Eighteen different methods have been tried for analysis, and 10 of them have the effect of performance improvement. Finally, I choose the current improvement method.

It is conceivable that it takes patient and scientific analysis to write Rust code with extreme performance.

LLVM intermediate code (Bitcode)

BitCode is a kind of intermediate code introduced by LLVM, which is the intermediate form in the process of source code being compiled into binary machine code, that is, it is neither source code nor machine code.

LLVM optimizes the code during compilation, which is based on BitCode. Various types of optimization are carried out on BitCode, and some logical equivalent exchange is carried out, so that the code execution efficiency is higher and the size is smaller.

For more information on BitCode, check out this article: https://xelz.info/blog/2018/11/24/all-you-need-to-know-about-bitcode/

Rust stores LLVM BitCode in rlib and dylib so that Rustc can perform cross-crate LTO optimization.

Last year, the author noticed from Rust's configuration file that rustc took some time to compress the LLVM BitCode it generated, especially in Debug mode. So the author tries to change it to not compress the BitCode which makes it a little faster but also significantly increases the size of the compiled artifacts on disk.

Then Alex Crichton (the official) tells the author something important: the compiler always generates object code and BitCode for crate. Object code is used for normal compilation, while BitCode is used for compilation through link time optimization (LTO). Users can only choose one at the same time, so generating two types of code is usually a waste of time and disk space.

So the author sent a RR # 66961, hoping not to store LLVM BitCode from rlib, otherwise it would cause the cache of incremental compilation to be too large. However, this caused widespread discussion, and after seven or eight PR refactorings, the problem was finally solved in # 71323.

In Debug mode, performance is improved by 18%, and rlibs disk footprint is reduced by 15 to 20 percent. If you use rustc instead of Cargo, you need to add-Cbitcode-in-rlib=no to apply this feature.

Other improvements

# 67079: improved shallow_resolved function for hot call mode (hot calling pattern) with 2% performance improvement.

# 67340: reduces the size of Nonterminal characters (generally thought of as variables, replaceable symbols) (to 40 bytes), greatly reducing the number of memcpy calls when building serde_derive. Performance improved by 2%.

# 68694: the borrowing of RefCell structure in InferCtxt is reduced and the performance is improved by 5%.

# 68848: the compiler's macro parsing code contains a loop that instantiates a large (Parser-type) complex value at each iteration, but most of these iterations do not modify the value. This PR changes the code, so it initializes a parser value outside the loop, and then uses Cow to avoid Clone it (except for modification iterations), resulting in a 15% increase in html5ever benchmark speed. (interestingly, the author says that he often uses Cow, but he can never remember the details about the use of CoW, so he can only go to the document every time.

A pending Bug that bothers the speed improvement of links

Using LLD (introduced by LLVM 4.0) as a linker can increase the link time exponentially. However, issues 39915 reported a Bug, so far LLD has not been able to become the default linker for rustc.

Features of LLD:

Cross-compilation is very friendly (focusing on embedded targets).

It's very fast. For incremental compilation, link time accounts for a large part of the compilation time, so it is important to halve this time.

Current status of Rust and LLD:

Rust publishes a copy of lld as a binary file, rust-lld, which can be used on most platforms

Rust-lld targets bare metal (bare metal) by default.

Rust-lld is used by default for wasm

You can use "- C linker-flavor" to explicitly require the use of rust-lld

Problems with using LLD elsewhere (Linux/ Mac/ Windows):

Lld's macOS backend crashed, although it has begun to rewrite, but it is still too early

On linux / unix platforms, ld / lld should not be called directly. Instead, the linker should be called through the system c compiler (that is, gcc), whose job is to find system symbols such as crt1.o and provide them to ld. This means that you can't just use rust-lld, but you have to type it into gcc / clang, and so on.

Windows-msvc is obviously OK, and there seems to be limited support for using rust-lld on the back end, but Rust officials don't know what needs to be done here.

Windows-mingw seems to be roughly similar to linux / unix, except that you might get an old GCC, and things are a little weird because pseudo-Windows-Linux is not a strictly tested configuration?

More generally, lld is new, it's not the default setting for most operating systems, and if we use it more often, random compound errors will almost certainly occur.

Compared with last year's compilation performance

Thank you for reading this article carefully. I hope the article "how to Speed up the Rust Compiler" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.