Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how Rust deals with memory segments". In daily operation, I believe many people have doubts about how to deal with memory segments in Rust. The editor consulted all kinds of data and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts about "how to deal with memory segments by Rust". Next, please follow the editor to study!

A brief history of garbage collection

When you look at Rust's website and read the introduction, you will suddenly find a proud statement that Rust does not have a garbage collector. If you are the same age as me, it will cause some bad memories. Sometimes you have to allocate memory manually, malloc (), and then release it later. If you release it prematurely, you will encounter attacks such as invalid memory access exceptions. If you forget to release it, it will cause a memory leak and block the application. Few people get it right the first time. It's not funny at all.

Before we look at the approach taken by Rust, let's take a brief look at the actual meaning of garbage. In Wikipedia, there is a good definition: garbage includes data. The program running on it will not be used in any future calculations. This means that only the developer can decide whether the memory segment that stores some data can be freed. However, the runtime of the application can automatically detect a subset of garbage. If a reference to a memory segment no longer exists at some point, the program will not be able to access the segment. And, therefore, it can be safely deleted.

In order to actually implement this support, the runtime must analyze all active references in the application and must check all allocated memory references if they can be accessed for the current application state. This is a task with a lot of calculation. On the first day of the birth of Java, JVM suddenly froze and had to be garbage collected for a long time. Today, there are many complex algorithms for garbage collection, which usually run at the same time as applications. However, the computational complexity remains the same.

On the plus side, application developers do not need to consider releasing memory segments manually. There will never be invalid memory access exceptions. She can still create memory leaks by referencing data, which is no longer needed. (with all due respect, the main example is a self-written caching implementation. Old man's advice: do not use methods such as ehcache.) However, with the introduction of garbage collectors, there are fewer and fewer memory leaks.

How does Rust handle memory segments

At first glance, Rust looks a lot like C, especially its references and dereferencing. But it has a unique way of dealing with memory. Each memory segment is owned by a reference. From the developer's point of view, only one variable always owns the data. If this variable is out of scope and is no longer accessible, transfer ownership to another variable or free memory.

With this method, it is no longer necessary to calculate the reachability of all data. Instead, each time the naming context is turned off (for example, by returning from a function call), a simple algorithm is used to verify the accessibility of the memory used. It sounds so great that every experienced developer may immediately think of a question: what's the problem?

The problem is that developers must take care of ownership. Instead of carelessly spreading references to data throughout the application, developers must mark ownership. If ownership is not clearly defined, the compiler prints an error and stops working.

In order to evaluate whether this method is really useful compared to traditional garbage collectors, I see two questions:

How hard is it for developers to mark ownership at development time? If all her efforts are focused on fighting the compiler rather than solving domain problems, then the benefits of this approach go far beyond help.

How fast is the Rust solution compared to traditional garbage collectors? If the benefits are small, then why bother?

To answer these two questions, I performed a task in Rust and Kotlin. This task is typical for an enterprise environment and produces a lot of garbage. The first question is based on my personal experience and point of view, and the second is based on specific measurements.

Task: working with the database

The task I chose was to simulate a typical database-centric task and calculate the average income of all employees. Each employee is loaded into memory and the average is calculated cyclically. I know you should never do this in real life, because the database can accomplish this task faster on its own. But first, I see that this often happens in real life, and second, for some NoSQL databases, you have to do this in your application, and second, it's just code to create a lot of garbage that needs to be collected.

I chose Kotlin on JVM as the representative of the garbage collection-based programming language. JVM has a highly optimized garbage collector, and if you are used to Kotlin, using Java is like working in the Stone Age.

Use Kotlin to process

Calculate a series of employees, summarize their salaries, calculate the number of employees, and finally divide by these numbers:

Fun computeAverageIncomeOfAllEmployees (employees: Sequence): Double {val (nrOfEmployees, sumOfSalaries) = employees. Fold (Pair (0L, 0L), {(counter, sum), employee-> Pair (counter + 1, sum + employee.salary)}) return sumOfSalaries.toDouble () / nrOfEmployees.toDouble ()}

There's nothing exciting here. You may notice a functional programming style. This is because I like functional programming very much. But this is not the theme of this article.) Garbage is created when employees are created. I create random employees here to avoid using real databases. However, if you use JPA, you will have the same number of object creations.

Fun lookupAllEmployees (numberOfAllEmployees: Long): Sequence {return (1L..numberOfAllEmployees) .asSequence () .map {createRandomEmployee ()}}

The creation of random objects is also very simple. A string is a charPool created from a list of characters.

Fun createRandomEmployee (): Employee = Employee (createRandomStringOf80Chars (), createRandomStringOf80Chars (),... / code cut Out) fun createRandomStringOf80Chars () = (1.. 80) .map {nextInt (0, charPool.size)} .map (charPool::get) .joinToString ("")

One of the little surprises of the Rust version is how I have to deal with the character list mentioned earlier. Because only one singleton is needed, it is stored in a companion object. Here is its outline:

Class EmployeeServices {companion object {privateval charPool: List = ('averse.. roomz') + (' Agar.. roomz') + ('0mm.. 9') fun lookupAllEmployees (...). Fun createRandomEmployee (): Employee... Fun computeAverageIncomeOfAllEmployees (...).}}

Now, deal with it in Rust way

The first thing I stumbled upon was where to put this singleton character list. Rust supports static data directly embedded in binaries and constant data that can be inlined by the compiler. The two options support only a small number of expressions to evaluate the value of the singleton. My solution for calculating the allowed character pool is as follows:

Let char_pool = ('averse.. roomz') .salary:: > ()

Because the vector is evaluated based on type inference, it cannot be specified as constant or static. My current understanding is that the usual method of Rust is to add all the objects that the function needs to handle as parameters. Therefore, the main call to calculate the average salary in Rust is as follows:

Let average = compute_average_income_of_all_employees (lookup_all_employees (nr_of_employees, & char_pool,))

In this way, all dependencies become clear. Developers with C experience will immediately recognize the address operator & which returns the memory address as a pointer and is the basis for efficient and possibly unmaintainable code. When many of my colleagues play with Rust, this C-based negative experience is projected onto Rust.

I don't think this is fair. The problem with C's & operator design is that there are always unpredictable side effects because every part of the application can store pointers to storage blocks. In addition, each part can free memory, which may cause all other parts to throw an exception.

In Rust, & operators work differently. Each data is always owned by a variable. If you use & this ownership to create a reference to the data, the ownership is transferred to the reference scope. Only the owner can access the data. If the owner is out of range, the data can be released.

In our example, char_pool uses the & operator to transfer ownership of to the parameters of the function. When the function returns, ownership is returned to the variable char_pool. Therefore, it is an address operator similar to C, but it adds the concept of ownership, which makes the code more concise.

Domain Logic in Rust

The main functions of Rust look similar to those of Kotlin. It feels a little basic because of the implied numeric type, such as f6464-bit floating-point numbers. However, this is something you will soon get used to.

Fn compute_average_income_of_all_employees (employees: impl Iterator)-> f64 {let (num_of_employees, sum_of_salaries) = employees.fold ((0u64, 0u64), | (counter, sum), employee | {return (counter + 1, sum + employee.salary);}) Return (sum_of_salaries as f64) / (num_of_employees as f64);}

With all due respect, this is a good example of how Rust is a very modern clean programming language and provides good support for functional programming styles.

Create garbage in Rust

Now let's take a look at part of the program where many objects are created and need to be collected later:

Fn lookup_all_employees impl Iterator + 'a {return (0..number_of_all_employees) .map (move | _ | {return create_random_employee (char_pool);}) .into _ iter ();}

At first glance, this looks a lot like Kotlin. It uses the same functional style to create random employees in the loop. The return type is Iterator, which is similar to the sequence in Kotlin, which is a list of deferred calculations.

From the second point of view, these types look strange. Since the Rust compiler cannot know when the return value is actually evaluated, and the return value depends on the borrowed reference, there is now a problem of determining when char_pool can release the borrowed value. The lifespan char_pool specified in the'a comment of must be at least as long as it is the return value.

This is a new concept for developers accustomed to classic garbage collection. In Rust, she sometimes has to explicitly specify the lifetime of an object. The garbage collector does all the things you don't need to clean up.

Third, you can find the move keyword. It forces the closure to take ownership of all variables it uses. Since char_pool (again), this is necessary. Map is deferred, so from the compiler's point of view, closures may exceed the lifetime char_pool of variables. Therefore, the closure must have its ownership.

The rest of the code is very simple. These structures are created from randomly created strings:

Fn create_random_employee (char_pool: & Vec)-> Employee {return Employee {first_name: create_random_string_of_80_chars (char_pool), last_name: create_random_string_of_80_chars (char_pool), address: Address {/ / cut out. }, salary: 1000,} } fn create_random_string_of_80_chars (char_pool: & Vec)-> String {return (0.80) .map (| _ | {char_pool [rand::thread_rng () .gen _ range (0) Char_pool.len ()]}) .into _ iter () .collect () }

So, how hard is Rust?

The implementation of this tiny test program is very complicated. Rust is a modern programming language that enables developers to maintain code quickly and cleanly. However, its concept of memory management is directly reflected in all elements of the language, which developers must understand.

Tool support is good, with all due respect. Most of the time, you only need to do what the compiler tells you. But sometimes you have to actually decide what to do with the data.

Now, is it worth it?

Even if it sounds convincing, I am more than happy to make some measurements to see if the reality is also convincing. So I ran the Rust and Kotlin applications for four different input sizes, measured the time, and put the results in a logarithmic scale:

Looking at these numbers, I have a long face. Rust is always slow; for 10 ^ 6 elements, a very bad factor is 11. That's impossible. I checked the code and found no errors. Then I checked the optimization and found-- the flag prod that release switched from dev mode to. Now, the result looks much better:

This is much better. Today, Rust is always faster than Kotlin and provides linear performance. On Kotlin, we see typical performance improvements in long-running code, which may be due to timely compilation. Judging from the input size of 10 ^ 4, Rust is about three times faster than Kotlin. This is very impressive given the maturity of JVM and the resources invested in infrastructure over the past few decades (the first version of Java was released in 1995).

To me, it is surprising that the development of configuration files is much slower than production profiles. The coefficient of 40 is so large that you should never use a development configuration file for distribution.

At this point, the study of "how to deal with memory segments in Rust" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.