Rust has a reputation of having good compiler error messages citation needed.
I generally agree! However, blindly following the hints given by the compiler may sometimes hurt beginners who don't fully understand the language. It doesn't help that due to Rust's excellent formatting of code suggestions citation needed, the suggestions really seem Correct and Canonical.
Case Study
Consider this problem:
Write a function that takes in a word and a mutable reference to an output string, then appends an asterisk to the output if the entire word is made up of ASCII characters.
They likely don't! But the pattern of "appending to an output" happens a lot, and that's the main focus here.
Here's how someone who is a beginner to Rust but is familiar with other programming languages might approach the problem:
fn (: , : &mut ) {
// ...
}
With the following thought process:
My function needs to receive a string, which is a
Stringin rust, and also a mutable reference to anotherString, which I know I can do by using&mut.
And here's how they might write the function body:
fn (: , : &mut ) {
if .() {
= + ::("*");
}
}
With the following thought process:
If the target is ASCII, I need to add an asterisk to the result. I read the documentation on
std::string::String, and know that I can create theStringfor the asterisk by usingString::from.
And now the beginner is trapped:
[K[0m[1m[38;5;9merror[E0369][0m[0m[1m: cannot add `String` to `&mut String`[0m
[0m [0m[0m[1m[38;5;12m--> [0m[0msrc/main.rs:3:25[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;12m3[0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m result = result + String::from("*");[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;12m------[0m[0m [0m[0m[1m[38;5;9m^[0m[0m [0m[0m[1m[38;5;12m-----------------[0m[0m [0m[0m[1m[38;5;12mString[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m[1m[38;5;9m|[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m[1m[38;5;9m`+` cannot be used to concatenate a `&str` with a `String`[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;12m&mut String[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;14mhelp[0m[0m: create an owned `String` on the left and add a borrow on the right[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;12m3[0m[0m [0m[0m[1m[38;5;12m| [0m[0m result = result[0m[0m[38;5;10m.to_owned()[0m[0m + [0m[0m[38;5;10m&[0m[0mString::from("*");[0m
[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m[38;5;10m+++++++++++[0m[0m [0m[0m[38;5;10m+[0m
[K[0m[1mFor more information about this error, try `rustc --explain E0369`.[0m
[K[0m[0m[1m[31merror[0m[1m:[0m could not compile `testing` (bin "testing") due to previous error
Leading to:
fn (: , : &mut ) {
if .() {
= .() + &::("*");
}
}
Resulting in:
[K[0m[1m[38;5;9merror[E0308][0m[0m[1m: mismatched types[0m
[0m [0m[0m[1m[38;5;12m--> [0m[0msrc/main.rs:3:18[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;12m1[0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0mfn append_asterisk_if_ascii(target: String, result: &mut String) {[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;12m-----------[0m[0m [0m[0m[1m[38;5;12mexpected due to this parameter type[0m
[0m[1m[38;5;12m2[0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m if target.is_ascii() {[0m
[0m[1m[38;5;12m3[0m[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m result = result.to_owned() + &String::from("*");[0m
[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[1m[38;5;9m^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^[0m[0m [0m[0m[1m[38;5;9mexpected `&mut String`, found `String`[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;14mhelp[0m[0m: consider dereferencing here to assign to the mutably borrowed value[0m
[0m [0m[0m[1m[38;5;12m|[0m
[0m[1m[38;5;12m3[0m[0m [0m[0m[1m[38;5;12m| [0m[0m [0m[0m[38;5;10m*[0m[0mresult = result.to_owned() + &String::from("*");[0m
[0m [0m[0m[1m[38;5;12m|[0m[0m [0m[0m[38;5;10m+[0m
[K[0m[1mFor more information about this error, try `rustc --explain E0308`.[0m
[K[0m[0m[1m[31merror[0m[1m:[0m could not compile `testing` (bin "testing") due to previous error
Leading to:
fn (: , : &mut ) {
if .() {
* = .() + &::("*");
}
}
And now the program compiles.
Space Analysis
Rustaceans People familiar with Rust might have been screaming for the past few paragraphs. Let's get the irrelevant (in this particular case study, but is very relevant in general and should be fixed) improvement out of the way:
fn (: , : &mut ) {
// ...
}
This function signature is overly specific. Since the only thing we need target for is the method .is_ascii, which does not mutate the String, we can avoid taking ownership of the String and use a &str instead, which is an immutable string slice.
In a similar vein, result should be &mut str, since you can "provide" a &mut str with types other than a String, so enforcing the restriction that it must be a String object is needlessly restrictive when all we are doing is appending a &str.
Now to the meat and potatoes:
if target.is_ascii() {
*result = result.to_owned() + &::("*");
}
This code is Not Good because of one reason: It makes plenty of unnecessary memory allocations. In fact, it makes 2 extra allocations per call, when in the ideal case it makes 0. The allocations are
result.to_owned(), which creates a clone ofresult, which is aString.String::from("*"), which creates a clone of the&'static strthat is"*".
Note that the + does not allocate a new string, but rather reuses the buffer of the LHS, which in this case is result.to_owned().
Let's find out! We'll use the heap profiling crate dhat-rs. Here's the code:
#[]
static : :: = ::;
fn (: &, : &mut ) {
if .() {
* = .() + &::("*");
}
}
fn () -> <(), <dyn ::::>> {
let = ::(10);
let = ::::().().();
("ascii!", &mut );
let = ::::();
!(" Max blocks:\t{}", .);
!(" Max bytes:\t{}", .);
!("Total blocks:\t{}", .);
!(" Total bytes:\t{}", .);
(())
}
A few things are of note here:
We ensure the
resultstring has sufficient capacity before the loop to avoid growing the string during the loop. Note that in this case, ensuring capacity does not change the memory used because the function replacesresulteach call.We create the heap profiler *after* creating the
resultstring to avoid measuring the heap allocation during the creation ofresult.
Here are the results:
[1m$ [33mcargo[0m run --release
Max blocks: 2
Max bytes: 9
Total blocks: 2
Total bytes: 9
From our analysis earlier we know why the maximum number of blocks is 2. The breakdown for maximum number of bytes is rather complicated, but the TLDR is that the minimum heap allocation size when growing a String is 8 bytes. If you're interested, here's the stack trace:
alloc::raw_vec::finish_grow (core/src/result.rs:0:23)
alloc::raw_vec::RawVec<T,A>::grow_amortized (alloc/src/raw_vec.rs:404:19)
alloc::raw_vec::RawVec<T,A>::reserve::do_reserve_and_handle (alloc/src/raw_vec.rs:289:28)
alloc::raw_vec::RawVec<T,A>::reserve (alloc/src/raw_vec.rs:293:13)
alloc::vec::Vec<T,A>::reserve (src/vec/mod.rs:909:18)
alloc::vec::Vec<T,A>::append_elements (src/vec/mod.rs:1992:9)
<alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<&T,core::slice::iter::Iter<T>>>::spec_extend (src/vec/spec_extend.rs:55:23)
alloc::vec::Vec<T,A>::extend_from_slice (src/vec/mod.rs:2438:9)
alloc::string::String::push_str (alloc/src/string.rs:903:9)
<alloc::string::String as core::ops::arith::Add<&str>>::add (alloc/src/string.rs:2264:14)
testing::append_asterisk_if_ascii (testing/src/main.rs:6:19)
testing::main (testing/src/main.rs:10:5)
with the relevant constant being MIN_NON_ZERO_CAP.
The 8 bytes, plus the 1 byte for String::from("*"), makes 9 bytes.
Elementary, my dear duckson. String::from takes a different code path!
<alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc/src/alloc.rs:241:9)
alloc::raw_vec::RawVec<T,A>::allocate_in (alloc/src/raw_vec.rs:184:45)
alloc::raw_vec::RawVec<T,A>::with_capacity_in (alloc/src/raw_vec.rs:130:9)
alloc::vec::Vec<T,A>::with_capacity_in (src/vec/mod.rs:670:20)
<T as alloc::slice::hack::ConvertVec>::to_vec (alloc/src/slice.rs:162:25)
alloc::slice::hack::to_vec (alloc/src/slice.rs:111:9)
alloc::slice::<impl [T]>::to_vec_in (alloc/src/slice.rs:441:9)
alloc::slice::<impl [T]>::to_vec (alloc/src/slice.rs:416:14)
alloc::slice::<impl alloc::borrow::ToOwned for [T]>::to_owned (alloc/src/slice.rs:823:14)
alloc::str::<impl alloc::borrow::ToOwned for str>::to_owned (alloc/src/str.rs:209:62)
<alloc::string::String as core::convert::From<&str>>::from (alloc/src/string.rs:2612:11)
testing::append_asterisk_if_ascii (testing/src/main.rs:6:40)
testing::main (testing/src/main.rs:10:5)
What this code path does exactly is outside my attention span pay grade.
Here it is:
fn (: &, : &mut ) {
if .() {
.('*');
}
}
Or this:
fn (: &, : &mut ) {
if .() {
* += "*";
}
}
Both do a whopping 0 extra allocations provided the result still has enough capacity to fit the new content. This is because the underlying buffer in result is reused, instead of a new string being created to replace it. We can see the effects more pronounced by doing more iterations of append_asterisk_if_ascii:
let = 100_000;
let mut = ::();
let = ::::().().();
for _ in 0.. {
append_asterisk_if_ascii("full ascii!", &mut );
}
let = ::::();
which still results in 0 allocations for the better version, but for the original...
[1m$ [33mcargo[0m run --release
Max blocks: 3
Max bytes: 399995
Total blocks: 299992
Total bytes: 14999949980
Time Analysis
Let's use hyperfine to benchmark two versions of our program!
First, we'll add the following to our Cargo.toml:
[features]
slow = []
fast = []
Then, we can include our two versions of append_asterisk_if_ascii:
#[(feature = "slow")]
fn append_asterisk_if_ascii(target: &, result: &mut ) {
if target.is_ascii() {
*result = result.to_string() + &::from("*");
}
}
#[(feature = "fast")]
fn append_asterisk_if_ascii(target: &, result: &mut ) {
if target.is_ascii() {
*result += "*";
}
}
We'll keep the same number of iterations as before, and run a comparison of the two features:
[1m$ [33mhyperfine[0m --warmup 5 [32m"cargo run --release --features fast"[0m [32m"cargo run --release --features slow"[0m
[2K[1mBenchmark [0m[1m1[0m: cargo run --release --features fast
[2K Time ([1;32mmean[0m ± [32mσ[0m): [1;32m 40.4 ms[0m ± [32m 0.8 ms[0m [User: [34m30.5 ms[0m, System: [34m9.7 ms[0m]
Range ([36mmin[0m … [35mmax[0m): [36m 39.6 ms[0m … [35m 43.8 ms[0m [2m67 runs[0m
[1mBenchmark [0m[1m2[0m: cargo run --release --features slow
[2K Time ([1;32mmean[0m ± [32mσ[0m): [1;32m759.6 ms[0m ± [32m 2.2 ms[0m [User: [34m257.7 ms[0m, System: [34m496.8 ms[0m]
Range ([36mmin[0m … [35mmax[0m): [36m755.8 ms[0m … [35m762.3 ms[0m [2m10 runs[0m
[1mSummary[0m
[36mcargo run --release --features fast[0m ran
[1;32m 18.81[0m ± [32m0.36[0m times faster than [35mcargo run --release --features slow[0m
18.8 times faster. Cool!
Conclusion
To be clear, I am not saying you should disregard the hints or help messages given by the Rust compiler. However, you should not assume that the help provided is accurate or solves the underlying problem exactly. The diagnostics given by the compiler is usually narrowly focused, and local rather than global.
Unfortunately, this is a tough problem to solve. For people looking to learn Rust, there's no way around taking the time to grok the reason for the language's existence. Tools like Clippy help with writing idiomatic code, but it isn't a panacea either. You just have to write code, possibly bad code, and keep telling yourself there must be a better way!