Understanding Memory Management, Part 5: Fighting with Rust

[steveklabnik]

This is a good post! A few comments:

> Function Overloads

Strictly speaking, Rust doesn't support overloaded functions. Function overloading is when you define the same function but with different arguments, and the language selects the correct one based on the argument type. In this case, it's two different implementations of a trait for two different types. That said, it's close enough that this isn't really an issue, more of just a technical note, since this is a series trying to get into details.

> I can't find an explanation in the Rust documentation but I expect the reason is that someone could implement another trait that provides .into_iter() on whatever the x is in for y in x, thus resulting in a compiler error because there would be two candidate implementations.

Yep, I'm not sure that there is an official explanation anywhere else, but this is exactly what I would assume as well. This ensures that the correct implementation is called. This is also one of the reasons why adding a trait implementation isn't considered a breaking change, even if it could create a compiler error, because you can always expand it yourself to explicitly select the correct choice. Of course, these situations are usually treated more carefully then they have to be, because breakage isn't fun, even if it's technically allowed.

> But wait, you say, I'm doing exactly this in the first program, and indeed you are.

It's not the same, as the next paragraphs explain.

> We are able to examine the function and realize it's safe, but because the compiler wants to use local reasoning, it's not able to do so.

This is a super important point!

[junon]

> I can't find an explanation in the Rust documentation but I expect the reason is that someone could implement another trait that provides .into_iter() on whatever the x is in for y in x, thus resulting in a compiler error because there would be two candidate implementations.

Because nothing in Rust is identifier-based. Unlike python, all syntax magic (even the ? operator) relies on traits defined by `core`.

    for x in y {

desugars to

    let mut iter = IntoIterator::into_iter(y);
    while let Some(x) = y.next() {

and

    x?

desugars to

    match x {
        Ok(x) => x,
        Err(e) => {
            return Err(From::from(e));
        }
    }

and

    x + y

desugars to

    core::ops::Add::add(x, y)

etc.

All of those traits are expected to live explicitly in the core crate at well known paths. Otherwise you'd be writing methods with absolutely no idea how the language would interact with it. And if you had a Set type implement `add`, it'd have to accept exactly 2 arguments to be compatible with the language's `add` or something equally as unergonomic.

It's traits all the way down! There'd be no explanation needed because it'd be antithetical and contradictory to traits to begin with. Once one understands how traits are intended to be used, the explanation for why there aren't identifier based resolution semantics becomes obvious.

[demurgos]

> Strictly speaking, Rust doesn't support overloaded functions. Function overloading is when you define the same function but with different arguments, and the language selects the correct one based on the argument type. In this case, it's two different implementations of a trait for two different types.

You're right that there is no function overload in this article, just some implicit derefs.

I would argue however that Rust has overloaded functions (function resolution based on argument types) due to how it handles trait resolution with implicit inference. Rust may not have syntax sugar to easily define overloads and people generally try to avoid them, but using argument-based dispatch is extremely common. The most famous example is probably `MyType::from(...)`, but any single-method generic trait using the generics for the method arguments is equivalent to function overloading. There are also other techniques. Using nightly features you can get far enough so a consumer can use native function call syntax.

Overload on nightly: https://play.rust-lang.org/?version=nightly&mode=debug&editi...

Overload on stable: https://play.rust-lang.org/?version=stable&mode=debug&editio...

The mechanism and syntax may be different from overloading in C++ or Java, but as a user the result is the same and it causes the same pain points.

[kibwen]

Re: function overloading, Rust doesn't support overloaded functions in the same manner as Java or C++, which is important to emphasize to people coming from those languages.

However, if you then show those people that this is legal Rust:

    let addr1 = Ipv4Addr::from([13, 12, 11, 10]);
    let addr2 = Ipv4Addr::from(218893066);

...that also runs the risk of confusing them, because this is indistinguishable from function overloading, because it basically is function overloading.

I'm not actually sure what to call this, because unlike Java-style overloading, Rust isn't ad-hoc; everything it's doing is fundamentally integrated into the type system. But unlike strict parametric polymorphism (and like Java), both of the above functions have totally distinct implementations.

[Animats]

This is pretty good.

A useful way to think about this:

- All data in Rust has exactly one owner.

- If you need some kind of multiple ownership, you have to make the owner be a reference-counted cell, such as Rc or Arc.

- All data can be accessed by one reader/writer, or N readers, but not both at the same time.

- There is both compile time and run time machinery to strictly enforce this.

Once you get that, you can see what the borrow checker is trying to do for you.

[diath]

The first code snippet, which is as simple as it gets, perfectly illustrates why Rust is extremely annoying to work with. I understand why you need the into_iter bit and why the borrow checker complains about it, but the fact that even the simplest "for x in y" loop already makes you wrestle the compiler is just poor ergonomics.

[0xdeafbeef]

#include <vector> #include <iostream>

  int main() {
      std::vector<int> vec = {1, 2, 3};
      vec.reserve(3);

      std::cout << "Initial capacity: " << vec.capacity() << std::endl;
      std::cout << "Initial data address: " << (void*)vec.data() << std::endl;

      int* ptr = vec.data();

      std::cout << "Pointer before push_back: " << (void*)ptr << std::endl;
      std::cout << "Value via pointer before push_back: " << *ptr << std::endl;

      std::cout << "\nPushing back 4...\n" << std::endl;
      vec.push_back(4);

      std::cout << "New capacity: " << vec.capacity() << std::endl;
      std::cout << "New data address: " << (void*)vec.data() << std::endl;

      std::cout << "\nAttempting to access data via the old pointer..." << std::endl;
      std::cout << "Old pointer value: " << (void*)ptr << std::endl;
      int value = *ptr;
      std::cout << "Read from dangling pointer (UB): " << value << std::endl;

      return 0;
  }

./a.out Initial capacity: 3 Initial data address: 0x517d2b0 Pointer before push_back: 0x517d2b0 Value via pointer before push_back: 1

Pushing back 4...

New capacity: 6 New data address: 0x517d6e0

Attempting to access data via the old pointer... Old pointer value: 0x517d2b0 Read from dangling pointer (UB): 20861

[throwawaymaths]

i think really the problem is not the compiler so much but that you need to be thinking about what `for` desugars to.

[masklinn]

> I understand why you need the into_iter bit and why the borrow checker complains about it

The borrow checker is not really involved in the first snippet (in fact the solution involves borrowing). The compiler literally just prevents a UAF.

[baq]

> wrestle the compiler

This is quite literally a skill issue, no offense.

'wrestle the compiler' implies you think you know better; this is usually not the case and the compiler is here to tell you about it. It's annoying to be bad at tracking ownership and guess what: most people are. The ones who aren't have decades of experience in C/C++ and employ much the same techniques that Rust guides you towards. If you really know better, there are ways to get around the compiler. They're verbose and marked unsafe to 1) discourage you from doing that and 2) warn others to be extra careful here.

If this is all unnecessary for you - and I want to underscore I agree it should be in most software development work - stick to GC languages. With some elbow grease they can be made as performant as low level languages if you write the critical parts in a way you'd have to do it in Rust and will be free to write the rest in a way which doesn't require years of experience tracking ownership manually. (Note it won't hurt to be tracking ownership anyway, it's just much less of an issue if you have to put a weakref somewhere once a couple years vs be aware at all times of what owns what.)

[teleforce]

We'd expect in the 21st century we have programming languages that have impeccable GC for automatic memory management rather than forcing programmer to wrestling and fighting for manually managing the memory.

Auto industry kind of solved this automation mechanism for example with the new high performance Toyota GR Corolla has a new automatic gear transmission that's proven as fast if not faster than the manual version [1]. The same goes to F1, the epitome of car racing performance.

[1] 2025 Toyota GR Corolla's New Automatic Gearbox Democratizes Fun:

https://www.caranddriver.com/reviews/a62672128/2025-toyota-g...

[rollcat]

I'm starting to think Zig's strategy to memory management is generally friendlier to a developer. If a function needs to allocate memory, it must take an allocator as a parameter. If it needs scratch space to perform computation, it can use that allocator to create an arena for itself, then free it up before it returns (defer). If it returns a pointer, the caller should assume the object was allocated using that allocator, and becomes the owner. It may still be unclear what happens to a pointer if it's passed as a parameter to another function, but I'd normally consider that a borrow.

It's a lot of assumptions, and if you trip, Rust will yell at you much more often than Zig; and it will likely be right to do so. But in all seriousness, I'm tired of the yelling, and find Zig much more pleasant.

[cornholio]

A 20 page document on how to use basic variables, function calls and methods. Except for the threading paragraph, which is hard in any language, this is all complexity and refactoring pain that Rust hoists onto every programmer every day, for relatively modest benefits, somewhat improved performance and memory usage vs the garbage collected/ref-counted version of the same code.

Essentially, you wouldn't and shouldn't make that tradeoff for anything other than system programming.

[baq]

> this is all complexity and refactoring pain that Rust hoists onto every programmer every day

This is what you should be doing when working with C/C++, except there is no compiler to call you names there if you don’t.

If you’re saying ‘use a GC language unless requirements are strict about it’, yeah hard to disagree.

[bsaul]

I see why you're saying that, and i almost entirely agree with you. However, i would say that if all you're doing is glueing calls to third party systems (like what most backend code is), then you won't fall into complex lifetime problems anyway, and the experience will remain quite pleasant.

Another point, is that the rust ecosystem is absolutely insanely good ( i've recently worked with uniffi and wasmbindgen, and those are 5 years ahead of anything else i've seen..)

[Surac]

I really like the text. Giving more light to the memory management of rust will help me understand more of the language. I still think some concepts of rust are over verbose but I slowly understand the hype around rust. I myself use C or C++ but I will „borrow“ some of the rust ideas to make my code even more robust

[ultimaweapon]

I'm coming from C++ now I don't want to use C++ anymore. When C++ was still my primary language I always frustrated with some of its feature like non-destructive move, copy by default and dangling references then I found Rust fixed all of those problems. At the beginning I very frustrated with Rust because the borrow checker prevent me from doing what I usually do in C++ but I keep going.

[dmitrygr]

> So why does this result in a move and why does replacing x with &x fix it

Precisely the sort of question I do not want to waste time on.

[bsaul]

"If we change .set_value() to take a &self instead of a &self"

guess it's "instead of a &mut self"

[sidcool]

This is brilliantly written n

[bnjms]

Im sure it is. I don’t know how to program but read most of part 4 when it appeared here last.

[gitroom]

been banging my head against this same stuff trying to learn rust - honestly memory rules make me miss how easy c feels sometimes, but i'm sticking with it cuz i want fewer bugs

[pjc50]

All of the same rules exist in C, they're just tracked manually inside the programmer's head and you don't find out about mistakes until much later.