top | item 46507956

(no title)

shatsky | 1 month ago

And yet Rust ecosystem practically killed runtime library sharing, didn't it? With this mentality that every program is not a building block of larger system to be used by maintainers but a final product, and is statically linked with concrete dependency versions specified at development time. And then even multiple worker processes of same app can't share common code in memory like this lib, or ui toolkit, multimedia decoders, etc., right?

PS. Actually I'll risk to share my (I'm new to Rust) thoughts about it: https://shatsky.github.io/notes/2025-12-22_runtime-code-shar...

discuss

order

RadiozRadioz|1 month ago

As a user and developer, runtime is my least favourite place for dependency and library errors to occur. I can't even begin to count the hours, days, I've spent satisfying runtime dependencies of programs. Cannot load library X, fix it, then cannot load library Y, fix it, then library Z is the wrong version, then a glibc mismatch for good measure, repeat.

I'd give a gig of my memory to never have to deal with that again.

no_wizard|1 month ago

if I recall correctly Rust does not support any form of dynamic linking or library loading.

Most of the community I’ve interacted with are big on either embedding a scripting engine or WASM. Lots of momentum on WASM based plugins for stuff.

It’s a weakness for both Rust and Go if I recall correctly

SkiFire13|1 month ago

> if I recall correctly Rust does not support any form of dynamic linking or library loading.

Rust supports two kinds of dynamic linking:

- `dylib` crate types create dynamic libraries that use the Rust ABI. They are only usesul within a single project though, since they are only guaranteed to work with the crate that depended on them at the compilation time.

- `cdylib` crate types with exported `extern "C"` functions; this creates a typical shared library in the C way, but you also need to implement the whole interface in a C-like unsafe subset of Rust.

Neither is ideal, but if you really want to write a shared library you can do it, it's just not a great experience. This is part of the reason why it's often preferred to use scripting languages or WASM (the other reason being that scripting languages and WASM are sandboxed and hence more secure by default).

I also want to note that a common misconception seems to be that Rust should allow any crate to be compiled to a shared library. This is not possible for a series of technical reasons, and whatever solution will be found will have to somehow distinguish "source only" crates from those that will be compilable as shared libraries, similarly to how C++ has header-only libraries.

shatsky|1 month ago

It does support dynamic libs, but virtually all important Rust software seems to be written without any consideration for it.

PunchyHamster|1 month ago

Go supports plugins (essentially libraries) but its has a bunch of caveats. You can also

You can also link to C libs from both. I guess you could technically make a rust lib with C interface and load it from rust but that's obviously suboptimal

oooyay|1 month ago

Go definitely supports dynamic libraries

pezezin|1 month ago

In any modern OS with CoW forking/paging, multiple worker processes of the same app will share code segments by default.

rollcat|1 month ago

COW on fork has been a given for decades.

You can't COW two different libraries, even if the libraries in question share the source code text.

Groxx|1 month ago

Not really? You just need to define the stable ABI: you do that via `[repr(C)]` and other FFI stuff that has been around since essentially the beginning. Then it handles it just fine, for both the code using a runtime library and for writing those runtime libraries.

People writing Rust generally prefer to stay within Rust though, because FFI gives up a lot of safety (normally) and is an optimization boundary (for most purposes). And those are two major reasons people choose Rust in the first place. So yeah, most code is just statically compiled in. It's easier to build (like in all languages) and is generally preferred unless there's a reason to make it dynamic.

terafo|1 month ago

Dynamic libraries are a dumpster fire with how they are implemented right now, and I'd really prefer everything to be statically linked. But ideally, I'd like to see exploration of a hybrid solution, where library code is tagged inside a binary, so if the OS detects that multiple applications are using the same version of a library, it's not duplicated in RAM. Such a design would also allow for libraries to be updated if absolutely necessary, either by runtime or some kind of package manager.

vlovich123|1 month ago

OSes already typically look for duplicated code pages as opportunities to dedupe. It doesn’t need to be special cases for code pages because it’ll also find runtime heap duplicates that seem to be read only (eg your JS code JIT pages shared between sites).

One challenge will be that the likelihood of two random binaries having generated the same code pages for a given source library (even if pinned to the exact source) can be limited by linker and compiler options (eg dead code stripping, optimization setting differences, LTO, PGO etc).

The benefit of sharing libraries is generally limited unless you’re using a library that nearly every binary may end up linking which has decreased in probability as the software ecosystem has gotten more varied and complex.

shatsky|1 month ago

I believe NixOS-like "build time binding" is the answer. Especially with Rust "if it compiles, it works". Software shares code in form of libraries, but any set of installed software built against some concrete version of lib which it depends on will use this concrete version forever (until update replaces it with new builds which are built against different concrete version of lib).

johncolanduoni|1 month ago

The system you’re proposing wouldn’t work, because without additional effort in the compiler and linker (which AFAIK doesn’t exist) there won’t be perfectly identical pages for the same static library linked into the same executable. And once you can update them independently, you have all the drawbacks of dynamic libraries again.

Outside of embedded, this kind of reuse is a very marginal memory savings for the overall system to begin with. The key benefit of dynamic libraries for a system with gigabytes of RAM is that you can update a common dependency (e.g. OpenSSL) without redownloading every binary on your system.

littlestymaar|1 month ago

I wish the standard way of using shared libraries would be to ship the .so the programs want to dynamically link to alongside the program binary (using RUNPATH), instead of expecting them to exist globally (yes, I mean all shared libraries even glibc, first and foremost glibc, actually).

This way we'd have no portability issue, same benefit as with static linking except it works with glibc out of the box instead of requiring to use musl, and we could benefit from filesystem-level deduplication (with btrfs) to save disk space and memory.

SkiFire13|1 month ago

What you're describing is not static linking, it's embedding a dynamically linked library in another binary.

tliltocatl|1 month ago

IMHO dynamic libraries are a dumpster fire because they are often used as a method to provide external interfaces, rather then just share common code.

lmm|1 month ago

> And yet Rust ecosystem practically killed runtime library sharing, didn't it?

Yes, it did. We have literally millions of times as much memory as in 1970 but far less than millions of times as many good library developers, so this is probably the right tradeoff.

VorpalWay|1 month ago

C++ already killed it: templated code is only instantiated where it is used, so with C++ it is a random mix of what goes into the separate shared library and what goes into the application using the library. This makes ABI compatibility incredibly fragile in practise.

And increasingly, many C++ libraries are header only, meaning they are always statically linked.

Haskell (or GHC at least) is also in a similar situation to Rust as I understand it: no stable ABI. (But I'm not an expert in Haskell, so I could be wrong.)

C is really the outlier here.

BobbyTables2|1 month ago

Static linking is still better than shipping a whole container for one app. (Which we also seem to do a lot these days!)

It still boggles my mind that Adobe Acrobat Reader is now larger than Encarta 95… Hell, it’s probably bigger than all of Windows 95!

speed_spread|1 month ago

It's not just about memory. I'd like to have a stable Rust ABI to make safe plugin systems. Large binaries could also be broken down into dynamic libraries and make rebuilds much faster at the cost of leaving some optimizations on the table. This could be done today with a semi stable versionned ABI. New app builds would be able to load older libraries.

The main problem with dynamic libraries is when they're shared at the system level. That we can do away with. But they're still very useful at the app level.

goodpoint|1 month ago

It's really bad for security.