[ahupp]

It’s not like this is unique to rust; you see similar issues with node and python. Distributions have many jobs, but one was solving the lack of package management in C. Now that every modern language a package manager, trying to apply the C package management philosophy is untenable. Specifically, the idea of a single version, globally installed, and producing distro packages for every language specific packages.

[llm_trw]

Apart from the fact building it the 'non-C way' results in a magic meat package that you have no idea what it contains: https://hpc.guix.info/blog/2021/09/whats-in-a-package/

Guix is also a distro that allows for any number of versions of the same package globally, something that language specific dependancy managers do not.

Distors are there for a reason, and anyone who doesn't understand that reason is just another contributor to the ongoing collapse of the tower of abstractions we've built.

[kpcyrd]

This is outdated information. Debian (and other distros) already had their own SBOM format called buildinfo files that encodes this kind of information.

In Debian stable ripgrep on amd64 is currently on version 13.0.0-4+b2.

The relevant buildinfo file can be found here:

https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/...

It encodes the entire Rust dependency graph that was used for this binary, with exact versions of each crate.

[tcfhgj]

> Distors are there for a reason

for me: make an os out of the kernel

[ahupp]

Using language-native packaging doesn't imply that you have to use binaries from wherever. In the pytorch example you can still build it as a regular part of the distribution, using the C++ dependencies/toolchain, it just means you don't try to stuff it into a versioning/distribution/install model that doesn't match the languages expectations.

[pjmlp]

Making an userspace to Linux kernel, and sorting out disagreements by creating yet another fork.

[XorNot]

Except from a management and maintenance perspective...this is a nightmare. When a security vulnerability drops somewhere, everywhere needs to be patched ASAP.

Distros (and the people who run most scales of IT org) want to be able to deploy and verify that the fix is in place - and its a huge advantage if it's a linked library that you can just deploy an upgrade for.

But if it's tons and tons of monolithic binaries, then the problem goes viral - every single one has to be recompiled, redeployed etc. And frequently at the cost of "are you only compatible with this specific revision, or was it just really easy to put that in?"

It's worth noting that docker and friends also while still suffering from this problem, don't quite suffer from it in the same way - they're shipping entire dynamically linked environments, so while not as automatic, being able to simply scan for and replace the library you know is bad is a heck of a lot easier then recompiling a statically linked exe.

People are okay with really specific dependencies when it's part of the business critical application they're supporting - i.e. the nodejs or python app which runs the business, that can do anything it wants we'll keep it running no matter what. Having this happen to the underlying distributions though?

(of note: I've run into this issue with Go - love the static deploys, but if someone finds a vulnerability in the TLS stack of Go suddenly we're rushing out rebuilds).

[ahupp]

This is conflating static linking with how the distribution handles updates. If a language is always statically linking dependencies (like Go or Rust), the distribution will have to rebuild everything that depends on a patched package whether or not they are using the language's native tools or some import into the distro package system.

What I'm specifically suggesting is:

  * Distributions package *binaries*, but not the individual libraries that those binaries depend on.
  * Distributions mirror all dependencies, so that you can (in principle) have a completely offline copy of everything that goes into the distribution.  Installing a binary uses the language-specific install tools to pull dependencies, targeting the distribution's mirror.  
  * Enough dependency tracking to know what needs to be rebuilt if there's a security update.
  * Any outside dependencies (e.g openssl) will continue to depend on whatever the distribution packages.
  * Dependencies are not globally installed, but use whatever isolation facilities the language has (so e.g, a venv for python, whatever npm does)

[rat87]

Why is this an issue? Simply recompile and download each package. If the distro worries that the maintainers would take too low just fork and recompile the packages themselves. These days its really not that big of a problem in terms of disk space or network traffic. And if some packages are large its often because of images resources which can be packaged separately. It seems like a lot less effort then trying to guess if dynamicly linked library will work with every package in every case after the update.

[curt15]

>When a security vulnerability drops somewhere, everywhere needs to be patched ASAP.

Is that ultimately the responsibility of the application developer or the OS developer?

[rlpb]

> Now that every modern language a package manager...

...they fail to integrate with dependencies written in any other language.

It's fine if you just want to sit a monoculture language software stack on top of a multilingual base platform. You can't make a functional system with one language alone, yet those who criticise distribution packaging architecture do so while simultaneously depending on this ability that language-specific package managers do not have. There is no viable alternative today. Most critics think they understand the general problem but only have narrow practical experience, so end up believing that their solution is superior while not considering the general multilingual software supply problem.

Nix isn't a solution either, because in the general case Nix isn't security-supporting arbitrary and multiple dependency versions either.