I love these re-implementations of things. Technology advances by the continued reinvention of the wheel, and even if some efforts end up being of merely didactic interest, it is still important to make them. Coreutils is one of the pillars of our civilization, and thus it should be re-implemented several times on many programming languages.
Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things.
An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users. For example, when I use GNU "ls", I know that I can always look at its source code and change it to my whim (or hire a programmer to change it for me). However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code. I suppose that the removal of this freedom is accidental, because it was a nice thing.
> Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things.
It's not just a matter of having all the individual tools, it's a matter of having them be functionally complete and behaving either the same as GNU or POSIX - note the graph part way down the readme that talks about having only slightly better than half of the tests passing. And the tests probably aren't truly comprehensive, especially for things that people didn't realize were implementation specific or non-guaranteed behaviors. It's good to see them making progress, I just want to someone temper the idea that they're close to done.
"However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code."
Solution: Do not use "re-distributions" by middlemen.
For example, when I use BSD UNIX I get it directly from one of the two original BSD projects. I compile everything from source.
> An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users.
Copyleft is very nice but I think the competition that liberally licensed software means to proprietary alternatives is way higher than the competition that liberally licensed software means to copylefted software. Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. This code base will always exist as a risk to the GPL licensed coreutils package. If there is a free (as in money) alternative however, the proprietary vendor will have a much harder time to convince some company to buy their product.
I think copyleft shines the most when there is a lot of competition with proprietary solutions. Without copyleft, the competing solutions could just incorporate improvements of some package into their own. With copyleft, the copylefted software package has a chance to fight. But in a domain where Free Software has won, and I would consider coreutils to be such an environment, one can also have open source software.
I am not so sure it is accidental. Many people prefer BSD, MIT, or Apache style licenses. There is open debate which is more or less free ( them or GPL ).
In practice, when the Open Source distribution is the dominant implementation, the source tends to be available and modifiable even when the source is from a for profit company. One might look at all the .NET stuff from Microsoft as an example. Clang and LLVM would be other great examples. For the BSD license, the operating system family it was designed for has been available as Open Source since before the GPL has existed. You can still get their equivalent of the core utils in source form today. In fact, there is a Linux distribution going Alpha very soon ( Chimera ) that is going to use them.
If most distributions were to adopt Uutils ( this Rust reimplementation ), I fully expect that not only would the source remain available for them but that there would remain essentially a single version that they would all share. I do not think that the GPL is require to maintain that momentum.
The big difference is that some commercial company could take this and extend it as their own as you say. In that case though, they would likely just have gone their own way anyway which would mean less sharing, not more. You could argue that MacOS for example is what is wrong with BSD as they took some BSD stuff and have not shared everything back. They have shared some stuff though, likely more than if there was no BSD code to borrow. With the BSD code, it seems very unlikely that MacOS would have used GPL.
Anyway, it is all just opinions and preferences. I may not be any more right than you. Who knows? What you are saying though is not 100% obviously right to me either.
>if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code
You are still free to go and work off of the upstream version of uutils.
Nixpkgs experimented with a build environment based on uutils-coreutils instead of GNU coreutils[0]. A lot of bugs or unimplemented features were uncovered which were fixed upstream and as a result software such as chromium, vim, emacs or even rustc itself builds properly now.[1] So it definitely could be a viable alternative in the future.
I'm always cautious about who I recommend Nix to, but I usually say that one of the best things waiting on the other side of the learning curve is what an incredible always-improving lever nix and nixpkgs can be for just about any task that involves or benefits from ~integrating a lot of software.
But it's also hard to communicate what I mean by that without diving into a bunch of detail on some case where it proved useful to me.
The ability to give uutils a shake at scale is a paragon.
At some point I imagine it'll be able help ferret out a good fraction of remaining long-tail issues with oilshell's mostly-bash-compatible `osh` in short order.
The maintainers of that project have said that a lot of the code was written by new Rust programmers, so some of the code could probably use some help from experienced systems programmers.
I did a little cleanup on their `dd` tool a while ago[1], and I would be pretty cautious about actually relying on it anytime soon.
Lots of cool details about stuff like the gradual migration of the kernel from 100% assembly C and the origin stories of a lot of these ubiquitous utility programs like cp, ls, etc.
I’m working on a new set of file tools that replace touch, mkdir, mv, cp and rm, in Rust. They are intended to have better defaults for using them interactively. Upvote if you’re interested (or shoot me an email/tweet), I’ll comment here when v1 is ready.
For an alternative implementation that aims to be compatible with POSIX (not necessarily compatible with GNU coreutils), see: https://github.com/GrayJack/coreutils
What's the status of upstream GNU coreutils' attitude on incrementally rewriting in a safe language like rust? Isn't one of the selling points of Rust that it can incrementally be adopted in a C source tree?
[+] [-] enriquto|3 years ago|reply
I love these re-implementations of things. Technology advances by the continued reinvention of the wheel, and even if some efforts end up being of merely didactic interest, it is still important to make them. Coreutils is one of the pillars of our civilization, and thus it should be re-implemented several times on many programming languages.
Looking at the source code, it is impressive that this re-implementation is essentially complete! Look at this: https://github.com/uutils/coreutils/tree/main/src/uu It's only missing some obscure and fringe things.
An important feature of GNU coreutils is, somewhat sadly, not reproduced by this re-implementation: the copyleft license of the original implementation. This takes out some freedom of the users. For example, when I use GNU "ls", I know that I can always look at its source code and change it to my whim (or hire a programmer to change it for me). However, if I use uutils "ls", this is not necessarily the case, for it may be a re-distribution by some middle-men that has modified it slightly and does not provide the source code. I suppose that the removal of this freedom is accidental, because it was a nice thing.
[+] [-] yjftsjthsd-h|3 years ago|reply
It's not just a matter of having all the individual tools, it's a matter of having them be functionally complete and behaving either the same as GNU or POSIX - note the graph part way down the readme that talks about having only slightly better than half of the tests passing. And the tests probably aren't truly comprehensive, especially for things that people didn't realize were implementation specific or non-guaranteed behaviors. It's good to see them making progress, I just want to someone temper the idea that they're close to done.
[+] [-] 1vuio0pswjnm7|3 years ago|reply
Solution: Do not use "re-distributions" by middlemen.
For example, when I use BSD UNIX I get it directly from one of the two original BSD projects. I compile everything from source.
NB. I use both Linux and BSD.
[+] [-] codetrotter|3 years ago|reply
Not necessarily. Some people prefer the MIT/BSD/ISC family of licenses over GPL.
[+] [-] est31|3 years ago|reply
Copyleft is very nice but I think the competition that liberally licensed software means to proprietary alternatives is way higher than the competition that liberally licensed software means to copylefted software. Meaning, if there is GNU coreutils around but no liberally licensed alternative, there will be a market for alternatives that proprietary vendors can use. Thus, somewhere out there will maintain a proprietary coreutils library and charge money for it. This code base will always exist as a risk to the GPL licensed coreutils package. If there is a free (as in money) alternative however, the proprietary vendor will have a much harder time to convince some company to buy their product.
I think copyleft shines the most when there is a lot of competition with proprietary solutions. Without copyleft, the competing solutions could just incorporate improvements of some package into their own. With copyleft, the copylefted software package has a chance to fight. But in a domain where Free Software has won, and I would consider coreutils to be such an environment, one can also have open source software.
[+] [-] LeFantome|3 years ago|reply
In practice, when the Open Source distribution is the dominant implementation, the source tends to be available and modifiable even when the source is from a for profit company. One might look at all the .NET stuff from Microsoft as an example. Clang and LLVM would be other great examples. For the BSD license, the operating system family it was designed for has been available as Open Source since before the GPL has existed. You can still get their equivalent of the core utils in source form today. In fact, there is a Linux distribution going Alpha very soon ( Chimera ) that is going to use them.
If most distributions were to adopt Uutils ( this Rust reimplementation ), I fully expect that not only would the source remain available for them but that there would remain essentially a single version that they would all share. I do not think that the GPL is require to maintain that momentum.
The big difference is that some commercial company could take this and extend it as their own as you say. In that case though, they would likely just have gone their own way anyway which would mean less sharing, not more. You could argue that MacOS for example is what is wrong with BSD as they took some BSD stuff and have not shared everything back. They have shared some stuff though, likely more than if there was no BSD code to borrow. With the BSD code, it seems very unlikely that MacOS would have used GPL.
Anyway, it is all just opinions and preferences. I may not be any more right than you. Who knows? What you are saying though is not 100% obviously right to me either.
[+] [-] charcircuit|3 years ago|reply
You are still free to go and work off of the upstream version of uutils.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] Gigachad|3 years ago|reply
Copyleft fanatics feel a lot like libertarians thinking they need to bring a utility belt full of weapons for their trip to the supermarket.
[+] [-] siraben|3 years ago|reply
[0] https://github.com/NixOS/nixpkgs/pull/116274
[1] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-85...
[+] [-] abathur|3 years ago|reply
But it's also hard to communicate what I mean by that without diving into a bunch of detail on some case where it proved useful to me.
The ability to give uutils a shake at scale is a paragon.
At some point I imagine it'll be able help ferret out a good fraction of remaining long-tail issues with oilshell's mostly-bash-compatible `osh` in short order.
[+] [-] Arcuru|3 years ago|reply
I did a little cleanup on their `dd` tool a while ago[1], and I would be pretty cautious about actually relying on it anytime soon.
[1] https://jackson.dev/post/rust-coreutils-dd/
[+] [-] danudey|3 years ago|reply
[+] [-] doomrobo|3 years ago|reply
[+] [-] Ameo|3 years ago|reply
https://www.cs.dartmouth.edu/~doug/reader.pdf
Lots of cool details about stuff like the gradual migration of the kernel from 100% assembly C and the origin stories of a lot of these ubiquitous utility programs like cp, ls, etc.
[+] [-] matt_daemon|3 years ago|reply
[+] [-] xixixao|3 years ago|reply
[+] [-] zajio1am|3 years ago|reply
[+] [-] throwbadubadu|3 years ago|reply
[+] [-] nmitchko|3 years ago|reply
[+] [-] Smaug123|3 years ago|reply
[+] [-] googlryas|3 years ago|reply
[+] [-] silasdb|3 years ago|reply
[+] [-] sophacles|3 years ago|reply
[+] [-] pengaru|3 years ago|reply
What's the status of upstream GNU coreutils' attitude on incrementally rewriting in a safe language like rust? Isn't one of the selling points of Rust that it can incrementally be adopted in a C source tree?
[+] [-] slackfan|3 years ago|reply
[deleted]
[+] [-] LinuxBender|3 years ago|reply
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] bitwize|3 years ago|reply
The Rust Evangelism Strike Force is coming for your children -- and the foundation of the Linux userland.
Speaker anagram: TUSSLED REVELRY
Speaker affiliation: Mozilla (business model: Uber for not working on browsers)