First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager.
Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager.
Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first.
Fourth, finally, we get to install a simple Unix program, that any package manager could have provided.
For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic.
And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers.
Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
First, dependency manager vs package manager. Potayto potatoh.
Second. The reason I recommend direnv is because of ergonomics, so you don't have to remember to reload your shell. Most current version managers either do it themselves through shell hooks or ask you to install direnv. Point taken on using brew: I've changed it to use nix-env.
Third. Yep. Sort of like having to learn English before you can learn programming.
Fourth. As they say: start with the basics.
Fifth. No other package manager can freeze the complete dependency set you are using AND allow you to revert it exactly when things don't work. That's the power of declarative, reproducible builds: you try to update and it doesn't work? No problem, revert to the last working version and things are all good again.
Sixth. Ever tried updating a project using a Dockerfile that uses an old version of ubuntu? Then you can't find the packages anymore or no is building them anymore for the new version of Ubuntu and you have to pull random PPAs from the Internet to get things to work? In Nix, you can keep two pinned nixpkgs versions at the same time. In this way you can keep working on upgrading your project one working step at the time.
So yeah, it's not just cHeCKmATe FP-atheists sort of stuff.
> Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
As a Gentoo user and someone who has tried and grokked Nix/Guix, this is not true at all. Gentoo is essentially a system for creating distributions, a meta-distro if you like. It just so happens to be something that appeals to someone like me to run at home because, well, why not? I know exactly what I need/want on my computer and this lets me build my own custom distro tailored for me.
Nix/Guix is something completely different. It addresses real problems that arise in a variety of places in an elegant way. Containers are another solution to some of these problems, but come with their own tradeoffs.
Your blanket statement about "professionals" makes no sense. I don't use Gentoo at work because it's not my job to maintain custom Linux systems or create distros. But guess what? Not everyone has the same job. You don't speak for all "professionals". I can distinctly remember around 2007 people saying this exact thing about git. I was told geeks were only learning it to make themselves feel clever and that SVN was fine. Well look how that turned out. I'm glad I ignored those people.
It's not about what is possible, it is about ergonomics. We could have used email for conversations online, yet we use Slack. Other tool require lots of thought and carefull execution for what Nix gives you for free.
> First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager.
This is not true. The defining point about Nix is that it allows to define all dependencies explicitly. Every derivation in it is built inside of a sandbox with no access to network and restricted filesystem. This ensures that nothing is accidentally missed. Package management is just one of features.
> Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager.
direnv is not needed to use Nix, it's just a nice add-on. It's like you don't need Ranger to navigate in a file system, but it is nice. Besides, you can install direnv through nix, I use it this way. Even nix is installed by itself as well.
> Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first.
Well, yes you do, but language is actually quite simple it's the nixpkgs (repo containing all packages) that's quite complex. Nix's purely functional, lazily evaluated properties exactly fit as a language describing dependencies.
> Fourth, finally, we get to install a simple Unix program, that any package manager could have provided.
If you were trying to install a package, you don't need to learn anything, you just install it (for example to install ranger you just run nix-env -iA nixpkgs.ranger). Nix language is only needed if you want to create a new package.
> For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic.
> And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers.
> Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
Nix delivers what docker promised and ultimately failed to deliver. Docker promised to reproduce developer's environment to production. What it did was to zip developer's computer and deploy it. When docker got adopted, relying on images was impractical, so for deployment Dockerfile was used, but that file is not much different than a shell script.
Nix instead describes the entire dependency tree down to libc. Because the starting state and all dependencies are known it can always create the same result, that's the biggest selling point of Nix to me.
> My ideal dependency manager would allow me to specify each and every dependency that is required to work on my projects. It should be easily reproducible, declarative and easy to upgrade.
To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much.
For example, a new project that was created with "create-react-app" takes 181 MB of space on disk and has 35'894 files in it. That includes about 1467 modules, all for a relatively simple web application. It doesn't matter if you're using package.json/package.lock, or any other tool or solution out there - with that amount of dependencies you're simply not doing dependency "management" of any kind.
I'd argue that if you have >100 dependencies in your project, it's probably too big, unless you have a team that's dedicated to managing and auditing all of them. Of course, no one actually audits their dependencies when faced with such large numbers, and so what's left for most developers is to just trust what's out there.
You're not wrong, but that's a different problem. Nix makes sure that you get a predictable and reproducible tree of dependencies, and allows different applications to depend on different versions of the same dependencies. That is, it's a solution to DLL hell. It's solid engineering based on solid theory, and it really does let you manage configuration with a level of reliability that most other package managers only pretend to have.
Now auditing dependencies, knowing that the packages you depend on aren't malicious and have no known vulnerabilities... well, that's a whole separate problem. And yeah, we don't really have a solution to that right now. The best we can do, as you say, is keep the attack surface small.
But if we did try to solve the auditing problem, the solution would have to sit on top of nix or something like it. If you can't precisely specify a dependency graph and reliably install from that specification, it doesn't matter how good your auditing is or what sort of system of trust you can create. You don't know what you're getting anyway.
I don't have an issue with the size of a project. On the other hand, I worry about a large number of dependencies and the need to understand the security and compatibility models of their respective authors.
I'm fine to depend on, say, JVM or Qt - large projects. Not with thousands of small packages developed by thousands of independent developers.
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing.
But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions.
Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem.
> Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem.
There a large number of developers around now who've never done traditional (RPM, DEB, SysV) packaging and don't understand it. So while the large distros like Redhat and Debian push on with it, the development community only sees Dockerfiles, Snaps, Flatpaks etc.
One thing I like with Nix is that it can build containers, so it's totally compatible with deployment systems that require containers. I don't mean using Nix commands in a Dockerfile, I mean having a Nix derivation that produces a container.
I've been writing normal Dockerfiles at work recently and the whole toolchain is annoying on multiple fronts; I'd love to jump to Nix instead, except it's really hard to convince other people to learn non-trivial technology that's off the beaten path :(.
Arrived expecting to vehemently disagree, given how much I rely on version managers for everything (Ruby, Node, etc.) in my work. Came away thinking, yeah, okay, Nix probably is a better solution to all of this and more.
Managing dependencies is a big problem, and I feel like we've given up on solving it directly, and instead built workarounds.
If we question our assumptions, the first question is: Why do we need multiple Ruby versions at all? Why isn't the latest version of Ruby sufficient? Well, obviously, Ruby's behavior has changed over time. But why isn't it backward-compatible? Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter?
Okay another one. "Nokogiri" is the Ruby gem (library) for parsing XML (including HTML). You always need to remember to install libxml when you install it. Why? Why doesn't Nokogiri include an XML parser? Because it would be slow? You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Also, what if I just need some quick XML parsing and don't care if the parsing is 1000x slower? Can I just get Nokogiri with the Ruby-implemented parser? No? Why not?
Then every single Ruby gem out there starts using newer Ruby features and thus, you must update your Ruby version. Why can't library authors gracefully handle older versions of Ruby?
This just adds burden to programming language and library authors. Sure I could almost certainly support Python 2.7 and Python < 3.6 in all my libraries, but it introduces more code and shims I've got to maintain. Given I'm not being paid for this work and do it for fun, there's no compelling reason to support the small subset of users that cannot or will not update.
I imagine variations of the above apply to nokogiri and some amount of Ruby core developers.
> These dependencies are met by default by Nokogiri's packaged versions of the libxml2 and libxslt source code, but a configuration option --use-system-libraries is provided to allow specification of alternative library locations.
Some authors work hard to have their tools do the right thing and consistently.
> Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter?
I suspect it would be a nightmare - it's not just small parts of the code, it would be other parts of the code that depend on it.
Take for instance Python 2 & Python 3's approach to if ("hello" > "goodbye") - in Python 2 that would be False and in Python 3 that would be TypeError. So if you are running Python 3 with a Python 2.7 flag, how should Python's internal modules that are written in Python 3 handle this? Do all Python's internal modules need to work on every legacy version of Python and load the correct version, or does the interpreter need to somehow load up instructions for both and parse them together? Then there is the whole problem about libraries - are these allowed to selectively choose which version of python they run on?
This is of course only one small example of the complexity - If it was easy to make these things backwards compatible it would be great, but I would expect the cost for implementing full backwards compatibility to be high and the language authors are probably motivated to make people move towards the latest versions.
Even if you can somehow convince the major tooling to take that route, you're never going to convince every piece of software to follow suite. It's a problem that must be tackled, there's no side stepping it.
> You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message.
Promptly ignored by most people, all while doubling maintenance efforts.
The obvious answer is that every customer runs a different Ruby and not every customer pays to upgrade it at the same time. I currently have customers and personal projects with any Ruby from 2.4 from 3.0. I'm not upgrading that Ruby 2.4 project for free so it's going to stay like that for a long time.
Yeah, a ruby -2.4 switch would be nice but then every gem should pick its dependencies from back at the 2.4 time. Some could be dead by now. Actually those old projects usually get updated when the OS reaches end of life and they can't install the software on the new OS.
So economical and people reasons, not technical ones.
It is, mostly. 1.8->1.9 was famously a big change, but breaking changes since have been comparatively minor.
IMX the most common cause for gem breakage is simply rot - old gem, maintainer lost interest, a tiny change required (often due to an open-ended dependency on another gem) but there's no one to do it. Unfortunately, adopting an abandoned gem is non-trivial if the original maintainer doesn't respond, and a lot of gems were written in the Rails goldrush but have fallen into abeyance since.
> Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter?
My understanding is that Perl 5 supports this, for any version of Perl released in the 1994-2021 range, but in large part because it hasn't had drastic changes in that time period.
Looks interesting enough... But how does one solve the issue of security level updates for some dependency language? Or when a particular version of some application reaches EOL and is no longer maintained, or theres some functionality in a newer version of Nodejs|Ruby|etc thats needed?
From what I understand this would require an update to the Nix version that supports it... but that also potentially means bumping other environmental versions as well, which might not be desired.
But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder...
Or is there a better way to cater to this? And I suppose this still means that the node modules, gems, etc that are being used then anyway also need to be updated after this accordingly.
From my limited understanding of Nix, it seems interesting, and the article was actually useful to me. But I cant seem to shake the feeling that this is another packaging abstraction like others before it, and while it seems like a better variant, its not much different to having X, Y and Z listed as requirements and then letting the dev go off and install such dependencies on their system, in the way that they best know how. Juniors or those new to a specific environment might not know the ecosystem so well so as to know to use rbenv or nvm or whatever, but I'm not sure how Nix solves this issue differently than one of the specific tools its replacing.
Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. But purely for lang env set up, I'm not sure I see a point over other tooling...
> Nix is a tool that takes a unique approach to package management and system configuration.
Nix is the basis of an OS distro: NixOS.
This article is just re-articulating the idea that {the/an} OS distro should be managing this stuff, and not some fledgling language-specific programs (that go behind its back, and do a half-baked job by not controlling the packages external to their language ghetto).
I think NixOS is great, but you don’t need to switch to NixOS to use Nix. You can also install Nix on other Linux distributions or even on macOS, and use Nix to manage individual tools and projects without using it to manage your entire system.
the article is also pointing out that Nix can solve for multiple versions, shrink-wrapped environments, cross platform, etc, in a way that most OS distros can't. Nix is more than NixOS of course
For me, the version manager that provides the nicest experience cross-platform is Cargo from Rust.
It can specify and pin specific versions. It is super easy to give to a fellow developer and have them reproduce the build. In addition, because Rust is suitable for low-level, high performance work, many times all the libraries you actually need are written in Rust and everything works very seemlessly (for example, there is no need to have libxml like the Ruby xml library needed, because you can probably write an XML parser in Rust that will be on par or better performance wise with libxml).
Nix is very unixy so it probably could be made to work on WSL, but not native windows.
Yes, if all of your dependencies are written in a single language, you can use the language-specific version manager. TFA was pointing out a solution that work across different languages.
The more I see posts praising nix, the more I am confused by the decisions made.
In this post:
--- start quote ---
How about a different version of Ruby or Node? Let’s say that our project depends on Ruby 2.6 and Node 10. We can go and search for those specific versions
--- end quote ---
So, to begin with, we still need a "version manager", because we want specific package versions. But look at how this is implemented in nix:
let
pkgs = import <nixpkgs> { };
in
pkgs.mkShell {
buildInputs = [
pkgs.hello
pkgs.ruby_2_6
pkgs.nodejs-10_x
];
}
Why? Because unlike every sane package/dependency manager where you specify a package and a version, nix pretends each version is a separate package.
And these packages aren't even correct. If you do go and search for ruby, for example [1], you get the following:
This... This is laughable. How do I install ruby 2.6.8? Oh, there's no ruby_2_6_8, because of course there isn't. And this could be difference between a secure system and all your base are belong to us.
And they call this reproducible builds?
And that's before getting into the ridiculous
--- start quote ---
All the software that we installed depends on the specific version of the nixpkgs channel that we installed on our system [whose only version is a commit hash in a git repo]
--- end quote ---
So you need an extra tool [2] for, quote, "painless dependencies for Nix projects."
Yes, sure. I'm definitely ditching my version managers in favor of this tool, that hasn't solved these issues in 18 years of its existence.
Nix is not easy nor simple. It's complicated but not complex.
re: ruby version
In ubuntu I can only install ruby2.7 and I don't which minor version. [1] I would need to use rvm anyway.
It's the same with nix. What is possible is to pin the version you need by specifying the commit. [2] shows the diff of the commit that moved ruby_2_7 from 2.7.3 to 2.7.4.
Say for example ruby 2.7.4 has a regression and the project needs to stay on 2.7.3.
The revision has for 2.7.3 is used.
[1] > apt search ruby |egrep "^ruby2|^ruby3"
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
ruby2.7/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64
ruby2.7-dev/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64
ruby2.7-doc/hirsute-updates,hirsute-updates,hirsute-security,hirsute-security 2.7.2-4ubuntu1.2 all
As an alternative, I use asdf[0] which is written in bash. It allows you to manage versions of any package through plugins. Plugins can be written in any language (as long as they can be set to chmod +x). Most plugins are written in bash.
Though there is a wealth of plugins, what's nice is that it's trivial to write your plugin. So if something doesn't exist yet, or you have some internal tool you need to version, it takes less than an hour to write a plugin.
I had the feeling this would be an ad for Nix after reading the first few paragraphs.
Is there really a difference from the Docker example given (“your Ubuntu version is EOL”) though? Any of your language or tools can be EOL’ed even if installed via nix, and you have the exact same problem in your hands.
When that wasn't doable, I could google the name of the .tgz file, download a couple and find the one with the matching hash, drop the .tgz inside the directory and change the source to a relative path. Actually I do this even in the "release -> archive" case because "Move a URL once, shame on you. Move it twice, shame on me."
[edit]
If you really don't want to change the .nix expression, you can manually add the file to the cache, and things will Just Work.
Everything to install with Nix must be packaged "the Nix way", I assume? How is the breadth and quality of Nix packages? Just for fun I checked the kiwi image builder, from the openSUSE project, which I recently had some versioning problems with, and it was not there.
The problem I have with Nix (and Guix) is that you're shit out of luck if you need it to work together with some network based package manager or existing project that someone didn't add to it yet.
It gets really complicated really fast, and things that "just work" with other packages managers and projects, because they are opinionated and linux-y, most of the time don't work well with Nix or have to be shoehorned into it.
It always feels like you're fighting against everything to get things into Nix that just weren't meant to be.
The fact you need a version manager on top of your new, pristine and reproducible version manager is a bit concerning. Can’t whatever shortcomings exist be addressed in nix itself?
Good introduction to Nix. I wish I would have had this when I started.
But what do they mean with “works forever”? What if the Git repo or Web server of the project is gone in several years? How do you reproduce it? Does niv include a local source code mirror?
This seems like a good thing for system-level packages, but what about those language-level packages? Could Nix replace both dockerfiles (or a list of apt packages) and pip/conda?
For Docker, you just call a Nix function and specify what programs to include (both implicitly and explicitly via use) and Nix automatically includes the transitive closure of what you need
[+] [-] throwaway984393|4 years ago|reply
Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager.
Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first.
Fourth, finally, we get to install a simple Unix program, that any package manager could have provided.
For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic.
And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers.
Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
[+] [-] Arkh4m|4 years ago|reply
Second. The reason I recommend direnv is because of ergonomics, so you don't have to remember to reload your shell. Most current version managers either do it themselves through shell hooks or ask you to install direnv. Point taken on using brew: I've changed it to use nix-env.
Third. Yep. Sort of like having to learn English before you can learn programming.
Fourth. As they say: start with the basics.
Fifth. No other package manager can freeze the complete dependency set you are using AND allow you to revert it exactly when things don't work. That's the power of declarative, reproducible builds: you try to update and it doesn't work? No problem, revert to the last working version and things are all good again.
Sixth. Ever tried updating a project using a Dockerfile that uses an old version of ubuntu? Then you can't find the packages anymore or no is building them anymore for the new version of Ubuntu and you have to pull random PPAs from the Internet to get things to work? In Nix, you can keep two pinned nixpkgs versions at the same time. In this way you can keep working on upgrading your project one working step at the time.
So yeah, it's not just cHeCKmATe FP-atheists sort of stuff.
[+] [-] globular-toast|4 years ago|reply
As a Gentoo user and someone who has tried and grokked Nix/Guix, this is not true at all. Gentoo is essentially a system for creating distributions, a meta-distro if you like. It just so happens to be something that appeals to someone like me to run at home because, well, why not? I know exactly what I need/want on my computer and this lets me build my own custom distro tailored for me.
Nix/Guix is something completely different. It addresses real problems that arise in a variety of places in an elegant way. Containers are another solution to some of these problems, but come with their own tradeoffs.
Your blanket statement about "professionals" makes no sense. I don't use Gentoo at work because it's not my job to maintain custom Linux systems or create distros. But guess what? Not everyone has the same job. You don't speak for all "professionals". I can distinctly remember around 2007 people saying this exact thing about git. I was told geeks were only learning it to make themselves feel clever and that SVN was fine. Well look how that turned out. I'm glad I ignored those people.
[+] [-] shantnutiwari|4 years ago|reply
And if you want to do anything complicated, it's all get this hash, use this trick, this extra tool etc etc.
At which point, it's just easier to stick with what you have.
>> a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
yeah, getting the same vibe. Lots of hype not backed by any evidence.
For me, to start off, I'd like to see a real example, not some made up simple use case
[+] [-] rossmohax|4 years ago|reply
[+] [-] takeda|4 years ago|reply
This is not true. The defining point about Nix is that it allows to define all dependencies explicitly. Every derivation in it is built inside of a sandbox with no access to network and restricted filesystem. This ensures that nothing is accidentally missed. Package management is just one of features.
> Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager.
direnv is not needed to use Nix, it's just a nice add-on. It's like you don't need Ranger to navigate in a file system, but it is nice. Besides, you can install direnv through nix, I use it this way. Even nix is installed by itself as well.
> Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first.
Well, yes you do, but language is actually quite simple it's the nixpkgs (repo containing all packages) that's quite complex. Nix's purely functional, lazily evaluated properties exactly fit as a language describing dependencies.
> Fourth, finally, we get to install a simple Unix program, that any package manager could have provided.
If you were trying to install a package, you don't need to learn anything, you just install it (for example to install ranger you just run nix-env -iA nixpkgs.ranger). Nix language is only needed if you want to create a new package.
> For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic.
> And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers.
> Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
Nix delivers what docker promised and ultimately failed to deliver. Docker promised to reproduce developer's environment to production. What it did was to zip developer's computer and deploy it. When docker got adopted, relying on images was impractical, so for deployment Dockerfile was used, but that file is not much different than a shell script.
Nix instead describes the entire dependency tree down to libc. Because the starting state and all dependencies are known it can always create the same result, that's the biggest selling point of Nix to me.
[+] [-] GoblinSlayer|4 years ago|reply
[+] [-] KronisLV|4 years ago|reply
To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much.
For example, a new project that was created with "create-react-app" takes 181 MB of space on disk and has 35'894 files in it. That includes about 1467 modules, all for a relatively simple web application. It doesn't matter if you're using package.json/package.lock, or any other tool or solution out there - with that amount of dependencies you're simply not doing dependency "management" of any kind.
I'd argue that if you have >100 dependencies in your project, it's probably too big, unless you have a team that's dedicated to managing and auditing all of them. Of course, no one actually audits their dependencies when faced with such large numbers, and so what's left for most developers is to just trust what's out there.
[+] [-] cwp|4 years ago|reply
Now auditing dependencies, knowing that the packages you depend on aren't malicious and have no known vulnerabilities... well, that's a whole separate problem. And yeah, we don't really have a solution to that right now. The best we can do, as you say, is keep the attack surface small.
But if we did try to solve the auditing problem, the solution would have to sit on top of nix or something like it. If you can't precisely specify a dependency graph and reliably install from that specification, it doesn't matter how good your auditing is or what sort of system of trust you can create. You don't know what you're getting anyway.
[+] [-] krab|4 years ago|reply
I'm fine to depend on, say, JVM or Qt - large projects. Not with thousands of small packages developed by thousands of independent developers.
[+] [-] IceWreck|4 years ago|reply
But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions.
Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem.
[+] [-] nineteen999|4 years ago|reply
There a large number of developers around now who've never done traditional (RPM, DEB, SysV) packaging and don't understand it. So while the large distros like Redhat and Debian push on with it, the development community only sees Dockerfiles, Snaps, Flatpaks etc.
[+] [-] tikhonj|4 years ago|reply
I've been writing normal Dockerfiles at work recently and the whole toolchain is annoying on multiple fronts; I'd love to jump to Nix instead, except it's really hard to convince other people to learn non-trivial technology that's off the beaten path :(.
[+] [-] oneplane|4 years ago|reply
[+] [-] swsieber|4 years ago|reply
[+] [-] PikachuEXE|4 years ago|reply
[+] [-] jeffparsons|4 years ago|reply
[+] [-] phendrenad2|4 years ago|reply
If we question our assumptions, the first question is: Why do we need multiple Ruby versions at all? Why isn't the latest version of Ruby sufficient? Well, obviously, Ruby's behavior has changed over time. But why isn't it backward-compatible? Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter?
Okay another one. "Nokogiri" is the Ruby gem (library) for parsing XML (including HTML). You always need to remember to install libxml when you install it. Why? Why doesn't Nokogiri include an XML parser? Because it would be slow? You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Also, what if I just need some quick XML parsing and don't care if the parsing is 1000x slower? Can I just get Nokogiri with the Ruby-implemented parser? No? Why not?
Then every single Ruby gem out there starts using newer Ruby features and thus, you must update your Ruby version. Why can't library authors gracefully handle older versions of Ruby?
[+] [-] Master_Odin|4 years ago|reply
I imagine variations of the above apply to nokogiri and some amount of Ruby core developers.
[+] [-] jweir|4 years ago|reply
From https://nokogiri.org
> These dependencies are met by default by Nokogiri's packaged versions of the libxml2 and libxslt source code, but a configuration option --use-system-libraries is provided to allow specification of alternative library locations.
Some authors work hard to have their tools do the right thing and consistently.
[+] [-] Closi|4 years ago|reply
I suspect it would be a nightmare - it's not just small parts of the code, it would be other parts of the code that depend on it.
Take for instance Python 2 & Python 3's approach to if ("hello" > "goodbye") - in Python 2 that would be False and in Python 3 that would be TypeError. So if you are running Python 3 with a Python 2.7 flag, how should Python's internal modules that are written in Python 3 handle this? Do all Python's internal modules need to work on every legacy version of Python and load the correct version, or does the interpreter need to somehow load up instructions for both and parse them together? Then there is the whole problem about libraries - are these allowed to selectively choose which version of python they run on?
This is of course only one small example of the complexity - If it was easy to make these things backwards compatible it would be great, but I would expect the cost for implementing full backwards compatibility to be high and the language authors are probably motivated to make people move towards the latest versions.
[+] [-] suby|4 years ago|reply
[+] [-] mixedCase|4 years ago|reply
Promptly ignored by most people, all while doubling maintenance efforts.
It's a trade-off without an easy answer.
[+] [-] pmontra|4 years ago|reply
Yeah, a ruby -2.4 switch would be nice but then every gem should pick its dependencies from back at the 2.4 time. Some could be dead by now. Actually those old projects usually get updated when the OS reaches end of life and they can't install the software on the new OS.
So economical and people reasons, not technical ones.
[+] [-] Doctor_Fegg|4 years ago|reply
It is, mostly. 1.8->1.9 was famously a big change, but breaking changes since have been comparatively minor.
IMX the most common cause for gem breakage is simply rot - old gem, maintainer lost interest, a tiny change required (often due to an open-ended dependency on another gem) but there's no one to do it. Unfortunately, adopting an abandoned gem is non-trivial if the original maintainer doesn't respond, and a lot of gems were written in the Rails goldrush but have fallen into abeyance since.
[+] [-] remexre|4 years ago|reply
My understanding is that Perl 5 supports this, for any version of Perl released in the 1994-2021 range, but in large part because it hasn't had drastic changes in that time period.
[+] [-] zaSmilingIdiot|4 years ago|reply
From what I understand this would require an update to the Nix version that supports it... but that also potentially means bumping other environmental versions as well, which might not be desired. But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder... Or is there a better way to cater to this? And I suppose this still means that the node modules, gems, etc that are being used then anyway also need to be updated after this accordingly.
From my limited understanding of Nix, it seems interesting, and the article was actually useful to me. But I cant seem to shake the feeling that this is another packaging abstraction like others before it, and while it seems like a better variant, its not much different to having X, Y and Z listed as requirements and then letting the dev go off and install such dependencies on their system, in the way that they best know how. Juniors or those new to a specific environment might not know the ecosystem so well so as to know to use rbenv or nvm or whatever, but I'm not sure how Nix solves this issue differently than one of the specific tools its replacing.
Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. But purely for lang env set up, I'm not sure I see a point over other tooling...
[+] [-] kazinator|4 years ago|reply
Nix is the basis of an OS distro: NixOS.
This article is just re-articulating the idea that {the/an} OS distro should be managing this stuff, and not some fledgling language-specific programs (that go behind its back, and do a half-baked job by not controlling the packages external to their language ghetto).
[+] [-] anderskaseorg|4 years ago|reply
[+] [-] rrix2|4 years ago|reply
[+] [-] RcouF1uZ4gsC|4 years ago|reply
For me, the version manager that provides the nicest experience cross-platform is Cargo from Rust.
It can specify and pin specific versions. It is super easy to give to a fellow developer and have them reproduce the build. In addition, because Rust is suitable for low-level, high performance work, many times all the libraries you actually need are written in Rust and everything works very seemlessly (for example, there is no need to have libxml like the Ruby xml library needed, because you can probably write an XML parser in Rust that will be on par or better performance wise with libxml).
[+] [-] aidenn0|4 years ago|reply
Yes, if all of your dependencies are written in a single language, you can use the language-specific version manager. TFA was pointing out a solution that work across different languages.
[+] [-] dmitriid|4 years ago|reply
In this post:
--- start quote ---
How about a different version of Ruby or Node? Let’s say that our project depends on Ruby 2.6 and Node 10. We can go and search for those specific versions
--- end quote ---
So, to begin with, we still need a "version manager", because we want specific package versions. But look at how this is implemented in nix:
Why? Because unlike every sane package/dependency manager where you specify a package and a version, nix pretends each version is a separate package.And these packages aren't even correct. If you do go and search for ruby, for example [1], you get the following:
This... This is laughable. How do I install ruby 2.6.8? Oh, there's no ruby_2_6_8, because of course there isn't. And this could be difference between a secure system and all your base are belong to us.And they call this reproducible builds?
And that's before getting into the ridiculous
--- start quote ---
All the software that we installed depends on the specific version of the nixpkgs channel that we installed on our system [whose only version is a commit hash in a git repo]
--- end quote ---
So you need an extra tool [2] for, quote, "painless dependencies for Nix projects."
Yes, sure. I'm definitely ditching my version managers in favor of this tool, that hasn't solved these issues in 18 years of its existence.
[1] https://search.nixos.org/packages?channel=21.05&from=0&size=...
[2] https://github.com/nmattia/niv
[+] [-] rvanlaar|4 years ago|reply
re: ruby version In ubuntu I can only install ruby2.7 and I don't which minor version. [1] I would need to use rvm anyway. It's the same with nix. What is possible is to pin the version you need by specifying the commit. [2] shows the diff of the commit that moved ruby_2_7 from 2.7.3 to 2.7.4.
Say for example ruby 2.7.4 has a regression and the project needs to stay on 2.7.3. The revision has for 2.7.3 is used.
[1] > apt search ruby |egrep "^ruby2|^ruby3"
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
ruby2.7/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-dev/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-doc/hirsute-updates,hirsute-updates,hirsute-security,hirsute-security 2.7.2-4ubuntu1.2 all
[2] https://github.com/NixOS/nixpkgs/commit/5f9f17cc11caa07e4a9f...
[+] [-] gizdan|4 years ago|reply
Though there is a wealth of plugins, what's nice is that it's trivial to write your plugin. So if something doesn't exist yet, or you have some internal tool you need to version, it takes less than an hour to write a plugin.
[0] https://asdf-vm.com/
[+] [-] type0|4 years ago|reply
[+] [-] ricardobeat|4 years ago|reply
Is there really a difference from the Docker example given (“your Ubuntu version is EOL”) though? Any of your language or tools can be EOL’ed even if installed via nix, and you have the exact same problem in your hands.
[+] [-] aidenn0|4 years ago|reply
When that wasn't doable, I could google the name of the .tgz file, download a couple and find the one with the matching hash, drop the .tgz inside the directory and change the source to a relative path. Actually I do this even in the "release -> archive" case because "Move a URL once, shame on you. Move it twice, shame on me."
[edit]
If you really don't want to change the .nix expression, you can manually add the file to the cache, and things will Just Work.
[+] [-] mongol|4 years ago|reply
[+] [-] shtps|4 years ago|reply
It gets really complicated really fast, and things that "just work" with other packages managers and projects, because they are opinionated and linux-y, most of the time don't work well with Nix or have to be shoehorned into it.
It always feels like you're fighting against everything to get things into Nix that just weren't meant to be.
[+] [-] Kiro|4 years ago|reply
Don't you do that in Docker? I've read the sentence about Ubuntu EOL but don't really understand.
[+] [-] marcus_holmes|4 years ago|reply
[+] [-] moondev|4 years ago|reply
[+] [-] yewenjie|4 years ago|reply
[+] [-] ricardobeat|4 years ago|reply
[+] [-] hda111|4 years ago|reply
But what do they mean with “works forever”? What if the Git repo or Web server of the project is gone in several years? How do you reproduce it? Does niv include a local source code mirror?
[+] [-] blt|4 years ago|reply
[+] [-] whateveracct|4 years ago|reply
For Docker, you just call a Nix function and specify what programs to include (both implicitly and explicitly via use) and Nix automatically includes the transitive closure of what you need