You are right. It's unfair, I'm going to modify the post to mention alpine.
The post is both about tiny images AND how to build docker images with nix. I think it was an interesting tooling to make for our community. Some Nix people are already using it for obvious reasons.
A common problem with Docker is after running a compiler/preprocessor during an image build one ends up with the bulid tools inside the image. A workaround is to use a shell script that first gets the image with the compiler, run it and then pass the output to the Docker build. But this is non-standard and encourages running random scripts on the build host defeating the advantages of using Docker during development to isolate build system. It is nice that Nix addresses this.
Or you could remove the tools at the end of your Dockerfile...
Edit: am I missing something? This is a legitimate solution to the problem. Install the tools, compile, and remove them. The parent is suggesting a very clumsy approach (build on the host and pass the binary to the container as it's being built).
The idea is actually to do something similar to a heroku buildpack where you have a container with build tools that generates binary assets. You then inject the built binary into a new image that has only runtime dependencies installed.
I've experimented (and use) a variant of this workflow myself built around my marina tool [1]. The basic idea is to define a file that uses a dev/builder image to build, then exports a tarball into a runner image.
Nix has been a very cool project to watch over the years.
You can address part of the problem of picking up extra data in final images by declaring temporary build locations, such as `/var/lib/cache`, as a volume. Anything written to a volume won't be included in the final image.
If the goal is solely Docker images with a standard size in the 20-40MB range, this can be achieved without additional tooling. After switching our development and deployment flow to docker, my team quickly tired of the 200-400MB images that seemed to be accepted as commonplace. We started basing our containers on alpine (essentially, busybox with a package manager) or alpine derivatives, and dropped into that target size immediately. Spinning up 8-10 microservices locally for a full frontend development stack is a shockingly better experience when that involves a 200MB download rather than a 2GB one.
This is in no way a negative commentary on Nix; it looks like an interesting solution to a well-known problem.
Same here! Switching to Alpine for most services was essentially painless. To go a step further, the images with binaries that have no dependencies (mostly programs written in Go) use scratch Docker images. This way we get 5MB images, where the size overhead of Docker is nothing.
> Be aware that things like PAM configurations, or other stuff, created to be suitable for Debian may not work with Nix programs that use a different glibc.
So this would not be a factor using the method which has no base? The Debian base approach seems like a non-starter if negative emergent behavior like PAM config mismatches are common.
Also, to be sure, I can do this "no base Docker build" using Nix on let's say CentOS 7? Meaning, I'm not required to use NixOS natively?
I plan to read the post closer later today, so feel free to ignore these questions if they are answered in-post, but I usually don't post to HN from work computer, so I thought I'd get my questions out here early in case the thread drops and I forget to ask. :-)
Yes, AFAIK you can build docker containers with Nix on any Linux machine with the nix package manager installed. You can even build them on OSX if you configure your OSX Nix install with a remote build machine that runs Linux (yes, Nix can automatically and transparently distribute your builds).
Installed emacs with nix last weekend to use with spacemacs. Three errors at startup that didn't make sense but I had a feeling that nix was the problem so rolled back and reinstalled with brew. Worked perfectly. Nix has some great ideas but when the install of emacs took 10 times as long as homebrew, then didn't work correctly it didn't leave me wanting to use it for anything work related like my docker images.
Hopefully with time, their planned cli improvements and binary caching it will be a contender but that feels a ways off at this point.
> Nix has some great ideas but when the install of emacs took 10 times as long as homebrew, then didn't work correctly it didn't leave me wanting to use it for anything work related like my docker images.
I understand it's not fun to watch a long build process finish only to have the final product not work, but that assessment is not really fair. Nix itself is not the problem there; the problem is in the definition of that package. The same thing could just as easily occur in a package defined in homebrew. Saying that nix itself is at fault when an individual package (among hundreds of thousands) is faulty is sort of akin to encountering a buggy program and equating it to a bug in the language the program was written in.
I'd also add that due to the closed-source nature of OSX core libraries, it's hard to achieve the same degree of robust determinism in OSX that nix allows in other platforms. Fortunately things tend to be very solid on Linux. My company has been using Nix in production for about a year now and it has been a huge benefit to platform stability and speed of deployment.
Nix does have binary caching (but only if you use `/nix/store` for the nix store path), I'm using it all the time on linux. OSX support is definitly much less stable though and there have been problems with build machines for OSX in the past, so I do not know how the situation on OSX regarding binary caches is currently.
Regarding your point about build time, I'm not sure why installing with nix would make that much of a difference. Nix still just executes the build system of the underlying packages. However, nix might have to build more packages since it doesn't rely on the underlying system to provide dependencies which helps to make it more robust. Additionally, emacs has various build configurations, which might also require different dependencies, affecting build time.
If there are any bug reports you could file, that would be much appreciated. Installing Emacs should just be a case of downloading the prebuild binaries, which shouldn't take long at all. I use Spacemacs and have never encountered any problems. I have a feeling things are still a lot worse on OSX though...
I really like the Nix package manager, however is there an upside to using Nix to build a Docker image over just writing a regular Dockerfile? Is this an odd use case? Maybe it just for demo purposes? Is there a benefit I'm overlooking?
Interesting. Not being that familiar with Nix(OS), how much of a moving target Nix is? Can you do these kinds of things with stable versions or do you need to keep up with HEAD?
The blog post code is supposed to run with latest master since a few days. We've merged a big change that leads to reducing a lot the closure of our packages.
Nix moves fast enough, in the sense we usually do a good job at not breaking things. Yet we have to necessarily introduce innovations in our frameworks.
I'm wondering what would be the advantage of using Nix versus building on Alpine Linux with good understanding how Docker layers work. My main reason to be skeptical about Nix is the need to learn a new single-purpose language as opposed to just using Shell like you do in Dockerfiles.
Both the Nix language and the Dockerfile language include embedding shell scripts. Let's not pretend that Docker doesn't have its own DSL to learn. Dockerfile's are imperative, where as Nix is declarative and functional, which is a big improvement.
This is the same thing I was asking. As much as I like the idea of declarative functional package manager what value does it provide if you are just building docker images?
what docker build extensions would make it possible to do this without 'rolling your own' tarball docker layer? having access to volumes at build time?
[+] [-] nickjj|10 years ago|reply
It's unfair to say the official Redis image is 177mb because the Alpine version is available on the Docker Hub[1] and it's only 15.95mb.
Alpine is pretty awesome if your main goal is to shrink images without any effort[2].
[1] https://hub.docker.com/_/redis/
[2] http://nickjanetakis.com/blog/alpine-based-docker-images-mak...
[+] [-] Lethalman|10 years ago|reply
The post is both about tiny images AND how to build docker images with nix. I think it was an interesting tooling to make for our community. Some Nix people are already using it for obvious reasons.
[+] [-] markbnj|10 years ago|reply
[+] [-] _0w8t|10 years ago|reply
[+] [-] xienze|10 years ago|reply
Edit: am I missing something? This is a legitimate solution to the problem. Install the tools, compile, and remove them. The parent is suggesting a very clumsy approach (build on the host and pass the binary to the container as it's being built).
[+] [-] mmerickel|10 years ago|reply
I've experimented (and use) a variant of this workflow myself built around my marina tool [1]. The basic idea is to define a file that uses a dev/builder image to build, then exports a tarball into a runner image.
[1] https://pypi.python.org/pypi/marina
[+] [-] stevvooe|10 years ago|reply
You can address part of the problem of picking up extra data in final images by declaring temporary build locations, such as `/var/lib/cache`, as a volume. Anything written to a volume won't be included in the final image.
[+] [-] TomFrost|10 years ago|reply
This is in no way a negative commentary on Nix; it looks like an interesting solution to a well-known problem.
[+] [-] Artemis2|10 years ago|reply
[+] [-] iElectric2|10 years ago|reply
[+] [-] edofic|10 years ago|reply
[+] [-] josh-wrale|10 years ago|reply
> Be aware that things like PAM configurations, or other stuff, created to be suitable for Debian may not work with Nix programs that use a different glibc.
So this would not be a factor using the method which has no base? The Debian base approach seems like a non-starter if negative emergent behavior like PAM config mismatches are common.
Also, to be sure, I can do this "no base Docker build" using Nix on let's say CentOS 7? Meaning, I'm not required to use NixOS natively?
I plan to read the post closer later today, so feel free to ignore these questions if they are answered in-post, but I usually don't post to HN from work computer, so I thought I'd get my questions out here early in case the thread drops and I forget to ask. :-)
Nice work!
[+] [-] trishume|10 years ago|reply
[+] [-] k__|10 years ago|reply
So Nix makes even the use of Docker better, while some Nix user here claimed that you don't even need Docker if you're using Nix(OS).
[+] [-] Lethalman|10 years ago|reply
[+] [-] speedkills|10 years ago|reply
Hopefully with time, their planned cli improvements and binary caching it will be a contender but that feels a ways off at this point.
[+] [-] thinkpad20|10 years ago|reply
I understand it's not fun to watch a long build process finish only to have the final product not work, but that assessment is not really fair. Nix itself is not the problem there; the problem is in the definition of that package. The same thing could just as easily occur in a package defined in homebrew. Saying that nix itself is at fault when an individual package (among hundreds of thousands) is faulty is sort of akin to encountering a buggy program and equating it to a bug in the language the program was written in.
I'd also add that due to the closed-source nature of OSX core libraries, it's hard to achieve the same degree of robust determinism in OSX that nix allows in other platforms. Fortunately things tend to be very solid on Linux. My company has been using Nix in production for about a year now and it has been a huge benefit to platform stability and speed of deployment.
[+] [-] bennofs|10 years ago|reply
Regarding your point about build time, I'm not sure why installing with nix would make that much of a difference. Nix still just executes the build system of the underlying packages. However, nix might have to build more packages since it doesn't rely on the underlying system to provide dependencies which helps to make it more robust. Additionally, emacs has various build configurations, which might also require different dependencies, affecting build time.
[+] [-] ocharles|10 years ago|reply
[+] [-] bogomipz|10 years ago|reply
[+] [-] cstrahan|10 years ago|reply
https://news.ycombinator.com/item?id=11509065
[+] [-] isido|10 years ago|reply
[+] [-] Lethalman|10 years ago|reply
Nix moves fast enough, in the sense we usually do a good job at not breaking things. Yet we have to necessarily introduce innovations in our frameworks.
[+] [-] meta_AU|10 years ago|reply
[+] [-] fishnchips|10 years ago|reply
[+] [-] davexunit|10 years ago|reply
[+] [-] bogomipz|10 years ago|reply
[+] [-] awinter-py|10 years ago|reply