Until well into the 2000s we deployed all our services as chroots. I even sold a server side product successfully which was distributed as chroot so it would install almost on any Linux distro without having to care about dependencies. It worked really well and felt quite a bit more accessible and lighter than docker. We just had a script which basically did:
wget https://bla/image.tgz
tar xzf image.tgz
rm image.tgz
chroot image/start
and the product would be installed and up & running. A lot of clients thought it was magic. No Apache install? No MySQL install?? It works on Fedora & Centos & Debian & Ubuntu & ... ?
I use this currently still a lot to run new software on unsupported hardware/kernels. For instance, I run a lot of new (Ubuntu) software on my OpenPandora under the default OS on that device; I have an SD with a chroot with Ubuntu and things like dotnet core. Runs fine. Gives devices like that a second youth.
Docker has mostly succeeded in making this simple concept complicated. A "container image" is just a tape archive of some files. The runtime is just like your chroot, but with namespaces and cgroups. That's about it.
On top of all this, they designed a new programming language / build system for making those tar files. They invented a container registry so that instead of just "downloading a file", you have to use their particular protocol to download a file. They have their own version of systemd so you can see what programs are running. They tacked on some very complicated networking stuff. They tried to build a couple of container orchestration frameworks on top of all this. I think all the "extra stuff" is where people run into problems, and it's isn't even the core of the value it provides.
People have realized that it's all a little much and the good bits have been distilled into their own projects. Many build systems can just generate an OCI registry file directly (remember, it's just a tar file, and you don't need all that 1970s JCL stuff to make a tar file). Various runtimes (containerd/CRI-O) exist for running containers. And there are orchestration layers on top, for actually managing sets of containers, network rules, etc. (ECR, Kubernetes, etc.).
(Worth noting is that Docker itself does use all these spin-offs. It uses containerd as the runtime, it uses buildkit for building containers now, etc. They even ship a Kubernetes distribution, at least on Mac and Windows.)
Nowadays, creating a raw chroot is far from simple. Last time I tried, I had to handle the special mount points like /proc and /sys. I can't remember the details, but I believe there were other problems.
In my experience, systemd-nspawn provides a nice alternative. It's mainly a chroot that abstracts the low-level details, and it fixes some security concerns (with cgroups, etc) and adds some nice features: e.g. networking, or `systemctl status` from the host that will display the sub-tree for the processes of the sub-host.
The scale at which containerised scheduling of this outcome saves more time than it consumes is much higher than many have been led to believe. I measure it in sysadmin FTE. If you have >0.5 FTE dedicated to managing deployments then there may be a payoff, because (amortized over a reasonable period e.g. 12 months) it takes at minimum half an FTE to setup, manage, and maintain a PaaS properly.
I've become accustomed to folks claiming things like, "I spend 5 minutes a week managing my kubernetes cluster". Then it turns out it took them a solid month to get it going properly, two weeks to containerise their application reliabily, and then next quarter they're stuck for days on some weird networking or logging issue, and nothing else progresses. Basically, they either don't value their time, or they're misjudging how much they've actually allocated.
It's often the same people who boast about how cheap their bare-metal servers are vs public cloud instances.
I used that for build machines. Had a btrfs filesystem on a loopback device I could just download as needed to seed the agents and then make a subvolume to work in. When all was done, I could just drop it.
A lot of was automated with "schroot" and definitely faster than spawning a new instance of a VM every time!
For me, Docker is important because it lets me use Kubernetes. That's what I really want to use. If Docker used chroot instead of their own thing, I would be fine as well.
I dont even read or write bash shell scripts regularly and i can understand what that line is doing fine. I would not be able to understand if the entire thing was one line, so i think there is a significant difference.
Just because something doesn't follow general rules for readable code, doesn't mean it is actually unreadable.
That roughly corresponds to the following C-like pseudocode:
for (i = 0; i < argc && (arg = argv[i]).startswith("--"); ++i)
if (sep = arg.find("=")) opts[arg[..sep]] = arg[sep+1..]; else opts[arg] = "x";
It might be too much for one line, but it doesn't seem to be super complicated either. In fact that line is (as of now, commit 0006330) only the second longest in that file.
This isn't really that horrible; just condensing the boilerplate that comes with command line argument parsing. Sure it's quick-and-dirty, but it gets it done and moves focus your elsewhere.
Interestingly, I didn't know you could do this without an `eval`:
function bocker_help() { #HELP Display this message:\nBOCKER help
sed -n "s/^.*#HELP\\s//p;" < "$1" | sed "s/\\\\n/\n\t/g;s/$/\n/;s!BOCKER!${1/!/\\!}!g"
}
Initially I thought that bash supports reflection and is able to get the function contents, including comments. But this function scans its own script file ($1), looking for the comments starting with #HELP and formatting them. This way the usage info can live near the functions implementing sub-commands.
No doubt this is cool and represents good work! Nice job!
Can we really say it’s a reimplementation of Docker in 100 lines, though, when it requires other dependencies that probably total in the hundreds of thousands of lines? That’s still code that has to be audited by people so inclined. Not to mention the other setup specified in the readme and maybe having to build one of the dependencies from source. Usage doesn’t sound necessarily trivial.
Makes me appreciate Docker that much more though. Getting up to speed using it at my job and it certainly abstracts away many things, simplifying our workflow and conceptual models.
I think the idea is to show how much container functionality is actually available out-of-the-box from the operating system. It raises questions about Docker's billion dollar valuation when someone can come along and reproduce a bunch of it with a short shell script. Next someone needs to take on orchestration and show how silly it was for IBM to pay $34 billion for OpenShift. :P
Docker also depends on hundreds of thousands on lines as well. In fact, to run it on Windows, it requires both Windows and Linux as dependencies /s
More seriously - it’s not a complete docket of course, but lines of code in a project are a liability, not an asset. If you can reduce your project size with reasonable dependencies, you should.
It does seem like Docker will be holding back containers from achieving their true promise, as it flounders looking for profitability and chasing k8s. Its sad that we're still moving around tar balls without advancing some of the tooling around it.
One of my biggest gripes as a non-k8s container user are that we are still keeping all this versioned cruft with every container. I would like to see an easier way to just get the latest binaries not dozens of tar hashes with versions going back to the first iteration of the container. Probably closer to singularity https://singularity.lbl.gov/
Another area is the ability to update containers in place. Hass.io https://www.home-assistant.io/hassio/ does some of this with the ability to manage the container version from within the container, but seems like a little bit of a hack from what I've seen. Watchtower is closer to what I'd like https://github.com/containrrr/watchtower, but why can't we have a more integrated system built into the default tooling?
Docker is a good tool and deserves some resources to continue developing the product and funding marketing team. I wish docker hub to be more competitive among other rivals.
For me, the hardest thing about Docker was running my own registries. Or rather, configuring credentials so Docker would use them. Not to mention the hassle of doing that through Tor.
Fun anecdote: Early in the lifespan of CoreOS (early 2014 IIRC) I was meeting with technology leadership at a Fortune 500 customer. I remember the head of developer services asking all kinds of Docker questions. Finally, he was cut off by his boss who said:
"Bob¹*, unless we've solely staffed the organization with folks from coder camps a lot of the folks here should be able to carve out a couple of namespaces in the kernel. That's not interesting and that's not why we asked CoreOS to come in today. Let's talk about the actual problems they're trying to solve."
<3 <3 <3 <3
That being said, it's great that folks continue to dig in to decompose the tooling and show what's actually happening behind the scenes.
¹Bob was definitely not that guy's name.
EDIT: Added note of context appreciating the work by the author.
I was thinking about something like this for build systems. Everything in Docker is regular Linux. I get why Docker is so big for its use case as cloud deployments, but what I actually want from it is such a small piece of it. Hermetic, reproducible builds that produce the same binary on the same kernel release. No dependency hell because the dependencies are all part of the same build. (I know Bazel has a lot of this already.) The Docker solution of pulling in an entire distro is overkill, and it doesn't even solve the problem because dependencies are just downloaded from the package manager.
If that's your concern then Nix might be your thing, because that's what is targeting. It approaches the problem slightly differently. Instead of generating an image, it is used to describe all dependencies for your project down to glibc. Of course you're not expected to define full dependencies of your project, so you (and most people) will use nixpkgs. As long as nixpkgs is fixed to specific version (it's literally a github repo) you can get identical result every time.
Once you have that then you want to deploy it in whatever way you like it. Nixpkgs has functions to for example generate a docker image that contains only your app + essential dependencies. You could deploy it using nix package manager as you would install a typical app in the OS. You could also describe configuration of NixOS that has your application included and it could generate an image of it.
> it doesn't even solve the problem because dependencies are just downloaded from the package manager.
The advantage of Docker is that you can verify the container works locally as part of the build process rather than finding out it is broken due to some missing dep after a deployment. If you can verify that the image works then the mechanism for fetching the deps can be as scrappy as you like. Docker moves the dependency challenge from deployment-time to build-time.
"Everything in Docker is regular Linux" is a bit of a misleading statement IMO. You aren't required to pull an entire big distro like Ubuntu and install deps from a package manager. Are you familiar with scratch builds? You can create an image that is basically just a single executable binary with a few mounts handled for you behind the scenes so that chroot works.
Typically a minimal Alpine distro image will allow you to pull in deps however you want (either via package manager, manual download, or copy), run a build, and then copy only the artifact over to a scratch image for a final image that's only a few MB in size.
Huh, clever use of btrfs volumes, it does make it a little dependent on the filesystem though. Quite informative overall, probably implements only a fraction of docker, but it does do most of what you'd typically need.
Containerization makes sense for a tiny, tiny fraction of all products. It's just been overblown and overused, creating another point of potential failure for no great benefit at all.
Just chroot the damn thing if you need to and keep it simple.
[+] [-] tluyben2|6 years ago|reply
I use this currently still a lot to run new software on unsupported hardware/kernels. For instance, I run a lot of new (Ubuntu) software on my OpenPandora under the default OS on that device; I have an SD with a chroot with Ubuntu and things like dotnet core. Runs fine. Gives devices like that a second youth.
[+] [-] jrockway|6 years ago|reply
On top of all this, they designed a new programming language / build system for making those tar files. They invented a container registry so that instead of just "downloading a file", you have to use their particular protocol to download a file. They have their own version of systemd so you can see what programs are running. They tacked on some very complicated networking stuff. They tried to build a couple of container orchestration frameworks on top of all this. I think all the "extra stuff" is where people run into problems, and it's isn't even the core of the value it provides.
People have realized that it's all a little much and the good bits have been distilled into their own projects. Many build systems can just generate an OCI registry file directly (remember, it's just a tar file, and you don't need all that 1970s JCL stuff to make a tar file). Various runtimes (containerd/CRI-O) exist for running containers. And there are orchestration layers on top, for actually managing sets of containers, network rules, etc. (ECR, Kubernetes, etc.).
(Worth noting is that Docker itself does use all these spin-offs. It uses containerd as the runtime, it uses buildkit for building containers now, etc. They even ship a Kubernetes distribution, at least on Mac and Windows.)
[+] [-] idoubtit|6 years ago|reply
In my experience, systemd-nspawn provides a nice alternative. It's mainly a chroot that abstracts the low-level details, and it fixes some security concerns (with cgroups, etc) and adds some nice features: e.g. networking, or `systemctl status` from the host that will display the sub-tree for the processes of the sub-host.
[+] [-] inopinatus|6 years ago|reply
1. Put files on a device
2. Start a process
The scale at which containerised scheduling of this outcome saves more time than it consumes is much higher than many have been led to believe. I measure it in sysadmin FTE. If you have >0.5 FTE dedicated to managing deployments then there may be a payoff, because (amortized over a reasonable period e.g. 12 months) it takes at minimum half an FTE to setup, manage, and maintain a PaaS properly.
I've become accustomed to folks claiming things like, "I spend 5 minutes a week managing my kubernetes cluster". Then it turns out it took them a solid month to get it going properly, two weeks to containerise their application reliabily, and then next quarter they're stuck for days on some weird networking or logging issue, and nothing else progresses. Basically, they either don't value their time, or they're misjudging how much they've actually allocated.
It's often the same people who boast about how cheap their bare-metal servers are vs public cloud instances.
[+] [-] Orphis|6 years ago|reply
[+] [-] somesortofsystm|6 years ago|reply
We should never forget that at the root of containerization is a desire to keep everyones shit sorted. That's been a Unix thing forever.
EDIT: "Gives devices like that a second youth."
Hell yeah, long live the OP!
[+] [-] Riverheart|6 years ago|reply
[+] [-] rocho|6 years ago|reply
[+] [-] eleitl|6 years ago|reply
[+] [-] bapcyk|6 years ago|reply
[+] [-] Wowfunhappy|6 years ago|reply
[+] [-] maest|6 years ago|reply
[+] [-] soygul|6 years ago|reply
[+] [-] bawolff|6 years ago|reply
Just because something doesn't follow general rules for readable code, doesn't mean it is actually unreadable.
[+] [-] lifthrasiir|6 years ago|reply
[+] [-] boobsbr|6 years ago|reply
[+] [-] Xophmeister|6 years ago|reply
Interestingly, I didn't know you could do this without an `eval`:
[+] [-] phinnaeus|6 years ago|reply
[+] [-] t0mek|6 years ago|reply
[+] [-] _def|6 years ago|reply
[+] [-] sixstringtheory|6 years ago|reply
Can we really say it’s a reimplementation of Docker in 100 lines, though, when it requires other dependencies that probably total in the hundreds of thousands of lines? That’s still code that has to be audited by people so inclined. Not to mention the other setup specified in the readme and maybe having to build one of the dependencies from source. Usage doesn’t sound necessarily trivial.
Makes me appreciate Docker that much more though. Getting up to speed using it at my job and it certainly abstracts away many things, simplifying our workflow and conceptual models.
[+] [-] brianpgordon|6 years ago|reply
[+] [-] beagle3|6 years ago|reply
More seriously - it’s not a complete docket of course, but lines of code in a project are a liability, not an asset. If you can reduce your project size with reasonable dependencies, you should.
[+] [-] heavenlyblue|6 years ago|reply
[+] [-] nickjj|6 years ago|reply
It's being a toolset to build and deploy containerized applications.
In other words, being able to docker-compose up --build on any Linux, Windows and MacOS box is what's useful.
[+] [-] crdoconnor|6 years ago|reply
[+] [-] zantana|6 years ago|reply
https://blog.alexellis.io/building-containers-without-docker...
It does seem like Docker will be holding back containers from achieving their true promise, as it flounders looking for profitability and chasing k8s. Its sad that we're still moving around tar balls without advancing some of the tooling around it.
One of my biggest gripes as a non-k8s container user are that we are still keeping all this versioned cruft with every container. I would like to see an easier way to just get the latest binaries not dozens of tar hashes with versions going back to the first iteration of the container. Probably closer to singularity https://singularity.lbl.gov/
Another area is the ability to update containers in place. Hass.io https://www.home-assistant.io/hassio/ does some of this with the ability to manage the container version from within the container, but seems like a little bit of a hack from what I've seen. Watchtower is closer to what I'd like https://github.com/containrrr/watchtower, but why can't we have a more integrated system built into the default tooling?
[+] [-] adgasf|6 years ago|reply
[+] [-] zelly|6 years ago|reply
[+] [-] vpEfljFL|6 years ago|reply
Docker is a good tool and deserves some resources to continue developing the product and funding marketing team. I wish docker hub to be more competitive among other rivals.
[+] [-] temptemptemp111|6 years ago|reply
[deleted]
[+] [-] mirimir|6 years ago|reply
For me, the hardest thing about Docker was running my own registries. Or rather, configuring credentials so Docker would use them. Not to mention the hassle of doing that through Tor.
So maybe Bocker isn't so picky about registries.
[+] [-] thebeardisred|6 years ago|reply
"Bob¹*, unless we've solely staffed the organization with folks from coder camps a lot of the folks here should be able to carve out a couple of namespaces in the kernel. That's not interesting and that's not why we asked CoreOS to come in today. Let's talk about the actual problems they're trying to solve."
<3 <3 <3 <3
That being said, it's great that folks continue to dig in to decompose the tooling and show what's actually happening behind the scenes.
¹Bob was definitely not that guy's name.
EDIT: Added note of context appreciating the work by the author.
[+] [-] zelly|6 years ago|reply
[+] [-] takeda|6 years ago|reply
Once you have that then you want to deploy it in whatever way you like it. Nixpkgs has functions to for example generate a docker image that contains only your app + essential dependencies. You could deploy it using nix package manager as you would install a typical app in the OS. You could also describe configuration of NixOS that has your application included and it could generate an image of it.
Community also came up with other generators [1].
[1] https://github.com/nix-community/nixos-generators
[+] [-] adgasf|6 years ago|reply
The advantage of Docker is that you can verify the container works locally as part of the build process rather than finding out it is broken due to some missing dep after a deployment. If you can verify that the image works then the mechanism for fetching the deps can be as scrappy as you like. Docker moves the dependency challenge from deployment-time to build-time.
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] mi100hael|6 years ago|reply
https://www.mgasch.com/post/scratch/
Typically a minimal Alpine distro image will allow you to pull in deps however you want (either via package manager, manual download, or copy), run a build, and then copy only the artifact over to a scratch image for a final image that's only a few MB in size.
https://medium.com/@chemidy/create-the-smallest-and-secured-...
[+] [-] gauthamzz|6 years ago|reply
[+] [-] logophobia|6 years ago|reply
[+] [-] songzme|6 years ago|reply
I wonder.... is there a service (Maybe a chrome extension) that lets you ask questions on the page you are in? For example, I want to highlight:
`btrfs-progs` and ask... "What is a quick summary about this package?
`ship a new enough version of util-linux` and ask.... "What features in the new util-linux allows this to work?"
Then maybe someone who also visits this page with more knowledge can answer my questions.
This tool would augment my learning 10x and I would be a paying customer. Does something like this exist?
[+] [-] lbj|6 years ago|reply
[+] [-] jerome-jh|6 years ago|reply
[+] [-] yepthatsreality|6 years ago|reply
[+] [-] sub7|6 years ago|reply
Just chroot the damn thing if you need to and keep it simple.
[+] [-] maitredusoi|6 years ago|reply
[+] [-] pantulis|6 years ago|reply
[+] [-] dang|6 years ago|reply