This is getting a little long in the tooth but I created a few videos and did some tests around cgroups back in 2013-2014. Both these help explain the underpinnings of Docker and I guess Kubernetes now too. Turtles all the way up. I'm just mentioning this since these are still some of my most popular videos and still the core tech under the hood.
A personal note here too. cgroups were invented at Google in the early 2000's. If you're using search today, gmail, docs, maps, etc. You're using cgroups. It sounds simple but this tech really powered a whole wave of innovation/startups/projects that almost everyone interacts with on a daily basis. Either through touching a Google service or interacting with anyone using Docker or Kubernetes (running on cgroups/namespaces). Pretty impressive.
#14 - Introduction to Linux Control Groups (Cgroups)
cgroups were invented at Google in the early 2000's.
In my understanding, most of the initial commits (at least for LXC, which was the initial userspace) came from IBM, who funded it with interest in kernel-level resource balkanization for their largest mainframes. Google's kubernetes only appeared post-facto after docker, which itself was basically an lxc wrapper for a long time.
Source: I corresponded with two of the authors ~2009, was an early lxc user, and provided some security fixes for early docker.
Kubernetes is just another layer on top of Docker. It can use other container execution environments, but Kubernetes itself doesn't manage the container runtime; it just orchestrates containers at a higher level. If you're running Kubernetes, you're almost always still running Docker.
These things should strive for accuracy and this article is not accurate. The cgroups facility does not control "the number of CPU shares per process." Although you can put such a thing into effect with control groups, it's more accurate to say that a control group limits the resources of a set of tasks. Those tasks may be from one or several processes, and it's also the case that a single process can divide its own tasks into several cgroups.
While I agree that we should strive for accuracy, you're really splitting hairs there. Plus the author puts a disclaimer in there "NOTE: This is a gross simplification and is NOT technically accurate should you be looking to get involved with the underlying cgroup code. The above explanation is meant for clarity of understanding."
Someone managed to apply part three ? Systemd gets in the way for me.
Edit: well, at least part four, finally, properly document how to cope with systemd. And guess what, it is different than 90% of documentation on cgroups on the internet.
Found an interesting ongoing blogpost series that also tries to explain all the other low level kernel mechanism that make up Docker and other container technologies on Linux. They haven't reach the topic of control groups though.
I wrote a blog post on cgroups a couple years ago that's still accurate and goes further into depth and gives workable examples of using them both inside and outside of containers: https://www.grant.pizza/blog/understanding-cgroups/
Ah yes, "cgroups", according to notable no-nonsense kernel hacker Al Viro:
"it's not just badly written kernel/cgroup.c - the interfaces on both sides (userland and the rest of kernel) are seriously misdesigned. As far as I'm concerned, configuring it out solves my problem nicely."
That was in 2011, so things might have improved. What remains however is that cgroups was added to the kernel, by Googlers, for easier maintenance, but with an implicit understanding that no sane person would actually make use of it to do something important.
It's true that a better userspace API for cgroups might have involved something more like ioctl(), rather than a filesystem with text-based control files. But people had been trying to get such APIs into the kernel for years with no success. Which is why we tried to piggyback on something existing.
Cpusets, which had already been accepted into the kernel a year or two previously, provided the basic userspace API for cgroups. It was just expanded to support control files for more resource types. (And multiple resource hierarchies, although that's gone in cgroups v2 I think).
[+] [-] WestCoastJustin|5 years ago|reply
A personal note here too. cgroups were invented at Google in the early 2000's. If you're using search today, gmail, docs, maps, etc. You're using cgroups. It sounds simple but this tech really powered a whole wave of innovation/startups/projects that almost everyone interacts with on a daily basis. Either through touching a Google service or interacting with anyone using Docker or Kubernetes (running on cgroups/namespaces). Pretty impressive.
#14 - Introduction to Linux Control Groups (Cgroups)
https://sysadmincasts.com/episodes/14-introduction-to-linux-...
#24 - Introduction to Containers on Linux using LXC
https://sysadmincasts.com/episodes/24-introduction-to-contai...
[+] [-] contingencies|5 years ago|reply
In my understanding, most of the initial commits (at least for LXC, which was the initial userspace) came from IBM, who funded it with interest in kernel-level resource balkanization for their largest mainframes. Google's kubernetes only appeared post-facto after docker, which itself was basically an lxc wrapper for a long time.
Source: I corresponded with two of the authors ~2009, was an early lxc user, and provided some security fixes for early docker.
[+] [-] caymanjim|5 years ago|reply
[+] [-] obiwanpallav1|5 years ago|reply
[1]https://www.redhat.com/sysadmin/cgroups-part-four
[+] [-] FedericoRazzoli|5 years ago|reply
Yes, Kubernetes uses cgroup too.
Systemd units are also based on cgroups.
[+] [-] tonetheman|5 years ago|reply
[+] [-] tofflos|5 years ago|reply
[+] [-] LeCow|5 years ago|reply
Thank you.
[+] [-] jeffbee|5 years ago|reply
[+] [-] stuff4ben|5 years ago|reply
[+] [-] waynesonfire|5 years ago|reply
[+] [-] aschatten|5 years ago|reply
[+] [-] ldng|5 years ago|reply
Edit: well, at least part four, finally, properly document how to cope with systemd. And guess what, it is different than 90% of documentation on cgroups on the internet.
[+] [-] flexops|5 years ago|reply
https://www.schutzwerk.com/en/43/posts/linux_container_intro...
[+] [-] FedericoRazzoli|5 years ago|reply
[+] [-] grantseltzer|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] musicale|5 years ago|reply
I also tend to think that processes in cgroups are a sweet spot of lightweight containerization that can do quite a bit.
[+] [-] ape4|5 years ago|reply
[+] [-] dianhuji1|5 years ago|reply
[+] [-] meatspaceping|5 years ago|reply
[+] [-] craigsmansion|5 years ago|reply
"it's not just badly written kernel/cgroup.c - the interfaces on both sides (userland and the rest of kernel) are seriously misdesigned. As far as I'm concerned, configuring it out solves my problem nicely."
That was in 2011, so things might have improved. What remains however is that cgroups was added to the kernel, by Googlers, for easier maintenance, but with an implicit understanding that no sane person would actually make use of it to do something important.
... enter SystemD.
[+] [-] menage|5 years ago|reply
Cpusets, which had already been accepted into the kernel a year or two previously, provided the basic userspace API for cgroups. It was just expanded to support control files for more resource types. (And multiple resource hierarchies, although that's gone in cgroups v2 I think).
[+] [-] jeffbee|5 years ago|reply