[caymanjim]

It's unclear to me why running Docker directly in Proxmox (it's just Debian) and using it like any other Docker host is a bad idea, and why this extra layer of abstractions is preferable.

Docker has security issues if you're not careful, and it's frankly kind of a shitshow out of the box with defaults. Maybe that's part of the reason. But I struggle to see how a bespoke solution like this is the right answer.

[tristanj]

Proxmox is a hypervisor OS, and its value comes from its virtualization and container-management features. These features include being able to pause, resume, snapshot, backup/restore from snapshot, and live-migrate VMs or LXCs to another server in just a couple hundred milliseconds of downtime. Once you run docker on the hypervisor itself, you lose these features, which defeats the purpose of running Proxmox in the first place.

There's also the security angle. Containers managed by Proxmox are strongly isolated from the host, but containers running on Docker sidestep this isolation model. Docker is not insecure by design, but it greatly increases the attack surface. If the hypervisor gets compromised, the entire cluster of servers will also get compromised. In general, as little software as possible should be installed on the host.

[SirMaster]

What's the reason Proxmox can't just implement Kubernetes on the host for running docker across a set of Proxmox nodes though. I mean they implemented a system like Ceph for distributed storage.

[nicman23]

still then it would have been just a process in a namespace. there are ways to dump a process and then resume it

[dboreham]

It's a kind of apples vs pears problem:

You have a bunch of tooling that deals with apples. You have a clear conceptual picture of what an apple is and what it does.

Then someone brings you a pear. It's kind of like an apple but not exactly. Their pear however works well with some other toolscape that's beyond the shire. You want to do things with their pears.

You invent a way to put a pear inside an apple (docker in VM). That works but you lose some functionality and break some stuff in the conversion, plus now you don't have the clean conceptual integrity of your apple-only system.

This is a way to transform a pear into an apple.

[simcop2387]

Largely management, observability, and then the way that docker mucks with firewalls. Running them this way will allow proxmox to handle all that in the same way {I assume) as the LXC and VMS so automation, and all the rest can be consistent

[j1elo]

What about running Docker inside a Proxmox LXC container? Is that a common practice? Intuitively doing that would have a lower baseline resource usage than using a full-blown VM for Docker containers.

[zatarc]

I've been running Docker natively on the host since Proxmox 7. The only major problem was an iptables rule that I had to add so that the containers are accessible from outside. Besides that, it runs smoothly.