[iTokio]

Isn’t that limited to a single node?

How would you configure a cluster? I’m trying to explore lightweight alternatives to kubernetes, such as docker swarm, but I think that the options are limited if you must support clusters with equivalent of pods and services at least.

[mands]

I've found you can get pretty far with a couple of fixed nodes and scaling vertically before bringing in k8s these days.

Right now I'm running,

- podman, with quadlet to orchestrate both single containers and `pods` using their k8s-compatible yaml definition

- systemd for other services - you can control and harden services via systemd pretty well (see https://news.ycombinator.com/item?id=44937550 from the other day). I prefer using systemd directly for Java services over containers, seems to work better imo

- Pyinfra (https://pyinfra.com/) to manage and provision the VMs and services

- Fedora CoreOS as an immutable base OS with regular automatic updates

All seems to be working really well.

[jabl]

> Isn’t that limited to a single node?

Yes. Though unless you have a very dynamic environment maybe statically assigning containers to hosts isn't an insurmountable burden?

[sc68cal]

> How would you configure a cluster?

So, unless you have a service that requires a fixed number of running instances that is not the same count as the number of servers, I would argue that maybe you don't need Kubernetes.

For example, I built up a Django web application and a set of Celery workers, and just have the same pod running on 8 servers, and I just use an Ansible playbook that creates the podman pod and runs the containers in the pod.

[gf000]

In the off chance your search didn't expand to k3s, I can semi-recommend it.

My setup is a bit clunky (having a Hetzner cloud instance as controller and a local server as a node throught Tailscale), from which I get an occasional strange error that k3s pods fail to resolve another pod's domain without me having to re-create the DNS resolver system pod, and that I so far failed at getting Velero backups to work with k3s's local storage providers, but otherwise it is pretty decent.

[iTokio]

K3s is light in terms of resources, but heavy in operational complexity, I’m not looking for a smaller version of kubernetes but for a simple way to run container backed services when you’re not google but a small company, something that has few moving parts but is very reliable and low maintenance.

[MrDrMcCoy]

HashiCorp Nomad is probably the only real alternative. It's what in using, and I like it better than the overcomplexity of k8s.

[pianopatrick]

I've been reading and watching videos about how you can use Ansible with Podman as a simpler alternative to Kubernetes. Basically Ansible just SSHs into each server and uses podman to start up the various pods / containers etc. that you specify. I have not tried this yet though so take this idea with a grain of salt.

[mdaniel]

whew, "alternative" is doing a lot of work there.

Contrast:

  ansible -i server1,server2,server3 deploy_fake_pods.yaml
  ssh server1 sudo shutdown -h now
  # aww, too bad, now your pods on server1 are no longer

With

  kubectl apply -f deployment.yaml
  for i in $(kubectl get nodes -o jsonpath='{.status.hostIP}'); do
    ssh $i sudo shutdown -h now
    sleep 120
  done
  # nothing has changed except you have fresh Nodes

If you don't happen to have a cluster autoscaler available, feel free to replace the for loop with |head -1 or a break, but I mean to point out that the overall health and availability of the system is managed by kubernetes, but ansible is not that

[osigurdson]

>> lightweight alternatives to kubernetes

microk8s seems exceedingly simple to setup and use. k3s is easy as well.

[zelphirkalt]

I once tried Nomad for a very brief moment. Not sure if it fits your bill.

[0x457]

Nomad is weird. Its OSS version is like a very limited trial of paid version. At least last time I tried it. To a point that it was more productive for me to install k3s instead.