It depends on your size. For a fairly minimal close-to-best-practices you'll need for each DC, each on a separate physical host (I may be missing something):
3 x Consul server
3 x Nomad server
2/3 x Vault server
It's long since I operated k8s but IIRC I think you can get similar capabilities and redundancy with 3-5 machines?
That's before you start looking at actual runner nodes, load balancers, proxies, logging and monitoring infra, etc...
Unless you cheat (which I think many do) or you're big enough, that overhead can be meaningful.
FWIW we recognized this was too much overhead for many users. Nomad 1.3 supports service discovery so you can start without Consul, and 1.4 will support secure variables to get folks farther along without requiring Vault.
So 3 Nomad servers should give you a pretty featureful and highly available cluster these days.
Entirely true, but I also think that neither k8 nor Nomad are that useful if you're not at a scale where the above is negligeable? It costs roughly 500 usd a month on aws for those 9 servers.
3np|3 years ago
That's before you start looking at actual runner nodes, load balancers, proxies, logging and monitoring infra, etc...
Unless you cheat (which I think many do) or you're big enough, that overhead can be meaningful.
schmichael|3 years ago
FWIW we recognized this was too much overhead for many users. Nomad 1.3 supports service discovery so you can start without Consul, and 1.4 will support secure variables to get folks farther along without requiring Vault.
So 3 Nomad servers should give you a pretty featureful and highly available cluster these days.
nicolas_t|3 years ago