Good news about zombies: Kubernetes will soon solve this by having the pause container (which is automatically included in every pod) automatically reap children. [1]
Note that this change depends on the shared PID namespace support, which a larger, still-ongoing endeavour [2].
The zombie reaping problem is fixed in Docker. You can simply use `docker run --init` with no argument, and it will spawn a tiny init that will reap children processes correctly. We don't enable the flag by default to respect backwards compatibility.
This is an excellent check-list of both kubernetes and docker gotchas to avoid.
Coming into the k8s ecosystem with very little container experience has been a steep learning curve, and simple, concrete suggestions like this go a LONG way to leveling it out.
We've also published some other workshops for Docker and Kubernetes that we take customers through when onboarding (if needed): https://github.com/gravitational/workshop
Feel free to take them for a spin and feedback welcome and appreciated.
I would like to have seen more "patterns" regarding configuration.
Right now, we have a bunch of microservices. Most of them talk to our shared infrastructure. We started with single configuration file, which has grown to monstrous proportions, and is mounted on every pod as a config map.
What would be the correct approach? Multiple configmaps with redundant information are just as bad, if not worse.
Start by using done service discovery. That way service names stay the same but service implementations and locations can move.
Then ship the static list of names (should be short) and per-service credentials (highly highly recommended).
Another pattern is co-locating a proxy with your app. See e.g. linkerd on how to do that. This will also unify the handling of circuit breakers and connection pools across services - even without any shared code!
Have you tried out istio yet? It's the packaging of Lyft's Envoy that Google and IBM are putting together to handle your last two points, circuit breaking and rate limiting and much more.
Word of caution: Istio isn't production ready yet, still a lot of missing stuff and bugs. Definitely worth playing with though, once it's ready I think it's going to make managing microservices much easier.
Some background on these workshops: we (Gravitational) help SaaS companies package their applications into Kubernetes, this makes them deployable into on-premise environments [1]. This in itself is an unexpected and quite awesome benefit of adopting Kubernetes in your organization: your stack becomes one-click portable.
Depends on the tool in question. jq or awk are such common-place and light on dependencies that it's indeed unnecessarily complex.
The benefit is when a one-time tool is heavy on dependencies. For example, with OpenStack Swift (an S3-like object storage), a common one-time task is swift-ring-builder, which takes an inventory of available storage and creates a shared configuration file which describes how data is shared between storages. That's something you would run on a sysadmin's notebook, but it's included with Swift itself, so you would have to install a bunch of Python modules into a virtualenv.
In that case, it's probably easier to just use the Docker image for Swift that you have anyway, and run swift-ring-builder from there.
we use kubernetes, helm and gitlab.. runtime configuration lives in each repo next to code - values.yaml, dev.yaml, test.yaml, prod.yaml to store applications runtime configuration -- each environment is host to 40+ redundant services.. its working quite well but has required a pretty big upfront investment... surprised there was much discussion about monitoring- prometheus and grafana work well for that
No. If the node the pod is scheduled to goes away, the pod will not be restarted. 'restartPolicy: Always' applies to the containers in the pod, not the pod itself. Deployments (Or daemonsets, jobs, replicasets or replication controllers) actively maintain the pods, to assure in the event that the pods are deleted they're replaced.
Other posters have already commented on this but, if you are not using deployments (or at least, replication controllers), stop what you are doing right away and fix that. Otherwise you lose one of the biggest advantages of k8s.
[+] [-] atombender|8 years ago|reply
Note that this change depends on the shared PID namespace support, which a larger, still-ongoing endeavour [2].
[1] https://github.com/kubernetes/kubernetes/commit/81d27aa23969...
[2] https://github.com/kubernetes/kubernetes/issues/1615
[+] [-] shykes|8 years ago|reply
[+] [-] lemoncucumber|8 years ago|reply
[+] [-] CurtMonash|8 years ago|reply
I guess Craster was right after all.
:)
[+] [-] web007|8 years ago|reply
Coming into the k8s ecosystem with very little container experience has been a steep learning curve, and simple, concrete suggestions like this go a LONG way to leveling it out.
[+] [-] twakefield|8 years ago|reply
Feel free to take them for a spin and feedback welcome and appreciated.
[+] [-] mino|8 years ago|reply
I browsed it and immediately bookmarked to have a ready "here, read this first" answer :)
[+] [-] outworlder|8 years ago|reply
Right now, we have a bunch of microservices. Most of them talk to our shared infrastructure. We started with single configuration file, which has grown to monstrous proportions, and is mounted on every pod as a config map.
What would be the correct approach? Multiple configmaps with redundant information are just as bad, if not worse.
[+] [-] treffer|8 years ago|reply
Then ship the static list of names (should be short) and per-service credentials (highly highly recommended).
Another pattern is co-locating a proxy with your app. See e.g. linkerd on how to do that. This will also unify the handling of circuit breakers and connection pools across services - even without any shared code!
[+] [-] majewsky|8 years ago|reply
https://github.com/kubernetes/helm
[+] [-] lclarkmichalek|8 years ago|reply
Edit: oh, you kind of do. Well, it's not upcoming any more, it's in the latest Docker CE :)
[+] [-] moondev|8 years ago|reply
[+] [-] twakefield|8 years ago|reply
[+] [-] bryanlarsen|8 years ago|reply
[+] [-] thesandlord|8 years ago|reply
[+] [-] Langhalsdino|8 years ago|reply
If some of you are interested in Kubernetes GPU cluster for deep learning, this article might be good to read as well. https://news.ycombinator.com/item?id=14526807
[+] [-] pooktrain|8 years ago|reply
The k8s blog has some as well: http://blog.kubernetes.io/2016/06/container-design-patterns....
[+] [-] throwaway34802|8 years ago|reply
https://habitat.sh
https://www.youtube.com/watch?v=-yTeXCY3iM0
[+] [-] old-gregg|8 years ago|reply
[1] http://gravitational.com/telekube
[+] [-] the_common_man|8 years ago|reply
[+] [-] nunez|8 years ago|reply
i've seen this pattern before and it didnt make me feel very good. it reeks of unnecessary complexity.
[+] [-] majewsky|8 years ago|reply
The benefit is when a one-time tool is heavy on dependencies. For example, with OpenStack Swift (an S3-like object storage), a common one-time task is swift-ring-builder, which takes an inventory of available storage and creates a shared configuration file which describes how data is shared between storages. That's something you would run on a sysadmin's notebook, but it's included with Swift itself, so you would have to install a bunch of Python modules into a virtualenv.
In that case, it's probably easier to just use the Docker image for Swift that you have anyway, and run swift-ring-builder from there.
[+] [-] m0rganic|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] eldios|8 years ago|reply
[+] [-] humanfromearth|8 years ago|reply
Kind of.. but you can set `restartPolicy: Always` and will always restart in case of failure.
[+] [-] GauntletWizard|8 years ago|reply
[+] [-] outworlder|8 years ago|reply