This really helps with the dev-to-production story for containers.
When people first started using Docker containers, we were promised things would run identically in dev and production - no more "but it worked on my laptop" issues. Then the rise of orchestrators meant that there again became a significant difference between running an app locally (in compose) and in production (on Kubernetes). Docker for Mac/Windows will now bridge that gap, giving me a k8s node to run against in dev.
Whilst Kubernetes has provided a great production orchestration solution, it never provided a great solution for development, meaning most users kept developing with Docker and Compose. It's great to see these worlds now coming together and hopefully leading to a first-class solution all the way from dev to prod.
in order to test your app in prod-like env you need to run prod-like env locally. ie. a k8s-cluster that is close enough to prod. for that you will have to at least simulate multi-node-setup and run all cluster-addons like in production.
i am excited about this move from docker but i don't think it will solve all the problems. i think once you have a bigger team it is worthwhile to run a second k8s-cluster besides prod where people can just test things on it. otherwise it is actually not that hard to run a local k8s-cluster with vagrant, not sure how docker wants to top that - i think there is no need to top vagrant.
Disregarding the swarm compatibility bit (which is irrelevant because swarm is kind of irrelevant), I don't really like what this "support" really means. As others mentioned minikube and k8s-cluster is already providing dev-to-prod compatibility.
Kubectl was already providing docker cli commands like "exec" "logs" etc. So you can execute some of these commands on a k8 cluster with the docker binary too? And why would you do that?
All I see is struggling for relevance, duplication of functionality and a very unnecessary vendor lock-in vector.
I see the same thing too: a struggle for relevance. The center of gravity has already moved onto k8s, even if it will takw a while for the rest of the developer community to catch up.
There are a lot more exciting work around Helm (packaging for K8S), third party extensions (plugins for K8S), and Operators (embedded managed services). The K8S community already has more contributios going towards the stack, and the three technology I just mentioned reduces enough friction for contributing innovations that k8s will likely phase-shift.
I just don't see Docker catching up. Sure, developers know them and think it has a good developer story. It doesn't. Docker for Mac and Window is practically useless when you have to struggle with file mounting for dev work. Docker Swarm is just not as robust as K8S, and the source of innovation going into Docker Swarm is coming from ideas from K8S.
Someone said it. ... K8S, not Docker, is the Linux of the cloud native platform.
Not really sure I see the vendor lock-in here. If you use Docker EE sure you're going to be locked in to their solutions to an extent but then that's true of adopting any commercial supported solution that provides layers on top of the base k8s clustering tech (e.g. Openshift).
The API is still k8s and the YAML files are identical, so migration at a technical level off that platform should be easy enough.
I think this move is about Docker maintaining the trajectory in enterprise where people want the management GUIs and extra features but where Kubernetes and particularly Openshift is making progress at the expense of Docker EE.
Great news for everyone except for VMWare (this is a simple compelling operating system for data centers that spans both windows and mac) and Openshift (which was one of the few viable ways of actually purchasing Kubernetes support). A lot of egos on both sides had to be suppressed to make this happen. Docker Swarm was a key driver in making Kubernetes popular because everyone realized that they needed swarm, but the implementation was so poor, no one could use it. That kicked K8s up into hyperdrive. Parts of the K8s community have been particularly partisan in doing everything they can to minimize docker.
Hopefully both sides _now_ come together and sing kumbaya, and we don't see a continuing KDE versus Gnome war a embrace and extend attitude by Docker or a continuing push to marginalize docker by the Kubernetes folks.
Surpressing egos usually means one has found a joker to beat the other in the fight for leadership. I don't think that battle is over yet, though. It's a very strong move Docker does here, but at the same time k8s is considering choosing another container engine as their main component. Currently at least in Enterprise k8s has a lot more traction than docker (I personally love docker more, but every day need to focus 99% of my effort on k8s because of that).
And given enterprise support Openshift is still ongoing as the best solution. They are afaik the only ones that offer a complete set of answers to most questions you can have in the PaaS space. Everybody else is like "here's an API, choose one of 3 billion plugins" (just thinking CNI here). In the end for the customer it doesn't matter though. Customers just want things to run smoothly and if possible reduce their maintenance work force. They don't want choices, they want solutions.
> or a continuing push to marginalize docker by the Kubernetes folks.
It's not like kubernetes folks very pushing to marginalize docker for bad reasons. docker runtime was one of the bottlenecks of kubernetes in production and docker inc didn't feel like improving it. Hopefully, it is changing now.
There are some technical decisions in docker that could provide similar functionality with better production support. I am not expert and just saying what I heard from dev working on it: for example, btrfs would work better than overlayfs for COW file system or docker would be better of using parts of systemd rather than implementing everything from scratch.
VMWare's k8s play is Pivotal Container Services, or PKS (blame Google). It's a three-way joint project between Pivotal, Google and VMWare.
I've said before and I'll say again: Red Hat, Microsoft and Google (and Pivotal and VMWare) are going to wind up making more money from Docker than Docker Inc does. Kubernetes has swept the field at the container-orchestrator level, the rest is a fight for the upper part of the stack.
Disclosure: I work for Pivotal, though not on PKS.
This seems a lot to me like Docker Inc. caving in to what has been painfully obvious for a while: K8s won and Swarm/Mesos lost the battle for hearts and minds in container orchestration. We can argue about why it happened, but I got the impression Docker Inc. were desperately trying to wish it away.
Now reality has intruded and I am glad, though I predict they'll continue to maintain that Swarm is a first class platform for a while, then quietly let it wither on the vine until one day it's forgotten about.
Also of note is Rancher 2.0's dropping support for Swarm and Mesos and focusing solely on K8S going forward.
Not sure why you would use Rancher if you have Mesos DC/OS? Mesos eclipses every significant feature, but I'd say Rancher is easier to set up initially.
There is some real meat here, and things that should have been done long ago. A key thing is that the Docker network drivers (libnetwork) are becoming CNI compatable. This will vastly simplify one of the worst aspects of setting up Kubernetes, and ensure a consistent network space across containers, even if a given container is not in kube. That's nothing but awesome.
Yeah setting up networking has been one of the worst parts of using Kubernetes in my limited experience -- just using default Docker networking would be sweet.
Have to say, I don't see the value in being able to have Swarm and k8s in the same cluster.
"Docker: powered by Kubernetes" seems to be more of a marketing thing to move down the value chain, and not be seen as a basic piece of infrastructure.
Agreed, just seems like marketing vaporspeak. Not exactly a trustworthy source, too.
For example, containerd has been moved to the CNCF and made a broader project. Fine, but Docker runs a separate fork of containerd anyways. A neat marketing sleight-of-hand, but to what end?
You can disable Swarm or Kubernetes at will in each Docker EE cluster. Hybrid is very useful for enterprises who already have to manage both - it was a highly requested feature.
So now the last question is how long does it take for AWS to finally abandon ECS and formally support K8S as a service? I think this makes it kind of slam dunk, but it is forcing aWS to give up a lot of proprietary lock in.
I see them stating "...for developers using Windows and macOS" but not mentioning Linux. I feel like I'm missing something in how I'm reading that page. How can I make use of this on Linux?
We're going to support Linux also. But we want to careful not to disrupt the users of the original container engine, as we transition it to Moby. In the future there will be a cleaner separation between "Docker CE, the developer tools" and "Moby engine, the open-source container engine". The last thing we want is someone to upgrade their production Linux engine to find an unexpected and unwanted kubernetes distribution wedged in.
That separation is already in place for Windows and Mac, so we're starting there.
Seems that people think Docker has gave in, but I am not that sure. If you can switch between Swarm/Kubernetes transparently, then why wouldn't you start with Swarm? (I'm talking about small companies who're just starting with containers.)
This will give IT organizations the option of getting an Enterprise supported distribution of Kubernetes from Docker.
Historically, most IT orgs requiring supported k8s have either gone cloud with something like Google Container Engine or gone with OpenShift and get support from RedHat. OpenShift is a fork if kubernetes though and lags a year or so behind. It also adds opinionated features such as Image Streams.
Docker's announcement said they were using "real" kubernetes, not a fork or a wrapper. I've setup kubernetes by hand before and it is no easy feat. I'm looking forward to evaluating Docker's solution and maintenance upgrade process.
UPDATE: My goal with this post is not to sell people one way or another, but moreso to explain where some of Docker's reasoning for this integration is coming from.
Don't take this the wrong way, but I love how we are already talking about "historically" in the context of Kubernetes enterprise support when the first release of Kubernetes was in 2014 and AFAICS pro-support is something from the last 1.5 years. Crazy world!
This. It's hard not to see Redhat (and to a lesser degree VMware) as the big looser of today's news. I absolutely want to see how this is implemented. Openshift's pricing is highway robbery.
Disclosure: my company is (was?) a big Openshift consumer...
I'll be interested in how they go too; productionising k8s is surprisingly tricky and Red Hat have made a heavy investment to do so.
I have skin in this game as well, as my other comments on this post demonstrate. We (Pivotal & Google) kinda skipped the hard bits of keeping up with k8s by packaging it as a BOSH release, so we'll always be up to date. That's actually one of the project goals: parity with vanilla k8s, because it is vanilla k8s, operated by a lower-level system.
How are they doing this? The big difference in swarm and kubernetes is the ingress. If this is seamless, then it has to be a batteries-included version of kubernetes with ingress, overlay network choice, etc already mapped out.
What are the details here?
Docker Swarm is beyond awesome and a great path for someone to scale up at the lower end of the scale spectrum (two containers). I really hope that this brings more people into Swarm.
I'm also keen to see what it means for the Kompose project.
From The Information's article "When Docker Said No to Google":
> In 2014, Google approached a startup called Docker proposing the two collaborate on software each was developing to help companies manage lots of complex applications, according to people with knowledge of the proposal. But Solomon Hykes, Docker’s founder and CTO, said no. He wanted to go it alone.
> Three years later, the cost of Mr. Hykes’ previously unreported decision is becoming apparent. The software that Google was developing was Kubernetes, an open-source product that now dominates its segment of the cloud software market. Docker’s rival software, Swarm, is also open-source but isn’t anywhere near as popular, two former Docker employees say.
That was my initial thought as well. CRI-O hits 1.0, and then this. To me, it comes across as an attempt to stay in the news. Possibly to start changing the narrative from Docker vs Kubernetes to Docker <3's Kubernetes.
The only problem I didn't solve yet is debugging python code running in kubernetes using pycharm. If I run a container in pycharm using "Debug..." dialog it launches inside "docker context" rather than "kubernetes context". For example, I can't connect to kubernetes services via their ClusterIP - the container launched via pycharm does not see it. The only solution I found is using docker compose to set up an environment similar to kubernetes and using docker compose from pycharm. Hopefully, this announcement from docker will simplify this story.
Kubernetes currently has a lot of traction in Enterprise world. Even the CEOs of the big corps currently know what Kubernetes is. That's why you need to support it in some way if you want to continue to compete. At least for the time being.
I think technically and in some regards also politically the Docker people are smarter though. In the most pro-docker comment I answered with "don't start to ignore k8s yet" and here I feel like saying "don't start to ignore docker(-swarm) yet".
The battle is ongoing and these two are both possible leaders.
Go with OVH, unlimited resource usage (including traffic) and they allow you to create/own your own private network (vRack) of dark fiber. Look into using multiple points of presence that they offer. If you don't need it nownow, wait for OVH to offer local US machines rather than just geolocated IPs.
You can do this well with bare metal servers or with a number of whatever dedicated and/or shared cloud stuff they offer.
[+] [-] amouat|8 years ago|reply
When people first started using Docker containers, we were promised things would run identically in dev and production - no more "but it worked on my laptop" issues. Then the rise of orchestrators meant that there again became a significant difference between running an app locally (in compose) and in production (on Kubernetes). Docker for Mac/Windows will now bridge that gap, giving me a k8s node to run against in dev.
Whilst Kubernetes has provided a great production orchestration solution, it never provided a great solution for development, meaning most users kept developing with Docker and Compose. It's great to see these worlds now coming together and hopefully leading to a first-class solution all the way from dev to prod.
[+] [-] user15128|8 years ago|reply
i am excited about this move from docker but i don't think it will solve all the problems. i think once you have a bigger team it is worthwhile to run a second k8s-cluster besides prod where people can just test things on it. otherwise it is actually not that hard to run a local k8s-cluster with vagrant, not sure how docker wants to top that - i think there is no need to top vagrant.
[+] [-] oweiler|8 years ago|reply
[+] [-] eggie5|8 years ago|reply
[+] [-] grabcocque|8 years ago|reply
[+] [-] stonewhite|8 years ago|reply
Kubectl was already providing docker cli commands like "exec" "logs" etc. So you can execute some of these commands on a k8 cluster with the docker binary too? And why would you do that?
All I see is struggling for relevance, duplication of functionality and a very unnecessary vendor lock-in vector.
[+] [-] hosh|8 years ago|reply
There are a lot more exciting work around Helm (packaging for K8S), third party extensions (plugins for K8S), and Operators (embedded managed services). The K8S community already has more contributios going towards the stack, and the three technology I just mentioned reduces enough friction for contributing innovations that k8s will likely phase-shift.
I just don't see Docker catching up. Sure, developers know them and think it has a good developer story. It doesn't. Docker for Mac and Window is practically useless when you have to struggle with file mounting for dev work. Docker Swarm is just not as robust as K8S, and the source of innovation going into Docker Swarm is coming from ideas from K8S.
Someone said it. ... K8S, not Docker, is the Linux of the cloud native platform.
[+] [-] raesene6|8 years ago|reply
The API is still k8s and the YAML files are identical, so migration at a technical level off that platform should be easy enough.
I think this move is about Docker maintaining the trajectory in enterprise where people want the management GUIs and extra features but where Kubernetes and particularly Openshift is making progress at the expense of Docker EE.
[+] [-] InTheArena|8 years ago|reply
Hopefully this is K8s, but with Swarm simplicity to get started.
[+] [-] InTheArena|8 years ago|reply
Hopefully both sides _now_ come together and sing kumbaya, and we don't see a continuing KDE versus Gnome war a embrace and extend attitude by Docker or a continuing push to marginalize docker by the Kubernetes folks.
[+] [-] erikb|8 years ago|reply
And given enterprise support Openshift is still ongoing as the best solution. They are afaik the only ones that offer a complete set of answers to most questions you can have in the PaaS space. Everybody else is like "here's an API, choose one of 3 billion plugins" (just thinking CNI here). In the end for the customer it doesn't matter though. Customers just want things to run smoothly and if possible reduce their maintenance work force. They don't want choices, they want solutions.
[+] [-] kozikow|8 years ago|reply
It's not like kubernetes folks very pushing to marginalize docker for bad reasons. docker runtime was one of the bottlenecks of kubernetes in production and docker inc didn't feel like improving it. Hopefully, it is changing now.
There are some technical decisions in docker that could provide similar functionality with better production support. I am not expert and just saying what I heard from dev working on it: for example, btrfs would work better than overlayfs for COW file system or docker would be better of using parts of systemd rather than implementing everything from scratch.
[+] [-] jacques_chester|8 years ago|reply
I've said before and I'll say again: Red Hat, Microsoft and Google (and Pivotal and VMWare) are going to wind up making more money from Docker than Docker Inc does. Kubernetes has swept the field at the container-orchestrator level, the rest is a fight for the upper part of the stack.
Disclosure: I work for Pivotal, though not on PKS.
[+] [-] grabcocque|8 years ago|reply
Now reality has intruded and I am glad, though I predict they'll continue to maintain that Swarm is a first class platform for a while, then quietly let it wither on the vine until one day it's forgotten about.
Also of note is Rancher 2.0's dropping support for Swarm and Mesos and focusing solely on K8S going forward.
[+] [-] jadbox|8 years ago|reply
[+] [-] InTheArena|8 years ago|reply
https://github.com/docker/libnetwork/pull/1978/files
[+] [-] zenlikethat|8 years ago|reply
[+] [-] ealexhudson|8 years ago|reply
"Docker: powered by Kubernetes" seems to be more of a marketing thing to move down the value chain, and not be seen as a basic piece of infrastructure.
[+] [-] squeed|8 years ago|reply
For example, containerd has been moved to the CNCF and made a broader project. Fine, but Docker runs a separate fork of containerd anyways. A neat marketing sleight-of-hand, but to what end?
[+] [-] shykes|8 years ago|reply
[+] [-] krallistic|8 years ago|reply
[+] [-] InTheArena|8 years ago|reply
[+] [-] HatchedLake721|8 years ago|reply
[+] [-] erikb|8 years ago|reply
[+] [-] bonsai80|8 years ago|reply
[+] [-] shykes|8 years ago|reply
That separation is already in place for Windows and Mac, so we're starting there.
[+] [-] sz4kerto|8 years ago|reply
[+] [-] caleblloyd|8 years ago|reply
Historically, most IT orgs requiring supported k8s have either gone cloud with something like Google Container Engine or gone with OpenShift and get support from RedHat. OpenShift is a fork if kubernetes though and lags a year or so behind. It also adds opinionated features such as Image Streams.
Docker's announcement said they were using "real" kubernetes, not a fork or a wrapper. I've setup kubernetes by hand before and it is no easy feat. I'm looking forward to evaluating Docker's solution and maintenance upgrade process.
UPDATE: My goal with this post is not to sell people one way or another, but moreso to explain where some of Docker's reasoning for this integration is coming from.
Disclosure: I work for a Docker partner
[+] [-] tnolet|8 years ago|reply
[+] [-] InTheArena|8 years ago|reply
Disclosure: my company is (was?) a big Openshift consumer...
[+] [-] tachion|8 years ago|reply
[+] [-] jacques_chester|8 years ago|reply
I have skin in this game as well, as my other comments on this post demonstrate. We (Pivotal & Google) kinda skipped the hard bits of keeping up with k8s by packaging it as a BOSH release, so we'll always be up to date. That's actually one of the project goals: parity with vanilla k8s, because it is vanilla k8s, operated by a lower-level system.
[+] [-] sandGorgon|8 years ago|reply
What are the details here?
Docker Swarm is beyond awesome and a great path for someone to scale up at the lower end of the scale spectrum (two containers). I really hope that this brings more people into Swarm.
I'm also keen to see what it means for the Kompose project.
[+] [-] wyc|8 years ago|reply
> In 2014, Google approached a startup called Docker proposing the two collaborate on software each was developing to help companies manage lots of complex applications, according to people with knowledge of the proposal. But Solomon Hykes, Docker’s founder and CTO, said no. He wanted to go it alone.
> Three years later, the cost of Mr. Hykes’ previously unreported decision is becoming apparent. The software that Google was developing was Kubernetes, an open-source product that now dominates its segment of the cloud software market. Docker’s rival software, Swarm, is also open-source but isn’t anywhere near as popular, two former Docker employees say.
https://www.theinformation.com/when-docker-said-no-to-google (Sorry...it's paywalled :/)
[+] [-] InTheArena|8 years ago|reply
[+] [-] alexlarsson|8 years ago|reply
[+] [-] iamdeedubs|8 years ago|reply
[+] [-] kozikow|8 years ago|reply
The only problem I didn't solve yet is debugging python code running in kubernetes using pycharm. If I run a container in pycharm using "Debug..." dialog it launches inside "docker context" rather than "kubernetes context". For example, I can't connect to kubernetes services via their ClusterIP - the container launched via pycharm does not see it. The only solution I found is using docker compose to set up an environment similar to kubernetes and using docker compose from pycharm. Hopefully, this announcement from docker will simplify this story.
[+] [-] familyit|8 years ago|reply
[+] [-] malaporte|8 years ago|reply
[+] [-] andy_ppp|8 years ago|reply
[+] [-] erikb|8 years ago|reply
I think technically and in some regards also politically the Docker people are smarter though. In the most pro-docker comment I answered with "don't start to ignore k8s yet" and here I feel like saying "don't start to ignore docker(-swarm) yet".
The battle is ongoing and these two are both possible leaders.
[+] [-] why-el|8 years ago|reply
[+] [-] joevandyk|8 years ago|reply
[+] [-] chmielewski|8 years ago|reply
https://github.com/antoineco/kOVHernetes
Go with OVH, unlimited resource usage (including traffic) and they allow you to create/own your own private network (vRack) of dark fiber. Look into using multiple points of presence that they offer. If you don't need it nownow, wait for OVH to offer local US machines rather than just geolocated IPs.
You can do this well with bare metal servers or with a number of whatever dedicated and/or shared cloud stuff they offer.