top | item 14156954

A new upstream project to break up Docker into independent components

571 points| lclarkmichalek | 9 years ago |github.com | reply

305 comments

order
[+] shykes|9 years ago|reply
Hi all, Docker founder here. It appears that the explanation in the pull request is not very clear. Sorry about that. We didn't expect the PR to be the first thing people read: there is a new website at mobyproject.org. Unfortunately Github is struggling under the load, and I can't update the PR text to clarify.

Here is an updated version with some clarifications:

<<<

Docker is transitioning all of its open source collaborations to the Moby project going forward. During the transition, all open source activity should continue as usual.

IMPORTANT NOTE: if you are a Docker user, this does not change anything. Docker CE continues to exist as a downstream open-source product, with exactly the same interface, packages and release cycle. This change is about the upstream development of the components of Docker, and making that process more open and modular. Moby is not a replacement for Docker: it's a framework to help system engineers build platforms like Docker out of many components. We use Moby to build Docker, but you can use it to build specialized systems other than Docker.

You can learn more at http://mobyproject.org

We are proposing the following list of changes:

- splitting up the engine into more open components - removing the docker UI, SDK etc to keep them in the Docker org - clarifying that the project is not limited to the engine, but to the assembly of all the individual components of the Docker platform - open-source new tools & components which we currently use to assemble the Docker product, but could benefit the community - defining an open, community-centric governance inspired by the Fedora project (a very successful example of balancing the needs of the community with the constraints of the primary corporate sponsor)

>>>

EDIT: I'm happy to answer any follow-up questions here, if it helps clarify.

[+] ilovecaching|9 years ago|reply
Do you know that keeping your users in a constant state of confusion is not the way to convince them that you're building a stable and secure product?

Also, isn't it kind of ironic that you built your company on OSS and then invited a well known destroyer of OSS onto your main stage?

Plus, I want my DockerCon money back. What exactly did you announce besides multi-stage builds at the general sessions that is actually benefiting Docker users? I don't care about your plumbing and constant rebranding, other than finding it very discouraging, I want to know what you're actually doing with your product this year. I guess nothing.

[+] lewq|9 years ago|reply
Together with Solomon and a core group of maintainers, we drafted some words to clarify the relationship between Moby and Docker at the Moby Summit here at DockerCon. We're updating the website at mobyproject.org to include this text as well.

Docker is, and will remain, a open source product that lets you build, ship and run containers. It is staying exactly the same from a user's perspective. Users can download Docker from the docker.com website.

Work has been ongoing to break Docker into modular components for some time, with runc and containerd as examples. Containerd for example has been donated to the CNCF. We are now completing this work with the goal being that the monolithic docker repo eventually ceases to exist, instead being assembled from a set of components.

Moby is a project which provides a "Lego set" of dozens of components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.

Docker the product will be assembled from components that are packaged by the Moby project.

The Moby project provides a command-line tool called moby which assembles components. Currently it assembles bootable OS images, but soon it will also be used by Docker for assembling Docker out of components, many of which will be independent projects.

As the Docker Engine is split up into multiple components the Moby project will also be the home for those components until a more appropriate location is found.

[+] alsadi|9 years ago|reply
Tell us more about trademark and branding guidelines. What do you allow a downstream implemented by others to be called without requiring trademark fees ex by parties similar to amazon (public cloud) or ovh (data center) or redhat/fedora (software vendor) and even nvidia (cuda docker-like) . For example fedora and firefox has strict trademark policy eventhough they are opensource.

Can I call my tool moby-something?

[+] crb|9 years ago|reply
Hey Solomon, here are some 'starters for 10':

- Who will be able to use the Docker trademark? If someone built a fork of Docker CE from Moby, can they still call it Docker? What about distribution packagers?

- Right now a single binary ('docker') is used to build images and runs clusters (among other things). As the monolith is broken down, will the new binaries still be named/branded 'docker'? Will there be rules about packaging only some of them? Will Docker CE/EE still have a single binary for all purposes?

[+] zacblazic|9 years ago|reply
Many people really love the name Docker.

Could you explain the reasoning behind the need for a new name?

I ask this because would it not have been a better choice to name the other releases differently.

For example:

  Docker (docker/docker) = open source development.
  Docker CE (docker/docker-community) = free product release
  Docker EE (docker/docker-enterprise) = commercial product release based on Docker CE (private).
The components that the monolith is being broken into could just live beside the other repos in the docker organization:

  docker/component-1
  docker/component-2
  docker/component-3
Thoughts?
[+] zabcik|9 years ago|reply
So moby now uses containerd, which I guess is not related to systemd even though systemd also has its own container system. Are there any components to moby that are OSS and a part of the linux stack or is this whole endeavor to break away from linux dependencies?
[+] fpp|9 years ago|reply
Are you not concerned that this move (relabeling the open source version different from how the product is known to the broader public) might further diminish the number of contributors that are now already down 50% / back to 2014 levels?

see https://www.openhub.net/p/docker for statistics on that.

[+] dastbe|9 years ago|reply
Very tangential: do you have any concerns about or plans to buy mobyproject.com? seems like its oscillating between parked domain and shady ads.
[+] nobodyorother|9 years ago|reply
> Unfortunately Github is struggling under the load

What the flip did you do? They held off the Great Firewall!

/impressed

[+] mintplant|9 years ago|reply
Could you provide a succinct, up-to-date explanation of what constitutes "Docker" as a product?
[+] registrator|9 years ago|reply
If moby is not a replacement for docker (your words), why does docker/docker redirect to moby/moby (your actions)?
[+] frik|9 years ago|reply
So all books about Docker are now outdated.

What will happen with hub.docker.com? Is store.docker.com the replacement? Or will both coexist? Which one will be hosted on mobyproject.org ?

[+] zobzu|9 years ago|reply
Is there a "im stupid and didnt follow news, here's why docker is being renamed to moby:"?

Such as "docker reputation for security became too terrible" or whatever.

[+] glitcher|9 years ago|reply
Excerpts from mobyproject.org give a much better explanation:

Moby is an open framework created by Docker to assemble specialized container systems without reinventing the wheel. It provides a “lego set” of dozens of standard components and a framework for assembling them into custom platforms.

Audience

Moby is recommended for anyone who wants to assemble a container-based system, this includes:

Hackers who want to customize or patch their Docker build

System engineers or integrators building a container system

Infrastructure providers looking to adapt existing container systems to their environment

Container enthusiasts who want to experiment with the latest container tech

Open-source developers looking to test their project in a variety of different systems

Anyone curious about Docker internals and how it’s built

Moby is NOT recommended for:

Application developers looking for an easy way to run their applications in containers. We recommend Docker CE instead.

Enterprise IT and development teams looking for a ready-to-use, commercially supported container platform. We recommend Docker EE instead.

Anyone curious about containers and looking for an easy way to learn. We recommend the docker.com website instead.

[+] ernestbro|9 years ago|reply
Moby is the fedora like oss project and docker is the RHEL like commercial product. Purely a clever rebranding job. Timing is perfect - now that enterprise is adopting the technology, when you tell your boss you adopted moby he/she will say- but the CTO wants to be seen as being innovative for adopting this new technology and ita called docker not moby!
[+] ChristianGeek|9 years ago|reply
Thank you for ELI5! (The official explanation made no sense to me at all.)
[+] jacquesm|9 years ago|reply
> Enterprise IT and development teams looking for a ready-to-use, commercially supported container platform. We recommend Docker EE instead.

I'm sure they do. If I were involved in a situation like that I would definitely check this out, just to make sure that their recommendation is not just a way to protect their income stream. 'Don't look there' is the forbidden fruit theory at work, of course everybody will look there.

[+] sandGorgon|9 years ago|reply
@shykes - so im one of the lonely few that love Docker Swarmkit (not Swarm - cos, that would be too confusing right !). I have spent a lot of time in the Kubernetes ecosystem (including several SIG) and value the batteries-included setup that Docker Swarm brings.

More recently, I was part of a discussion where Kubernetes has decided to roll its own logging infrastructure, while leaving important parts like log rotation still undecided.... driving me even closer to the Docker ecosystem.

However, it seems that Swarmkit is a big casualty of the whole CE/EE (and now Moby) split. Docker EE (https://www.docker.com/enterprise-edition) still does not mention Swarmkit anywhere. In fact, there is NO top-level page on Docker's own site that actually mentions Swarmkit [1] and even Docker "Swarm-mode" is part of some second level documentation and help pages.

It makes me really worried about how you are seeing Swarm-mode/Swarmkit as part of the long term future of Docker Inc. You guys really dont talk about it - not even on twitter.

I'm assuming this is your handle as well (https://twitter.com/dockerswarm), which has not seen a tweet since Jan 2017. However, you guys do tweet a lot about "Docker Datacenter".

What's going on ?

[1] https://www.google.co.in/webhp?sourceid=chrome-instant&ion=1...

[+] alexk|9 years ago|reply
> More recently, I was part of a discussion where Kubernetes has decided to roll its own logging infrastructure, while leaving important parts like log rotation still undecided.... driving me even closer to the Docker ecosystem.

Can you please refer to this particular discussion and link to it here, I would like to make sure that this indeed is what happened.

[+] bru|9 years ago|reply
More information can be found there: https://github.com/moby/moby/pull/32691

> Docker is transitioning all of its open source collaborations to the Moby project going forward.

Should I understand that the core team wants to keep the brand "Docker" but use it in a commercial way, while Moby will be the underlying open source code?

Is it Docker/Moby = RHEL/Fedora ? or Docker/Moby = Mongodb.com/Mongodb.org?

[+] pi-rat|9 years ago|reply
From comment by @stykes on the pull request:

  Moby = open source development
  Docker CE = free product release based on Moby
  Docker EE = commercial product release based on Docker CE.

  Nothing is dead; and everything that was open-source remains open-source. In fact we are open-sourcing new things.
[+] kibwen|9 years ago|reply
Indeed, it seems that github.com/docker/docker now redirects to this. From a technological perspective it's irrelevant, but from a brand management perspective it's baffling. Did they perceive the Docker brand as tainted in some way?
[+] bonesss|9 years ago|reply
The Docker brand is "tainted" in the eyes of their commercial competitors who don't want to use shared tools/libraries.

By creating an independent project commercial actors in the Enterprise space can share resources, components, and standards without having to explain why they`re better than Docker even though they`re built on top of Docker.

This is a great move towards industry and container infrastructure orchestration standardization. Common APIs will go a long way to widespread container adoption.

[+] pilif|9 years ago|reply
> Did they perceive the Docker brand as tainted in some way?

quite to the contrary I'd say. Docker will be their commercial enterprise offering and moby will be the open source project.

[+] yeukhon|9 years ago|reply
This is the one of the worst kind of business decision one can make. But I am not theirCTO and CEO. While not the same as Vagrant being replaced (and then revived because the alternate project was a burden), feels the same shitty decision IMO. Plently of people run successful OSS under the same name. Disappointing.
[+] bonesss|9 years ago|reply
I believe you might be reading the situation backwards...

Dockers underlying components are quite usable for many solutions in other projects and platforms. Now, instead of having "Docker TM" components in their stack, OSS projects can connect the tools without seeming to support a specific commercial provider.

Presently there is a lot of confusion and OSS projects are hesitant to use Docker components because of the perception of a closed eco-system. Now the project has clearer commercial branding and a more appealing package set for OSS consumers.

Remember: many of the most important actors we need to collaborate in this space are direct commercial competitors to Docker Inc.

[+] exelius|9 years ago|reply
> Plently of people run successful OSS under the same name. Disappointing.

But those projects aren't true OSS in the way that something like Docker / Moby needs to be. Your Vagrant example is one of them; Vagrant is a useful tool but ultimately too narrowly-focused. Docker's biggest coup was seamlessly integrating orchestration tools with deployment tools -- and now they're separating the two (which is completely fine, because they need to be separate tools anyway to support complex use cases).

Docker has built a lot of valuable IP that they want to protect (branding, UI, API, etc.) That makes a lot of sense. It also makes sense that they want to open source a big chunk of the "glue" between everything because 1) it's generally useful to lots of people / projects and 2) it gets other companies to kick in dollars for maintenance.

You're not going to get non-customers to fund the maintenance of an open source project that is wholly run and owned by Docker. They know that, and hence they're creating a governance board. But once you do that, you lose full control of your brand (which is attached to the project), and thus you have to divorce the two.

Docker is still going to drive the development of Moby. But over time, other companies will enter the fold to build other complex system orchestration projects on top of Moby.

[+] jacquesm|9 years ago|reply
It's quite clever actually. First you get half the world behind your open source project and then with a little sleight of hand that suddenly becomes a valuable commercial brand.

Not the first time this is done either.

[+] solatic|9 years ago|reply
On the contrary. If you understand enterprise FOSS, then you understand that one of the primary benefits of FOSS for the enterprise contributing customer is the ability to to make the software more relevant for the enterprise through the enterprise's contributions to the FOSS project, at a rate which is typically unfeasible, if even possible, for closed commercial products.

By splitting up the open-source Docker CE codebase into smaller projects, each smaller project is more approachable to outside potential contributors. If Docker can succeed in increasing the number of contributors to the Moby projects, compared to the Docker CE project, then they'll have succeeded in increasing the relevance of their product in their target market, which translates to more customers, sales, and profits in the long-term.

Of course they run the business risk of a fork taking off, but in the real world, forks are arms races. They're easily won against dead or severely mismanaged projects, but against a project with a well-funded corporate sponsor? Few, if any, will have the resources to put into making a better fork than the original.

And like they said, for the vast majority of their customers, nothing is changing. Their customers bought a Docker product yesterday, and they're buying a Docker product tomorrow. That branding isn't changing.

[+] FooBarWidget|9 years ago|reply
People were disappointed in Fedora too when it was first introduced. People even mocked the name. It worked out just fine for Red Hat.
[+] wesleytodd|9 years ago|reply
I do not understand the amount of negative feedback on this. It seems logical to me to move the underlying open source components to a separate org from the proprietary stuff. Why the hate?
[+] BinaryIdiot|9 years ago|reply
Honestly I think it makes sense as well but the way it was announced it was so incredibly confusing that I find myself agreeing with some of the negative and positive comments here depending on what my state of understanding was at the time.
[+] stephenr|9 years ago|reply
What is it with tech companies and fucking terrible name/branding analogies?

Docker. Ok, works with containers I get... wait. Your logo is a WHALE with a bunch of shipping containers on it. You know that whales DISAPPEAR underwater for hours at a time, and shipping containers are NOT MEANT TO DO THAT?

Ok ok maybe it was an honest mistake hey whats this new thing moby.. wait. Whales. Moby. Moby dick? Your project is named after a mythical white whale that sank boats, the very thing that actually carry CONTAINERS.

Having said all that, based on this [1] summary of Docker's usefulness in production from February, maybe both names are absolutely on-point for the reality of this abysmal 'product'.

1. https://thehftguy.com/2017/02/23/docker-in-production-an-upd...

[+] Khao|9 years ago|reply
A+ post.

Funny rambling backed up by a crazy well detailed article and you get downvoted by the HN hive mind. Docker is the ...ahem... whale in the room that you shouldn't criticize ever!

[+] mdekkers|9 years ago|reply
You know that whales DISAPPEAR underwater for hours at a time

Just like the stuff that relies on Docker

[+] JCharante|9 years ago|reply
Moby was the perfect name for a chaos monkey docker equivalent.
[+] tmaly|9 years ago|reply
Why change the name? All the books and posts reference the name Docker.
[+] e1ven|9 years ago|reply
It looks like they want to rename the OSS version, to differentiate it from their commercial version. They're using the analogy of Fedora->RHEL.

Moby = open source development Docker CE = free product release based on Moby Docker EE = commercial product release based on Docker CE.

[+] bauerd|9 years ago|reply
If I read this correctly they split up the Docker repository into several components. The moby repo pulls these together and spits out a build (or many builds?). So for instance one might choose an alternative container runtime to containerd? Then there's an officially supported set of components from which they build Docker CE. But why is Docker now called moby? Doesn't make sense to me.
[+] amouat|9 years ago|reply
Personally, I think this a great move for the community, but the reasoning and implications could have been made a little clearer.

The intention is to have a clearer split between the Open Source community project - which is and will always be the heart of Docker (or now Moby) and the "product" pieces. The Docker product itself is not changing at all from an external perspective - you will still call "docker run" as before. However, Docker will be an "assembly" of pieces from the moby project plus some under the docker namespace, probably including "docker/docker-cli". Going forward, this will allow the docker-cli to make more product-centric moves like further Hub integration that would be too controversial for the community moby project. It also allows other organisations to take the moby code and reassemble it in different ways without offending the Docker "product". Furthermore, the intention is that in the long run, various Moby pieces will move to community foundations such as the CNCF and OCI.

[+] _jezell_|9 years ago|reply
Please rename to Moby Dock instead.
[+] wnevets|9 years ago|reply
This certainly won't cause any confusion going forward.
[+] alanfranzoni|9 years ago|reply
Bad title. That's not what's happening. Moby is the name of the open source project.
[+] api|9 years ago|reply
So devops conferences will no longer sound like "Docker docker Docker? Docker. Docker docker docker docker? Ahh, Docker docker Docker docker docker." Now they'll sound like "Moby moby moby...?"

This seems absurd from a branding point of view.

I personally find this a little bit vindicating. I'm 39, which is like 120 in programmer years. I feel like one of the advantages of being an "old" (in quotes!) programmer is that I'm pretty good at spotting a fad. Unfortunately whenever I point one out I get mocked and down voted to oblivion, so I've learned to just sit on the sidelines and watch the fads go past and just work to avoid them in my own projects.

Programming is very faddish, and I've seen fads come and go. Here's a few of the ones I've seen in my tenure:

- "Design patterns" heavy "enterprise" programming, such as commonly seen in the older work in the Java and .NET ecosystem.

- XML for everything.

- Agile. Oh god, agile.

- Test driven development.

- Dynamic languages (as productivity magic pixie dust).

- Service oriented architecture (SOA).

... I could go on.

Not everything in these fads is bad. Fads often contain good ideas and some fads contain non-fad elements that stick around. But they're fads insofar as they are over-hyped as magic cure-alls.

The key characteristic of a fad is this:

It's heavily hyped as a cure to some set of very hard problems in programming or system administration, but all it really does is move the problem somewhere else or hack around it in some trivial gimmicky way. There is no real innovation. Meanwhile the fad often introduces new problems that nobody thinks about until the shine wears off.

My simple heuristic for recognizing a fad is to ask "where's the innovation?" A real innovation is a conceptual leap forward. It has a certain "meaty" feel to it and seems worthy of at least one solid CS paper. It often reduces complexity, since when deployed you can now dispense with all the mountains of hacks you used to work around the problem prior to the innovation.

Here's some current things that I very strongly think are fads:

- Microservices, which is just a reboot of SOA. The idea is not inherently bad and often results in more scalable systems, but the faddish part is the idea that replacing local API calls with RPC API calls or event queues is going to make some major class of programming problems go away. No, and you've also just introduced a new set of problems around network unreliability.

- "Serverless" cloud, a.k.a. total lock-in to a proprietary mainframe. Everyone doing this is going to regret it in 5-10 years except Amazon's shareholders. It's a roach motel. Compute will get another order of magnitude cheaper and prices for everything else will drop accordingly, but these prices will not since you drank the kool-aid haha.

- Containers.

... yes, containers.

Like most fads, containers are a response to a real set of problems. These are mainly:

- System/VM configuration drift and variability.

- Dependency and DLL hell.

- The fact that Linux/Unix has devolved into a single-tenant operating system where it's hard to deploy more than one thing on one "server." (This debacle is deserving of a whole very long blog post.)

Containers are a fad because they don't address any of those problems with real innovation.

System configuration drift, dependencies, and DLL hell are are addressed by the gimmicky hack of basically tar'ing up whole Linux images and treating them like gigantic statically linked binaries.

If you're going to do that, why do you need Docker/moby/whatever? Just use a static Linux distribution like http://sta.li and run every service in its own home or chroot. Keep your service trees in git and manage systems with Chef, Puppet, or f'ing shell scripts.

I fail to see why orchestration (a.k.a. provisioning) systems like Kubernetes could not work in such an environment without the container cruft.

Containers don't solve multi-tenancy much in practice. Their security profile is not good enough for true multi-tenancy, and if you try to run too many on one server you're going to run into stability problems that derive from all the bugs that exist in all that extra complexity they add.

Real solutions to these problems would... you know... really solve them. A real solution would make it possible to deploy stuff easily and forget about DLL and dependency hell. A real solution would restore true multi-tenancy to Unix, allowing users (not root) to install and run services on commodity identically-configured Unix systems. A real solution would reduce complexity.

Edit:

I do wonder a little if the idea of throwing out dynamic linking, which both containers and some newer compilers like Go do, is not a bad idea. Maybe it's obsolete. Maybe the tiny memory savings of dynamic linking are no longer justified by the complexity overhead of dependency management.

[+] munin|9 years ago|reply
They should have just re-branded to qwikster.
[+] thedangler|9 years ago|reply
Not to be rude or anything but the people in charge of such changes need to take a class in change management.