wowmsi's comments

I would recommend "Architecture of a Database System" by Joe Hellerstein, Micheal Stonebraker, and James Hamilton [0] as a start if you want formal perspective without getting too much into theory.

[0] http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf

Mesos and Kubernetes share lot of commonalities: both are useful for managing (virtual) compute resources for a given application. Primarily, as an app developer, you are interesting in specifying the required resources at a high level: 10 front end instances, 20 middle tier, and 3 back end database instances. You may not care how the resources are provisioned as long as some underlying SLA is satisfied. This is useful for boosting service uptime since in any long running service the nodes may go offline anytime and the framework will re-provision the resources automatically--often within few mins.

Mesos is useful to share resources across multiple clusters (i.e., multiple departments using Hadoop within a same organization may want to share the resources), where as Kubernetes seems to focus on container applications using docker. In addition, Mesos implements dominant-resource-fairness (DRF) scheduling, which has some nice properties [0].

Lastly, Spark is just an application running on the top of Mesos or Kubernetes. Using my earlier example, Spark and Hadoop can run in a single datacenter on the top of Mesos without statically partitioning the clusters a priori each having an illusion that they own the entire datacenter.

[0] https://www.usenix.org/event/nsdi11/tech/full_papers/Hindman...

Also Riak. However, one interesting aspect of Dynomite seems to be offer default support for clustering across multiple data centers with pluggable stores.