This is nice but it's characteristic of most other Docker tutorials which say "just run this command" and don't bother to go into how or why it works. As a result, all the reader can do is copy and paste (if the command even works in the first place or the tutorial isn't out dated). What I think most people need to know is how they can do this but modify the app to work for their purposes instead of just "hello world".
The title should be corrected to read "How to set up and deploy a 1000 node botnet". There's no mention of securing the hosts or the swarm what-so-ever.
I liked the sentence: "There are a few other arguments that you’ll need to add to those commands but if you follow the docs you’ll have no problems at all."
This is clearly a doc based on a hypothetical assumption that he can deploy 1000 nodes. If thats the case, why not go for a cool million?
Just a suggestion to the OP, it not hard to setup and share a 5 node vagrant cluster on your laptop. Give concrete examples that people can run locally and test your assertions themselves. Once that foundation is laid, you can extrapolate to 10 nodes, 100 nodes, 1000 nodes.
Anyone that has deployed a cluster of that size knows that the article is missing a bunch of items, not limited to the following:
- Overhead Instances (manager, service discorvery, loggging, etc.)
- Configuration Management
- Security Implications
- Monitoring
- Failure mitigation (its going to happen at that scale)
- Update strategy at this scale
For those that are interested, one official doc and a good place to start when leaning how to deploy a large docker 1.12 cluster is this guide by docker.
Fully agree, this is a typical example of a shitty Swarm/Kubernetes/etc blog post, which gives no real substance - this stuff is covered in first 5 minutes.
We are just deploying our first Kubernetes cluster in production and anything more than basic hello world would be welcome. Like how to configure networking in production, how to route traffic to containers, how to provide volume storage (backups, etc..).
I mean, we'll get there of course, but we opted against Swarm as data is even more lacking than Kubernetes.
"replacing 3 in wherever you see 1000 in this post is probably a good idea"
...which really means "I have no clue where this will break when scaling".
Cute, but not terribly insightful, and possibly risky in an age where following recipes off the Internet is too often the first step towards production :)
Is the definition of bare-metal changing? It seems clear from the context that virtual machines are being used, but is that a distinction that those further up the stack don't worry about now?
basically this is how the howto handles the hardest part:
>Basically you will run docker swarm init on the first
node and then docker swarm join on all the other nodes.
There are a few other arguments that you’ll need to add
to those commands but if you follow the docs you’ll have
no problems at all.
worse part of the setup is how to build the cluster nodes store in a way which is redundant and reliable, since provisioning it for HA is largely undocumented and left to an exercise for the reader
Maybe I'm misunderstanding what you mean by "cluster node store" but in Docker 1.12 its built in, setting up a swarm is now really easy. No need for consul, etcd, etc. It's high available by default.
So this article talks about how you would deploy a 1000 node cluster without actually doing it? Why not saying this is how to deploy a 100000 node cluster?
Meh, the article doesn't really bring anything new to the table.
If you are doing something like this, please keep in mind that this kind of DNS failover is, at best, unreliable. You have no control of how DNS is being cached on client side, and whether the client is going to switch to the next IP in the cycle if the previous one is unavailable.
Proper way to do HA would be to use some kind of VIP + load balancer combination (eg, keepalived + HAProxy), which would allow you to failover the IP instead to just rely on the hostname. However if you also have a database backend to think about, then u will most likely need something like Pacemaker to ensure you don't end up with data inconsistency (brain split scenario).
[+] [-] n0us|9 years ago|reply
[+] [-] dockinator|9 years ago|reply
[+] [-] HyperLinear|9 years ago|reply
Oh, you mean like 50% of the work?
As for the reverse proxy: Traefik all the way! :D
[+] [-] gamedna|9 years ago|reply
Just a suggestion to the OP, it not hard to setup and share a 5 node vagrant cluster on your laptop. Give concrete examples that people can run locally and test your assertions themselves. Once that foundation is laid, you can extrapolate to 10 nodes, 100 nodes, 1000 nodes.
Anyone that has deployed a cluster of that size knows that the article is missing a bunch of items, not limited to the following: - Overhead Instances (manager, service discorvery, loggging, etc.) - Configuration Management - Security Implications - Monitoring - Failure mitigation (its going to happen at that scale) - Update strategy at this scale
For those that are interested, one official doc and a good place to start when leaning how to deploy a large docker 1.12 cluster is this guide by docker.
https://docs.docker.com/swarm/plan-for-production/
[+] [-] luka-birsa|9 years ago|reply
We are just deploying our first Kubernetes cluster in production and anything more than basic hello world would be welcome. Like how to configure networking in production, how to route traffic to containers, how to provide volume storage (backups, etc..).
I mean, we'll get there of course, but we opted against Swarm as data is even more lacking than Kubernetes.
[+] [-] vegasbrianc|9 years ago|reply
[+] [-] rcarmo|9 years ago|reply
...which really means "I have no clue where this will break when scaling".
Cute, but not terribly insightful, and possibly risky in an age where following recipes off the Internet is too often the first step towards production :)
[+] [-] rspeer|9 years ago|reply
[+] [-] dockinator|9 years ago|reply
[+] [-] timthorn|9 years ago|reply
[+] [-] chatmasta|9 years ago|reply
[+] [-] emmetogrady|9 years ago|reply
EDIT: I had been misusing the term bare-metal, thanks for picking up on it. Examples should hold on both bare metal and VMs though.
[+] [-] geggam|9 years ago|reply
Buy 1000 bare metal servers
This one is easy. Pick your favourite cloud provider and buy lots of servers
[+] [-] rco8786|9 years ago|reply
[+] [-] LoSboccacc|9 years ago|reply
>Basically you will run docker swarm init on the first node and then docker swarm join on all the other nodes. There are a few other arguments that you’ll need to add to those commands but if you follow the docs you’ll have no problems at all.
worse part of the setup is how to build the cluster nodes store in a way which is redundant and reliable, since provisioning it for HA is largely undocumented and left to an exercise for the reader
[+] [-] emmetogrady|9 years ago|reply
[+] [-] redwood|9 years ago|reply
[+] [-] discordianfish|9 years ago|reply
So this article talks about how you would deploy a 1000 node cluster without actually doing it? Why not saying this is how to deploy a 100000 node cluster?
[+] [-] crypt1d|9 years ago|reply
If you are doing something like this, please keep in mind that this kind of DNS failover is, at best, unreliable. You have no control of how DNS is being cached on client side, and whether the client is going to switch to the next IP in the cycle if the previous one is unavailable. Proper way to do HA would be to use some kind of VIP + load balancer combination (eg, keepalived + HAProxy), which would allow you to failover the IP instead to just rely on the hostname. However if you also have a database backend to think about, then u will most likely need something like Pacemaker to ensure you don't end up with data inconsistency (brain split scenario).
[+] [-] sinneduy|9 years ago|reply
[+] [-] drc0|9 years ago|reply
[+] [-] roddux|9 years ago|reply