The article perfectly summarizes my frustration and sentiment. These days I hear these buzzwords all time. I work as a consultant for an enterprise product and most people whom I meet they somehow catch these buzzwords and blurt it out in front of everyone during meetings and discussions to either showoff that they know technology and things that are in the market these days(also latest iphone, apple news, tesla, space exploration and what not) or I feel they are somewhat trying to hide their insecurities.
Anyway long story short, most of these people do not really understand why they need all this rocket science to manage < 500 internal users. One of the new buzzwords I am hearing these days is mostly related to bigdata and machine learning. One of my managers came to me and asked me why dont we integrate our product with hadoop it will solve the performance problems as it can handle lot of data.
I am frustrated by the industry as a whole. I feel industry is simply following marketing trends. Imagine the no. of man-hours are put into investigating technologies and projects dropped mid-way realizing the technology stack is still immature or not suitable for at all.
This is how we manage this problem at the times when Visual Basic was the king and we use instead Visual FoxPro.
People want theirs apps to be made with Visual Studio (BTW, FoxPro was part of the package).
So they ask: "In what is the app made"?
"In Visual, Sir."
Done. End of story (like most of the time, obviously some times people are more dangerous and press it ;) ).
----
The point is not focus in the exact word but in what the people know the word will give to them.
So, for example "Big Data". The meaning for us matter zero. The meaning to some customer is that it have a largeish excel file that with his current methods and tools take too long to get results.
So. Do you use "Big Data Tools"?
"Yes Sir."
And what about use Hadoop?
"We use the parts of big data tech necessary for solve this, and if we need to use hadoop or other similar tools that fit better with your industry and that use the same principles will depend in our evaluation. Not worry, we know this"
Or something like that ;). Know that worry the people behind the words have help me a lot, even with people with WORSE tech skills (damm, I have build apps for almost iliterate people with big pockets but only witch cellphones as references of tech!)
And the anecdote about the largeish excel file that was too big and take too long? Yep, true. And was for one of the largest companies in my country ;)
> I am frustrated by the industry as a whole. I feel industry is simply following marketing trends.
My work lands me in a number of different conferences in non-software industries. This is true for all industries. Its just that ours has a faster revolving door. That, in addition to a low barrier to entry (anyone can claim they're a web developer), leads to a higher degree of this madness. Its just part of human behavior to seek out, and parrot, social signals that let others know you, too, are an insider.
Personally, I have to avoid a great number of those gatherings, since the lot of them are just a circlejerk of low-density information. If I pay too much attention to those events, I catch myself looking down my nose, and since that isn't productive/healthy behavior, I avoid landing myself in a place where guys with buddy-holly glasses and <obscure-craft-beer> argue which Wordpress plugin is the best.
Unfortunately I have to agree as a developer. My job is to make a fast, reliable, stable product but at the same time I'm questioned the tools I use by people who don't have any knowledge but heard the latest trend.
But sometimes it's also very easy to please people.
Big data: just insert 10M records in a database and suddenly everyone is happy because they now have big data :|
I never thought I would appreciate Java...but the industry has really made me.
Take your .war file, drop it onto JBoss. It deploys across the cluster in a zero downtime manner, isolates configuration, provides consistent log structure, cert management, deployment. You can deploy dozens of small war's to the same server and they can talk to each other. Load balance across the cluster automatically based on actual load. Run scheduled jobs and load balance the scheduled jobs themselves. Allow them to be isolated and unique within the cluster.
I may not like Java as a language, but from an infrastructure standpoint Java was basically Heroku long before Heroku was Heroku. The infrastructure is just...solid. The downside was that the XML config stuff was just messy.
Oh, I feel your pain! There's too much fashion going on in this industry, and the high salaries and the youth rediscovering the same concepts from 40 years ago doesn't help..
I mean, it's great to have this new tech and all, but when you're trying to build something to last some years, sometimes it's hard to filter the crap between all the buzzwords. It just reinforces the thought that smart people should just leave this field entirely or search for other fields of knowledge (or business) where our knowledge of programming can be made of use.
I'm 35 now, but I'm starting to realize that I will not have the patience to keep with all the crap just to be employable.. There are some areas where being old and experient is valuable. Philosophy, science, psychology, teaching, etc., are maybe some of them, but this industry is definitely not one of those areas. It makes me think that what I'm building now will some day be completely wiped out of existence..
"-No, look into microservices. It’s the future. It’s how we do everything now. You take your monolithic app and you split it into like 12 services. One for each job you do.
That seems excessive"
A 100 times yes. We tried to split our monolithic Rails app into micro-services built in Go. 2 years and many fires later, we decided to abandon the project. It was mostly because the monitoring and alerting were now split into many different pieces. Also, the team spent too much time debating standards etc. I think micro-services can be valuable, but we definitely didn't do it right, and I think a lot of companies get it wrong. Any positive experiences with micro-services here?
If that was ever true it certainly doesn't seem to be true now. All the tools support WSDL-first. All the tools are compatible with each other. Fill in the URL, let it autogenerate the interface, write your code and it all just works.
Man, I dont think this is the future at all.
OK, Docker is good and has its propose, and is very good on what its do: "Run only one process in one brand new kernel", but beyond than that, its just a daemon that uses and abuses of linux containers, you can easily scale, but is a pain in the ass to upgrade apps, also you need to run only one process on that.
Does not looks like the future for me to have 30 different linux containers running just only one process in each of them, dude, you have a kernel in your hand, why the hell you will run only one process on it? (what the heck, you can protect yourself and scale without be the bitch of a daemon, you just need to know your best friend kernel), you dont need to make micro services for everything, its good ok, but its not the solution for everything like the people are saying...
I really dont have any idea why the people are are so excited about "docker" all the things.
I am going to ramble. Just move on if you don't care to hear the ramblings of a 62 year old development manager.
I'm pretty docker ignorant. I think I get it in concept. I manage >150 web sites (~15,000 pages total) that are php based with eXist-db and oracle (overkill but forced to use it) for database backends. My team develops on mac os x and pushes code to RHEL. We have never had a compatability problem between os x and RHEL except for some mgmt scripts in bash that were easily coded around.
Big data to me is a 400 MB apache log file.
I go home grateful I don't have to be in the buzz word mix.
I do read a lot about technology and over time that informs some changes like using apache camel for middleware, splunk for log file analysis yada dada...
I have had bosses that brought me buzz word solutions that don't ever match the problems we have. I hate that but right now I am not in that position. My boss leaves technology decisions to us.
Least you think we are not modern at all we do use a CDN, git and more.
Some days I get anxiety from reading HN, feeling stupid. Some days I get a lift from HN from reading articles like this one and the comments.
I am so glad I'm not in the business of chasing technology.
Funnily enough, I don't think this is a real concern. It's the stereotype, which is why I said it, but Google believes it will make as much money off cloud as from ads, so I wouldn't expect anything to get shut down.
What do you mean? I'm just running bytecode on a virtual machine ontop of a virtualized container on top of virtualized hardware on a CPU where the instruction set is virtualized in microcode...
1) Small teams (~1-5 people) trying to seem "big" by working at Google's scale.
2) Heroku's prices. We are currently (successfully so far) migrating a small Django project from bare Amazon EC2 instances to ECS with Docker. Even using 3 EC2 micro instances (1 vCPU, 1 GB RAM) for the Docker cluster we would spend ~8 USD/month/instance. With Heroku the minimum would be 25 USD/month/dyno. That's a 3x increase in expenses.
It's very possible to take advantage of technologies like containers without getting too caught in the hype.
wait. you're comparing $25 with $75. it is 3x but it's still accounting noise by any standard imaginable unless you're running a charity server for an open source project.
To answer the concerns raised in the comments: we are a real company and it took 2 weeks (while working on other features and bugfixes) to migrate to Docker. The plus is that now we have experience with the platform and we can streamline the process. Again: we are not using microservices or anything like that, simply Docker containers instead of EC2 instances, which makes life pretty damn easier (and cheaper).
And 25 and 75 are bogus numbers, what if we start running 10 instances?
Indeed, the goal is to solve your business problem with technology, not use Docker for everything that you can find in your infra. Many people are mixing up the two. Docker can be replaced with anything that is hyped at this level.
I worry when I can't tell if a comment like this is based on fact or just trying to be funny.
Because I've seen my share of nasty "legacy" automation but, surprisingly, I still think a good set of well thought-out shell scripts written by someone that understands what's being automated still beat modern tools, even when the person doing the automation is the same.
I don't quite know why this is, but there's something timeless about shell scripts. I've also seen shell script automation survive for a long time unattended and with zero issues. Not so with some of the modern tools that are supposed to be all unicorns and rainbows.
Or just put your app(s) into containers and run them through docker compose on a single VPS. That bypasses about 99% of the things listed in this article.
You can still easily set things up so it's a git based deploy which is hands free after the initial push.
Now you have a single $5-10/month server that runs your app's stack without a big fuss. Of course it's not "web scale" with massive resiliency but when you're just starting out, 1 server instance is totally fine and exactly what you want.
I've ran many projects for years on 1 server that did "business mission critical" tasks like accepting payments, etc..
I wonder if the original HN "Heroku is Dead..." title will cost Heroku money / market share in indirect ways.
When I see titles like that (despite the fact that it was intended as sarcasm), I think to myself, e.g., "I bet at least hundreds of people who scrolled past it thought it was sincere, and now they will have this subconscious 'Heroku is Dead... Docker...' thought at times when deploying projects. Maybe they'll even check out Docker. Maybe these hundreds of people will represent a tipping point of sorts for Heroku->Docker migrations, because one of them will write a really great blog post about it, and it will receive thousands of views..." (alternate endings of the same thought continue to be brute-forced for a few moments).
Along the same vein of thinking, back in 2008 I had this "realization" that Google could control the world by simply showing results based on headline titles (e.g., a search for "Obama" during the election could have resulted in articles / results whose titles have words/phrases whose presences are positively correlated to lower or higher stress levels, assumptions, other emotions, etc., resulting in a net positive or negative sentiment, respectively, about the subject of the search query, all while simply scanning the results to determine which one to click).
This pretty much sums it up. New stuff, stacked on existing stuff, with the theory that older stuff was hard, and new stuff is better, but you still needs the old stuff, so you basically end up doing exactly what you did, but now with more stuff in between, adding cost, complexity and additional failure modes while solving nothing.
Any of the proposed problems that containerization was supposed to fix are already fixed by using proper configuration management. In almost all cases so far, people yammering on about docker and containers (and CoreOS), it ended up being their idea of configuration management, because they didn't have any in the first place.
Say you want to fix your 'problems' with setting up servers, how about doing it the right way. You will need deployment services, regardless of containers, VMs or bare metal. You will also need configuration management services, and monitoring. Containers and special distributions solve none of it, knowledge to run systems is still required and not actually fixing your problems and layering stuff on top of it doesn't actually help.
Get something like SaltStack or Chef, and configure the out of everything. It doesn't care what you're running on, and actually solves the problems that need fixing.
I've spun up a lot of kubernetes clusters to test it out. A few months ago I also tested out Flynn, Deis, Deis Workflow, Openstack, and a lot of other options. I still haven't found a simple bootstrap script that gets everything set up on AWS and lets me simply deploy my application. And it's true that storage still seems to be an unsolved problem with kubernetes.
Heroku is great, and free for small services. On the other hand, a highly-available kubernetes cluster is going to set you back at least $100 per month, which is just too much for small startups and side projects before they take off.
I think I'm going to forget everything and head towards http://serverless.com/. No Heroku, no Docker, no micro-services, no servers. Just everything running on AWS Lambda and DynamoDB. And everything static in S3 behind Cloudfront.
Or maybe just Firebase. But I really am tired of managing servers.
Making sort of funny articles about Docker doesn't change the fact that circleci's support for docker is horrible. I don't use it on prod (and I probably won't) but since circleci forces me to use ubuntu 12.04 which is not something I use on prod I want a docker container which looks like my production host.
Having said that - circle really tries to support docker, but It doesn't.
I have to use ECS for caching (I am not happy about it)
Builds might fail due to the custom docker version/compilation
You can mock docker, but people are using it in one way or another and you should support it properly.
[+] [-] rajeshp1986|9 years ago|reply
Anyway long story short, most of these people do not really understand why they need all this rocket science to manage < 500 internal users. One of the new buzzwords I am hearing these days is mostly related to bigdata and machine learning. One of my managers came to me and asked me why dont we integrate our product with hadoop it will solve the performance problems as it can handle lot of data.
I am frustrated by the industry as a whole. I feel industry is simply following marketing trends. Imagine the no. of man-hours are put into investigating technologies and projects dropped mid-way realizing the technology stack is still immature or not suitable for at all.
[+] [-] mamcx|9 years ago|reply
People want theirs apps to be made with Visual Studio (BTW, FoxPro was part of the package).
So they ask: "In what is the app made"?
"In Visual, Sir."
Done. End of story (like most of the time, obviously some times people are more dangerous and press it ;) ).
----
The point is not focus in the exact word but in what the people know the word will give to them.
So, for example "Big Data". The meaning for us matter zero. The meaning to some customer is that it have a largeish excel file that with his current methods and tools take too long to get results.
So. Do you use "Big Data Tools"?
"Yes Sir."
And what about use Hadoop?
"We use the parts of big data tech necessary for solve this, and if we need to use hadoop or other similar tools that fit better with your industry and that use the same principles will depend in our evaluation. Not worry, we know this"
Or something like that ;). Know that worry the people behind the words have help me a lot, even with people with WORSE tech skills (damm, I have build apps for almost iliterate people with big pockets but only witch cellphones as references of tech!)
And the anecdote about the largeish excel file that was too big and take too long? Yep, true. And was for one of the largest companies in my country ;)
[+] [-] ryanmaynard|9 years ago|reply
My work lands me in a number of different conferences in non-software industries. This is true for all industries. Its just that ours has a faster revolving door. That, in addition to a low barrier to entry (anyone can claim they're a web developer), leads to a higher degree of this madness. Its just part of human behavior to seek out, and parrot, social signals that let others know you, too, are an insider.
Personally, I have to avoid a great number of those gatherings, since the lot of them are just a circlejerk of low-density information. If I pay too much attention to those events, I catch myself looking down my nose, and since that isn't productive/healthy behavior, I avoid landing myself in a place where guys with buddy-holly glasses and <obscure-craft-beer> argue which Wordpress plugin is the best.
[+] [-] huuu|9 years ago|reply
Unfortunately I have to agree as a developer. My job is to make a fast, reliable, stable product but at the same time I'm questioned the tools I use by people who don't have any knowledge but heard the latest trend.
But sometimes it's also very easy to please people. Big data: just insert 10M records in a database and suddenly everyone is happy because they now have big data :|
[+] [-] brightball|9 years ago|reply
Take your .war file, drop it onto JBoss. It deploys across the cluster in a zero downtime manner, isolates configuration, provides consistent log structure, cert management, deployment. You can deploy dozens of small war's to the same server and they can talk to each other. Load balance across the cluster automatically based on actual load. Run scheduled jobs and load balance the scheduled jobs themselves. Allow them to be isolated and unique within the cluster.
I may not like Java as a language, but from an infrastructure standpoint Java was basically Heroku long before Heroku was Heroku. The infrastructure is just...solid. The downside was that the XML config stuff was just messy.
[+] [-] jventura|9 years ago|reply
I mean, it's great to have this new tech and all, but when you're trying to build something to last some years, sometimes it's hard to filter the crap between all the buzzwords. It just reinforces the thought that smart people should just leave this field entirely or search for other fields of knowledge (or business) where our knowledge of programming can be made of use.
I'm 35 now, but I'm starting to realize that I will not have the patience to keep with all the crap just to be employable.. There are some areas where being old and experient is valuable. Philosophy, science, psychology, teaching, etc., are maybe some of them, but this industry is definitely not one of those areas. It makes me think that what I'm building now will some day be completely wiped out of existence..
[+] [-] karrotwaltz|9 years ago|reply
"We could store gigabytes of data on the clients without having to pay for servers"
[+] [-] mrhektor|9 years ago|reply
That seems excessive"
A 100 times yes. We tried to split our monolithic Rails app into micro-services built in Go. 2 years and many fires later, we decided to abandon the project. It was mostly because the monitoring and alerting were now split into many different pieces. Also, the team spent too much time debating standards etc. I think micro-services can be valuable, but we definitely didn't do it right, and I think a lot of companies get it wrong. Any positive experiences with micro-services here?
[+] [-] pbiggar|9 years ago|reply
To answer some questions: yes this is obviously poking fun at Docker, but I also do really believe in Docker. See the follow-up for more on that: https://circleci.com/blog/it-really-is-the-future/
In a self-indulgent moment I made a "making of" podcast about this blog post, which is kinda interesting (more about business than tech): http://www.heavybit.com/library/podcasts/to-be-continuous/ep...
And if you like this post you'll probably like the rest of the podcast: http://www.heavybit.com/library/podcasts/to-be-continuous/
[+] [-] rjvir|9 years ago|reply
- So shared webhosting is dead, apparently Heroku is the future?
- Why Ruby, why not just PHP?
- Wait, what's Rails? Is that different from Ruby?
- What's MVC, why do I need that for my simple website?
- Ok, so I need to install RubyGems? What's a Gemfile.lock? None of these commands work on Windows.
- I don't like this new text editor. Why can't I just use Dreamweaver?
- You keep talking about Git. Do I need that even if I'm working alone?
- I have to use command line to update my site? Why can't I just use FTP?
- So Github is separate from Git? And my code is stored on Github, not Heroku?
- Wait, I need to install both PGSql and SQLite? Why is this better than MySQL?
- Migrations? Huh?
[+] [-] nathell|9 years ago|reply
[+] [-] sytringy05|9 years ago|reply
Nooooooooooooooooo. Everytime someone says "service discovery" a kitten dies (Except for consul, that's the biz).
[+] [-] codeulike|9 years ago|reply
[+] [-] lmm|9 years ago|reply
[+] [-] chukye|9 years ago|reply
I really dont have any idea why the people are are so excited about "docker" all the things.
[+] [-] epalmer|9 years ago|reply
I'm pretty docker ignorant. I think I get it in concept. I manage >150 web sites (~15,000 pages total) that are php based with eXist-db and oracle (overkill but forced to use it) for database backends. My team develops on mac os x and pushes code to RHEL. We have never had a compatability problem between os x and RHEL except for some mgmt scripts in bash that were easily coded around.
Big data to me is a 400 MB apache log file.
I go home grateful I don't have to be in the buzz word mix.
I do read a lot about technology and over time that informs some changes like using apache camel for middleware, splunk for log file analysis yada dada...
I have had bosses that brought me buzz word solutions that don't ever match the problems we have. I hate that but right now I am not in that position. My boss leaves technology decisions to us.
Least you think we are not modern at all we do use a CDN, git and more.
Some days I get anxiety from reading HN, feeling stupid. Some days I get a lift from HN from reading articles like this one and the comments.
I am so glad I'm not in the business of chasing technology.
[+] [-] hoechst|9 years ago|reply
I read both articles a year ago and it really helped me grasp the whole container movement.
[+] [-] robteix|9 years ago|reply
"-You think that’s going to be around in 6 months?"
Isn't reputation a thing of beauty?
[+] [-] droidist2|9 years ago|reply
[+] [-] reitoei|9 years ago|reply
[+] [-] pbiggar|9 years ago|reply
[+] [-] TickleSteve|9 years ago|reply
[+] [-] kalleboo|9 years ago|reply
[+] [-] MichaelBurge|9 years ago|reply
[+] [-] lambdacomplete|9 years ago|reply
1) Small teams (~1-5 people) trying to seem "big" by working at Google's scale.
2) Heroku's prices. We are currently (successfully so far) migrating a small Django project from bare Amazon EC2 instances to ECS with Docker. Even using 3 EC2 micro instances (1 vCPU, 1 GB RAM) for the Docker cluster we would spend ~8 USD/month/instance. With Heroku the minimum would be 25 USD/month/dyno. That's a 3x increase in expenses.
It's very possible to take advantage of technologies like containers without getting too caught in the hype.
[+] [-] baq|9 years ago|reply
[+] [-] lambdacomplete|9 years ago|reply
And 25 and 75 are bogus numbers, what if we start running 10 instances?
[+] [-] sleepyhead|9 years ago|reply
[+] [-] StreamBright|9 years ago|reply
[+] [-] philliphaydon|9 years ago|reply
[+] [-] LukeB_UK|9 years ago|reply
[+] [-] xchaotic|9 years ago|reply
[+] [-] CrLf|9 years ago|reply
Because I've seen my share of nasty "legacy" automation but, surprisingly, I still think a good set of well thought-out shell scripts written by someone that understands what's being automated still beat modern tools, even when the person doing the automation is the same.
I don't quite know why this is, but there's something timeless about shell scripts. I've also seen shell script automation survive for a long time unattended and with zero issues. Not so with some of the modern tools that are supposed to be all unicorns and rainbows.
[+] [-] tormeh|9 years ago|reply
[+] [-] praptak|9 years ago|reply
[+] [-] nickjj|9 years ago|reply
You can still easily set things up so it's a git based deploy which is hands free after the initial push.
Now you have a single $5-10/month server that runs your app's stack without a big fuss. Of course it's not "web scale" with massive resiliency but when you're just starting out, 1 server instance is totally fine and exactly what you want.
I've ran many projects for years on 1 server that did "business mission critical" tasks like accepting payments, etc..
[+] [-] mangeletti|9 years ago|reply
When I see titles like that (despite the fact that it was intended as sarcasm), I think to myself, e.g., "I bet at least hundreds of people who scrolled past it thought it was sincere, and now they will have this subconscious 'Heroku is Dead... Docker...' thought at times when deploying projects. Maybe they'll even check out Docker. Maybe these hundreds of people will represent a tipping point of sorts for Heroku->Docker migrations, because one of them will write a really great blog post about it, and it will receive thousands of views..." (alternate endings of the same thought continue to be brute-forced for a few moments).
Along the same vein of thinking, back in 2008 I had this "realization" that Google could control the world by simply showing results based on headline titles (e.g., a search for "Obama" during the election could have resulted in articles / results whose titles have words/phrases whose presences are positively correlated to lower or higher stress levels, assumptions, other emotions, etc., resulting in a net positive or negative sentiment, respectively, about the subject of the search query, all while simply scanning the results to determine which one to click).
[+] [-] Fiahil|9 years ago|reply
This would be true for an average BuzzFeed-consuming-crowd, which -to my knowledge- isn't the case here.
[+] [-] oneplane|9 years ago|reply
Any of the proposed problems that containerization was supposed to fix are already fixed by using proper configuration management. In almost all cases so far, people yammering on about docker and containers (and CoreOS), it ended up being their idea of configuration management, because they didn't have any in the first place.
Say you want to fix your 'problems' with setting up servers, how about doing it the right way. You will need deployment services, regardless of containers, VMs or bare metal. You will also need configuration management services, and monitoring. Containers and special distributions solve none of it, knowledge to run systems is still required and not actually fixing your problems and layering stuff on top of it doesn't actually help.
Get something like SaltStack or Chef, and configure the out of everything. It doesn't care what you're running on, and actually solves the problems that need fixing.
[+] [-] nathan_f77|9 years ago|reply
Heroku is great, and free for small services. On the other hand, a highly-available kubernetes cluster is going to set you back at least $100 per month, which is just too much for small startups and side projects before they take off.
I think I'm going to forget everything and head towards http://serverless.com/. No Heroku, no Docker, no micro-services, no servers. Just everything running on AWS Lambda and DynamoDB. And everything static in S3 behind Cloudfront.
Or maybe just Firebase. But I really am tired of managing servers.
[+] [-] numbsafari|9 years ago|reply
Maybe the problem is AWS.
[+] [-] nzoschke|9 years ago|reply
Disclaimer: I work at Convox.
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] jakozaur|9 years ago|reply
[+] [-] degenerate|9 years ago|reply
Should I round up or down?
[+] [-] gchokov|9 years ago|reply
[+] [-] lopatin|9 years ago|reply
[+] [-] bbctol|9 years ago|reply
[+] [-] Maro|9 years ago|reply
[+] [-] eddd|9 years ago|reply
I have to use ECS for caching (I am not happy about it)
Builds might fail due to the custom docker version/compilation
You can mock docker, but people are using it in one way or another and you should support it properly.