I’ve come to the conclusion that Heroku must be using adaptive processes to sandbag us. For example, to save on their resources, they must be creating static caches of the pages which are being requested from these cron pings. They could avoid loading my whole app back into memory with a trick like this. Please correct me if I’m wrong, but I see no other way for these pings to be successful and my app to still have bad initial load times when it is not visited for a while.
You are incorrect. We idle single-dyno apps that have not recently received requests. We do no caching, unless you are running an old bamboo app and are therefore going through varnish. See https://devcenter.heroku.com/articles/dynos#dyno-idling
Regardless of his slight misunderstanding of the technical side of things, he is correct when he says that you can be paying over $100 a month for a service and still be subject to dyno idling, which does seem pretty ridiculous, especially since Heroku takes a 30% cut of all add-on revenue (https://addons.heroku.com/provider/resources/HerokuAddonsLic...)
I don't understand how my app could be set to idle when I have pings hitting it from three separate sources. Also, the pingdom ping is looking for a specific word on one of my pages. It has to find this word to be considered successful.
Edit: The obvious solution is to buy another dyno. This isn’t the issue, however. I wrote this post to show the unfortunate grey area in their pricing structure. It’s possible for low-traffic customers to pay a lot and still be subject to the poor performance that comes with a dyno being set to idle.
The greater underlying problem is there are some customers who only need one dyno (especially those of us who are running unicorn. My dyno can handle three times the number of requests) and we would rather spend a little extra on an upgraded database or some other addons.
Well, that's that it seems. Could it be that he's running a single dyno app (with a bunch of add-ons) and that's what explains the behavior he's seeing?
Imagine how many free apps are created that are never used again. Should Heroku really devote resources to these apps? I think it's perfectly reasonable to idle them after a period of inactivity. If they didn't, they would need more resources and ultimately the customers would be paying for that. They aren't a charity, they are a business.
It's pretty simple, if you don't want your app to idle out after 1 hour of inactivity, buy a dyno for $35 p/month.
From the documentation they "...will have that web dyno idled out after one hour of inactivity."
The issue is that there is no hour of inactivity because there is web monitoring/querying supposedly going on automatically. So there should be no idling.
I feel your pain, as a fellow bootstrapper, but a single paid dyno, which will keep your app unidled, is gonna cost you $35 a month. I did run into a bug recently where one of our dynos was being idled even after we paid up, but it has since been resolved.
I'd say just pay them another $35 a month. Considering the miracles Heroku regularly performs as an ops team, it's worth it.
Fine-tuning an image like that only matters when your site is getting a ton of traffic. I am only employing these other services for the learning experience.
for $108 / month you could have a very nice setup on any number of VPSs, RS, ec2, etc... you could have at least a 2GB setup either split between 2 hosts or all on one depending on how you plan on scaling... but really IMO if you plan on running anything for any matter of time beyond a (hey checkout my neat idea app) get off heroku and figure out how to setup your own systems, nginx/haproxy, etc... sure it's nice to focus only on the app side with heroku so learn stuff there but eventually unless you're getting funded - you'll need to know how to manage your own servers... just my thoughts.. and i do put lots of little projects on heroku but they're mostly none critical services... or ideas
Why do you need to know how to setup and manage your own services. If it's not your expertise, spend your time where you can add the most value. I think Heroku is a perfectly fine option until you reach the point where you could hire a full time dev ops.
It's a pretty cheap way to have servers set up with 24/7 monitoring and technical support.
I'd be curious what dsower's heroku logs look like. Would you mind posting the portion of this where it happened?
The logs should show 'idling' when idling and also show the 'pings' from pingdom and/or new relic. Looking through that it'd give us all something concrete to discuss - because quite possibly this is just a bug with Heroku's dyno idling that could have simply been submitted as a support ticket.
Reposting my comment from the website here (unreadable there):
I contacted support about this a while ago (was in a similar situation). Their response was "Use more than one web dyno". I didn't need more than one, but that was the only way they offered to allow my app to not "go to sleep".
This is kind of an annoying problem if you have a lot of small sites for whatever reason. I think their policy is reasonable and they are upfront about it, so I certainly don't begrudge them, but it is still annoying.
What is the best way to host a bunch of low-traffic sites for less than $35 (I believe that is what it costs for the second dyno, required to keep your app from being spun down) each per month? The usual answer is "PHP", but I sometimes wonder if there'd be a market (and way to make money) for Heroku-like ease of deployment/management, billed based on resources (much like a VPS) rather than per application.
I love the way the comments have steered into technology nonsense, while the issue has nothing to do with the way in which Heroku screws him over, just the mere fact that they ARE screwing him. Commenters who have suggested 'just pay $35/month for not having 10 second lag' are idiots.
Heroku is a great and easy service, but I find the add-on 'no lag for $35/month' which is what the dyno basically is to the poster, unacceptable. But if most commenters are willing to pay into the hype, go ahead.
How is giving him a free dyno (albeit one that spins down when it's idle) "screwing him over"? Be specific.
The cheapest Linode is $19.95/month, and that's just a bare VM. You have to do all the sysadmin stuff yourself. For $15/month more, Heroku takes care of all that for you. If your time is worth anything at all, maintaining your Linode is going to burn up more than $15/month.
Now, granted, for some use cases the Linode makes more sense (I have one of those, too, and have been happy with it and the company overall), but I don't think either company is screwing anybody over. Both are offering good value on their products, but those products are different. Linode gives maximum flexibility, but at the cost of full responsibility. Heroku gives maximum convenience, but at the cost of less flexibility.
I have some feedback on your site (http://www.mycelial.com) that is unrelated to your post. I had no idea what your site was until I went to the About page. I think the contents of the About page should be put on the homepage.
We use Pingdom to keep a couple low-traffic sites alive without issue. I think we could provide you with more help if you posted a condensed sample of your Heroku logs.
To clarify: I understand that getting another dyno would solve this problem. I love heroku but I think that they need to rethink the idle process. Some users only need 1 dyno and a bunch of addons. If we are paying decent money then the idle threshold should be increased, in my opinion.
It's just semantics. They are giving away the first dyno. Would you prefer they took that away when you purchased one for real? 2 for the price of 1 seems pretty nice.
That and fundamentally (as you noted about being optimistic) you're paying $100+ a month for an app that gets zero traffic. That seems to be the core issue.
A cron job where a request is sent at regular intervals to a dyno from Phantom should solve your issue. It should be justified if you are buying addons. For free sites with no revenue to Heroku, it's reasonable since they have many sites that only get visited monthly, if that.
Seems to me the real problem is you spend 50 for a bigger DB instead of 35 for a dyno. If you got no traffic why do you need so much db cache especially in addition to memcache. You can easily fix that with pgbackups and import into a basic db.
Your post says that you want to hear from Heroku, but it doesn't mention anything about you trying to contact them directly. Did you try to contact them before making a somewhat accusatory blog post without much proof? What did they say?
Write 'intelligent' automated tests (you're a HN'er so you have these already right ;) and schedule them to run CI on your production site. That may prevent any adaptive processes from caching.
[+] [-] bgentry|13 years ago|reply
You are incorrect. We idle single-dyno apps that have not recently received requests. We do no caching, unless you are running an old bamboo app and are therefore going through varnish. See https://devcenter.heroku.com/articles/dynos#dyno-idling
[+] [-] h4rrison|13 years ago|reply
[+] [-] dsowers|13 years ago|reply
[+] [-] dsowers|13 years ago|reply
The greater underlying problem is there are some customers who only need one dyno (especially those of us who are running unicorn. My dyno can handle three times the number of requests) and we would rather spend a little extra on an upgraded database or some other addons.
[+] [-] biesnecker|13 years ago|reply
[+] [-] redguava|13 years ago|reply
Imagine how many free apps are created that are never used again. Should Heroku really devote resources to these apps? I think it's perfectly reasonable to idle them after a period of inactivity. If they didn't, they would need more resources and ultimately the customers would be paying for that. They aren't a charity, they are a business.
It's pretty simple, if you don't want your app to idle out after 1 hour of inactivity, buy a dyno for $35 p/month.
[+] [-] grannyg00se|13 years ago|reply
The issue is that there is no hour of inactivity because there is web monitoring/querying supposedly going on automatically. So there should be no idling.
[+] [-] skinnymuch|13 years ago|reply
[+] [-] carsongross|13 years ago|reply
I'd say just pay them another $35 a month. Considering the miracles Heroku regularly performs as an ops team, it's worth it.
[+] [-] ericabiz|13 years ago|reply
Yet you have a 507KB (!) image on your site's home page. Optimizing that would go a long way toward improving the end-user experience.
[+] [-] dsowers|13 years ago|reply
[+] [-] taf2|13 years ago|reply
[+] [-] redguava|13 years ago|reply
It's a pretty cheap way to have servers set up with 24/7 monitoring and technical support.
[+] [-] scottmotte|13 years ago|reply
The logs should show 'idling' when idling and also show the 'pings' from pingdom and/or new relic. Looking through that it'd give us all something concrete to discuss - because quite possibly this is just a bug with Heroku's dyno idling that could have simply been submitted as a support ticket.
[+] [-] taybenlor|13 years ago|reply
I contacted support about this a while ago (was in a similar situation). Their response was "Use more than one web dyno". I didn't need more than one, but that was the only way they offered to allow my app to not "go to sleep".
[+] [-] eli|13 years ago|reply
I agree that Heroku could do a better job of explaining their offering, but I don't think they're doing anything wrong.
[+] [-] redguava|13 years ago|reply
I see no problem with their support response, it was accurate.
[+] [-] dsowers|13 years ago|reply
[+] [-] suresk|13 years ago|reply
What is the best way to host a bunch of low-traffic sites for less than $35 (I believe that is what it costs for the second dyno, required to keep your app from being spun down) each per month? The usual answer is "PHP", but I sometimes wonder if there'd be a market (and way to make money) for Heroku-like ease of deployment/management, billed based on resources (much like a VPS) rather than per application.
[+] [-] X-Istence|13 years ago|reply
It hosts email, various domains, static, non-static, Python, Ruby, Postgres, MySQL ... and I don't pay a cent more for each of those services.
One of the sites I host on there has been Reddited and it didn't even break a sweat...
[+] [-] mosselman|13 years ago|reply
Heroku is a great and easy service, but I find the add-on 'no lag for $35/month' which is what the dyno basically is to the poster, unacceptable. But if most commenters are willing to pay into the hype, go ahead.
Just go with Linode or appfog or something.
[+] [-] Turing_Machine|13 years ago|reply
The cheapest Linode is $19.95/month, and that's just a bare VM. You have to do all the sysadmin stuff yourself. For $15/month more, Heroku takes care of all that for you. If your time is worth anything at all, maintaining your Linode is going to burn up more than $15/month.
Now, granted, for some use cases the Linode makes more sense (I have one of those, too, and have been happy with it and the company overall), but I don't think either company is screwing anybody over. Both are offering good value on their products, but those products are different. Linode gives maximum flexibility, but at the cost of full responsibility. Heroku gives maximum convenience, but at the cost of less flexibility.
[+] [-] jtchang|13 years ago|reply
I agree though that if you are hitting the damn thing every second it shouldn't go idle.
[+] [-] aaronbrethorst|13 years ago|reply
[+] [-] 1123581321|13 years ago|reply
[+] [-] hayksaakian|13 years ago|reply
Forcing me to read in landscape.
http://www.imgur.com/SVebzkP.png
[+] [-] hayksaakian|13 years ago|reply
http://www.imgur.com/x8wCC8U.png
The same day as a post about about contrast.
[+] [-] Sami_Lehtinen|13 years ago|reply
Btw. Heroku and App Engine apps start fast, compared to what ep.io (free) app start took, it was more like 20-30 seconds than 2-5 seconds.
[+] [-] dsowers|13 years ago|reply
[+] [-] twanlass|13 years ago|reply
That and fundamentally (as you noted about being optimistic) you're paying $100+ a month for an app that gets zero traffic. That seems to be the core issue.
[+] [-] parasj|13 years ago|reply
[+] [-] endianswap|13 years ago|reply
[+] [-] nasmorn|13 years ago|reply
[+] [-] mrgoldenbrown|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] guywithabike|13 years ago|reply
[+] [-] ddagradi|13 years ago|reply
[+] [-] infinii|13 years ago|reply