"Disclaimer: This article is not about "I am so clever, Google is so stupid". This article is about some Google AppEngine problems (or peculiarities) which might not be obvious for newcomers."
It would be nice if the title reflected that sentiment.
There are many things to hate about AppEngine, but this is not one of them.
You're being aggressively rate-limited because your account does not have billing set up. If you only have the free quota to work with, they place continuous limits on resources to keep you from exhausting your daily supply early.
Even with billing, GAE expects gradual ramp-up of traffic, as would happen in the real world. Your app will fail to scale if you hit it with an artificial blast of benchmarked requests because the back-end wouldn't know to fire up more instances.
They also have an algo that protects against DDOS, and allows for resources to be allocated as organic (even viral) activity increases... see their article about load testing to see that you can get quite a lot of load by staggering it.
AppEngine is about scalability, not performance. You shouldn't care if a single hit on an AppEngine app takes 2 or 4 times as long as a single hit on your dedicated sever; the point is that when there are zillions of hits, your server will be on fire while AppEngine will be taking about as long as it did for that first hit.
Of course, this scalability is not without cost. Most web apps don't need this kind of scalability, so the development tradeoffs (like the chance of datastore operations failing, or the inability to wrap transactions around arbitrary sets of data operations) may be too much trouble.
AppEngine isn't for all web apps, and many web apps would be very painful to shoehorn onto AppEngine. Still, if you want to build your app so it scales like Google's own apps scale, AppEngine can help get you there.
There seem to be a few cases where it's far more than 2 or 4 times as long. I've found deleting takes ages - a couple hundred items can take a minute of "CPU" time. Discussions on the forum mirror that.
I think it's great for many individual users doing tiny operations that don't interact with others, which my use case is. But you will be surprised by some of the performance characteristics.
And what about a real-life case? I'm pretty sure that a GAE app can handle much more than what is said in the benchmarks.
In fact, firing 100 connections to the same host from the same IP is not a real-life case for me. But I'm pretty sure that those 50 req/sec (for example) aren't shared among all connected users to the same GAE host.
Anyway, I still don't like GAE but I think that this is not a real reason. And I personally don't like giving everything, including my apps, to Google. Uh.
I've started naming my App Engine apps bemmu1, bemmu2... so that if I decide to discontinue an app later, I can just reuse one without having an old name still attached to it. It doesn't really matter as the name would not usually be visible to a user.
Clearly GAE does not "suck". We've used it a lot, e.g. [1]. The principal advantages are:
- No sys admin. You can just focus on your app.
- The datastore (BigTable) is slick. Write and read times are independent of how much shizzle you stick in there.
- They're constantly making it better. E.g. the free quotas got more generous after a few months; and the new built in cron and off-line processing (taskqueue) support enables more complex apps.
Clearly not for everybody. We love it. Does not suck.
"All in all, we saw over 1,000,000 page views from 93 countries around the world, and experienced incredible stability and scalability from Google App Engine even when we were spiking 350 requests per second during the finals."
Dave Westwood of BuddyPoke shared a few slides on his performance:
It's a little hard to parse w/o more description, but basically he's done well serving what seems to be a large % of Brazil with a low error rate.
IMHO, the best ways to use app engine rely on (1) memcache to iron out any timeouts or datastore issues, and (2) a relatively rich client front-end so any problems on back-end can also be handled in a user-friendly way (like Gmail).
The issue is not with GAE, but his code. It may look simple, but actually it is not using the data store as one should. Putting a single entity repeatedly does not scale, this is well documented. In the case of the example, a sharded counter is needed: http://code.google.com/appengine/articles/sharding_counters....
Yes I've seen it. But it clearly shows some of the problems existing with GAE. It is possible to avoid them, as well as most of the rest, but we cannot deny the fact that there is a problem.
Also, there are read-only tests which are also not amazing.
I question the benchmarks that are claimed for a single server. 10 million hits per day? Stackoverflow is just approaching 1 million hits and they've just recently gone to 2 web servers. Never mind the fact that the DB is also on a different server.
The fact is that my argument makes as much sense as author's does. It all depends on what your server is doing.
The idea of GAE: we do care about scaling instead of you.
What I prefer to think: write proper code, and you will never need a cluster (probably not for all cases).
There are just 6.5billions people on the earth, so the maximum traffic is limited anyway.
It's about time. Google AppEngine (Map Reduce, GFS, etc) has always lacked behind the Dynamo architecture in terms of winning developer mind share and specing the right features. The inherent problem with Google App Engine is it's not backwards compatible with existing system.
[+] [-] boggles|16 years ago|reply
It would be nice if the title reflected that sentiment.
[+] [-] dchest|16 years ago|reply
[+] [-] mseebach|16 years ago|reply
[+] [-] blasdel|16 years ago|reply
You're being aggressively rate-limited because your account does not have billing set up. If you only have the free quota to work with, they place continuous limits on resources to keep you from exhausting your daily supply early.
[+] [-] ntoshev|16 years ago|reply
[+] [-] th0ma5|16 years ago|reply
[+] [-] shimon|16 years ago|reply
Of course, this scalability is not without cost. Most web apps don't need this kind of scalability, so the development tradeoffs (like the chance of datastore operations failing, or the inability to wrap transactions around arbitrary sets of data operations) may be too much trouble.
AppEngine isn't for all web apps, and many web apps would be very painful to shoehorn onto AppEngine. Still, if you want to build your app so it scales like Google's own apps scale, AppEngine can help get you there.
[+] [-] jbellis|16 years ago|reply
Actually, you should. Latency matters. Google knows this. http://radar.oreilly.com/2009/06/bing-and-google-agree-slow-...
[+] [-] richardw|16 years ago|reply
I think it's great for many individual users doing tiny operations that don't interact with others, which my use case is. But you will be surprised by some of the performance characteristics.
See this example: http://gaejava.appspot.com/
[+] [-] adrinavarro|16 years ago|reply
In fact, firing 100 connections to the same host from the same IP is not a real-life case for me. But I'm pretty sure that those 50 req/sec (for example) aren't shared among all connected users to the same GAE host.
Anyway, I still don't like GAE but I think that this is not a real reason. And I personally don't like giving everything, including my apps, to Google. Uh.
[+] [-] stevejohnson|16 years ago|reply
[+] [-] bemmu|16 years ago|reply
[+] [-] babyshake|16 years ago|reply
[+] [-] kljensen|16 years ago|reply
- No sys admin. You can just focus on your app.
- The datastore (BigTable) is slick. Write and read times are independent of how much shizzle you stick in there.
- They're constantly making it better. E.g. the free quotas got more generous after a few months; and the new built in cron and off-line processing (taskqueue) support enables more complex apps.
Clearly not for everybody. We love it. Does not suck.
[1] http://news.priorsmart.com/
[+] [-] lecha|16 years ago|reply
[+] [-] DocSavage|16 years ago|reply
http://googleappengine.blogspot.com/2009/09/agile-paddling-w...
"All in all, we saw over 1,000,000 page views from 93 countries around the world, and experienced incredible stability and scalability from Google App Engine even when we were spiking 350 requests per second during the finals."
Dave Westwood of BuddyPoke shared a few slides on his performance:
http://files.meetup.com/1135107/BuddyPoke_AppEngine_Stats.pd...
It's a little hard to parse w/o more description, but basically he's done well serving what seems to be a large % of Brazil with a low error rate.
IMHO, the best ways to use app engine rely on (1) memcache to iron out any timeouts or datastore issues, and (2) a relatively rich client front-end so any problems on back-end can also be handled in a user-friendly way (like Gmail).
[+] [-] bemmu|16 years ago|reply
[+] [-] BarsMonster|16 years ago|reply
Also, there are read-only tests which are also not amazing.
[+] [-] thetrumanshow|16 years ago|reply
One one project (RoR), I have a BIG-IP load balancer in front of 4 dedicated servers, one of which is a db.
On another project, I am running a straight Python app on App Engine.
Guess which one I've had to wake up in the middle of the night to fix on occasion...
[+] [-] sanswork|16 years ago|reply
[+] [-] damagednoob|16 years ago|reply
The fact is that my argument makes as much sense as author's does. It all depends on what your server is doing.
[+] [-] BarsMonster|16 years ago|reply
There are just 6.5billions people on the earth, so the maximum traffic is limited anyway.
[+] [-] gfodor|16 years ago|reply
Edit: Yup, AppEngine is hosed!
http://code.google.com/status/appengine/detail/datastore/200...
[+] [-] gfodor|16 years ago|reply
[+] [-] jaspertheghost|16 years ago|reply
[+] [-] tybris|16 years ago|reply
[+] [-] tybris|16 years ago|reply
[+] [-] onreact-com|16 years ago|reply
This type of "pulling a Calacanis" headline is sheer link bait and rarely meant to stir an honest discussion.
[+] [-] scrame|16 years ago|reply