Ask HN: I built a site that just went ridiculously viral. What do I do now?

[markbao]

Hey HN,

So I built http://threewords.me - literally an MVP that I posted on Facebook and my friends started using. Fast forward 3 days. The entire Twitter results page for "threewords.me" is of tweets that happened less than five minutes ago. The site grew 2x in pageviews over the past hour.

Two problems: 1) what do I do now? 2) how do I afford this?

Advertising? Hosting partner? ...daresay... investment?

Sorry to be brief. If you're curious, the stack is Rails + Ruby Enterprise Edition + Passenger + nginx, which isn't cheap like PHP to host. On the $40 Linode right now but maxing out CPU at 350%.

227K pageviews today. 50K uniques. 71% traffic referred. 8,285 users. 3,100 new users in the past 1 hour.

What happens now?

EDIT: Hello! This thread is not going unnoticed while I take my metaphorical fire extinguisher to the fires that are happening. Will reply soon.

EDIT: cranked the Linode up to 4096. $160 server, woo!

EDIT: David from Duostack (http://duostack.com) is helping with the load on his cloud Ruby platform. Many many thanks to him.

[jeromec]

No offense, but putting it on HN in the middle of traffic spike pressures would not help things. ;)

I was able to sign up and see just enough to get a feel before the site became unresponsive. I like that it's dead simple. I can also see how it's incredibly viral as it asks you to ask others to describe you -- very smart. At first glance it looks to me like something that could enjoy tons of traffic, but which would probably be short-lived. The challenge would be finding a way to have that traffic stick around...

I view this as a bit of a long shot, but I would do a few things. First, you've got to have the site handle the traffic. No advice on how, but find a way to get that done. Next, I would try to become known as the place to get a quick summary on anybody -- even celebrities. I imagine people will receive multiple adjectives from different people, but I'd have the site tally the three most popular and promote those as best describing of the user. :) I see a couple of possible pages for "viewer stickiness". The homepage could feature very popular users. Imagine showing the three most submitted words to describe /paulg or /marrington and other users with large responses... Next, on each user's page it could show their "friends" and the words that describe them. The site also provides the most recently submitted three words on the user's page. This might at least make each user curious about checking back at their own profile to see how people were labeling them. As for monetizing, that's probably with ads, as usual, but you have to be careful about when and where to put them. Like I said, probably a long shot, but who knows? Good luck!

[astrofinch]

>As for monetizing, that's probably with ads, as usual, but you have to be careful about when and where to put them.

How about analytics? For each person who writes about me, I could pay to see where in the world they lived, how long it took them to type in the three words (or even watch a movie of them typing them in), whether they clicked the link that was on my Facebook or the one on my Twitter feed (based on HTTP referrer), etc.

If you were confident you could give each user the ability to see this information for 1 user for free, and then have them pay if they wanted to see it for the rest of the users.

[phlux]

>>Next, I would try to become known as the place to get a quick summary on anybody -- even celebrities.

You mean like About.me is attempting to do? This is an interesting twist though...

However - the quick summary I look for on anyone of interest (professionally) is linkedin.

I do not and shall not have a facebook account.

[gdeglin]

I recently built a Facebook application that approached the same kind of growth.

Sites like this typically see eCPM between $0.10 and $0.30 for advertising. So you're looking at around $30/day revenue right now.

Consider switching to EC2. This will allow you to easily scale up to a more powerful server if growth continues or down to a less powerful one if traffic tapers off.

Depending on how you set things up, a common bottleneck for Facebook applications is API calls. Ensure you are not making any API calls from Rails, since this will cause Rails processes to hang until Facebook returns results. With high traffic this could easily cause your server to become overwhelmed. Move all API calls to Delayed Job or another background processing system if you haven't already.

It looks like you have relatively few remotely loaded assets, which is great. You can likely speed things up more by making sure that your database is well optimized for the server (Make sure query cache is enabled, and you have indexes on the right columns for example), and identify any actions that take a long time to return.

As for what to expect in the future, growth will not be infinite. There are a finite number of people on earth that are going to love your site and Facebook integration is a fantastic way to reach nearly all of those people in a very short amount of time. After a while most of those people will get bored and you will see traffic begin to fall. The best ways to address this are by consistently releasing new features to reengage users or by referring your users to other similar sites that you build.

[mw63214]

New Feature Idea if it's not already in the works: threewords.me places (foursquare?). would leverage current users engagement to tag locations with a dead simple description. could keep users coming back, provide utility(value)to businesses, and make you some more money. email me ([email protected]) if you want more ideas, although it seems you have your hands full as it is. Good Luck!

[klbarry]

Would it be possible to see your Facebook app? I'm curious to see what traits these fast growing apps have in common in person.

[jacquesm]

hey Mark, if you need server capacity I can set you up with four machines on very short notice, let me know. [email protected] (no charge, help you handle the spike and see if it has staying power without having to get in to long term contracts).

[jasonlbaptiste]

^ This is reason why I love this place and never believe the occasional "hacker news is going to shit" posts.

[markbao]

Jacques, you are, and continue to be, one of my favorite people on Hacker News. Thank you. Shooting you an email. :)

[cd34]

Where is your current bottleneck? mysql? check your slow query log, turn on the log-queries-not-using-indexes, do a quick analysis of those to see if you have queries that aren't using an index or queries that can be optimized. Probably 90% of the scaling problems are database issues.

The quick solution is to scale up your current linode package, however, I think you might find that your traffic is going to peak and will wane next week after everyone has gone back to school/work and is out of the holiday mindset at which point you can scale back.

Put in a throttle - /proc/loadavg

Limit it to X signups per hour, asking them to check back next hour. Collect email and notify when they can sign up. Point them at a facebook page, ask them to like the page, do announcements when the next 'batch' of users is able to sign up.

If you want some help analyzing the slow query logs and/or the normal logs, email me a url of a .gz/.bz2 of a reasonable snippet at [email protected].

Since you're running nginx, consider doing an alias and use proxy caching. I would doubt your static files are causing too many issues, but, if you can cache computationally expensive pages a little, that can help.

[Luyt]

Last week I optimized some MySQL queries which triggered the 'not-using-index' warning, even when I had all fields which were used for selecting and sorting indexed. I think a JOIN on a nested subselect somehow confused MySQL: when I duplicated some extra restraining WHERE clauses into the subselect, response shot up from 800msec to 12msec per query. Very nice. But the query still shows up as 'not-using-index', which is strange.

[kabuks]

Your site seems to be down.

Think about moving the code over to Heroku now, and keep scaling to meet demand.

If you don't have the cash, email me and I'll front you what you need to keep it up until you figure out how you want to roll with this. This is not a time to trip over pennies.

Congratulations.

[latch]

Might not be a time to trip over pennies, but there's also no reason to rush into things. Heroku has always seemed to be horribly overpriced as a scaling solution.

A more traditional approach of 1 web and 1 db server (possibly even physical boxes) might end up providing better value and give him enough time to figure out how to scale this horizontally on EC2 (the web part should be easy...remove any in memory state data, the db might be a bit more work).

[dools]

EDIT: HN doesn't render that code very nicely, here's a pastie: http://pastie.org/1416631

Hi there, your home page is very simple, render it in plain HTML and post the signup to queue.php.

In queue.php put:

<?php $data = serialize($_POST); mysql_connect('localhost','dbuser','dbpass'); mysql_select_db('dbname'); mysql_query('INSERT INTO cache(cache_data) VALUES(\''.mysql_real_escape_string($data).'\')'); echo 'YOUR REAL PAGE SHOULD BE HERE WITH A MESSAGE ABOUT BEING IN THE QUEUE'; ?>

The cache table should just be a PK cache_id and a text field (or perhaps mediumtext). I've suggested using a database instead of a filesystem based cache because, if you're doing this quickly, doing it using MySQL means you're less likely to run into file permissions or security problems.

Then you're at least capturing everyone's information whilst you figure out how to scale.

[nzadrozny]

You've since mentioned that you've gone for a 4 gig box with 55 Passenger workers. Definitely a good start.

If I were in your shoes, these three items would be next on my emergency-scaling checklist:

1. Database sanity check: Are you using indexes for calls like "Find user by username"? Does your database need its own dedicated box?

2. Pick up another box for memcached. Full-page cache everything you can with rack-cache. Don't worry about expiration, just use some reasonably low TTL to put a throttle on your read requests for now.

3. Move all of your static assets to S3 + CloudFront. Gzipped, with a far-future Expires header. While you're in there, move the JS to the footer of your layouts. No sense serving any of that stuff from your Linode boxes if they're as pegged as they sound from your other comments.

As to making money? No clue. Maybe I'll have an idea when I can get a request through and see what the app is all about ;)

One thought occurs: this seems to be a very vanity-driven kind of app. Find a way to charge $1, $2 or even $5 for some simple premium vanity-stoking feature and you could be in good shape. Better than advertising, in my book.

Have fun!

[jimboyoungblood]

1) In general, the database is always the bottleneck. It's easy to throw up more webservers- can't do that with db's.

2) Throw up an ad or two, with the goal of having the ad revenue pay your server costs. This will allow you to not worry about the financial situation while you're fighting fires, so you can defer thinking about investment etc until later.

(now after you have the servers tamed...)

3) Make sure you are incorporated. You probably want to be an S corp or LLC.

4) Look at your metrics and decide whether you have a sustainable property or not. (Look at your retention numbers) Based on the data, decide whether you have

(a) a self-sustaining cash cow, (b) something that can be huge and take over the world, or (c) a fad that will grow and die quickly.

If (b), and you need money to get there, start talking to investors. But don't take money too hastily- it's a long term commitment and you'd better know what you're getting yourself into.

If (c), try to sell it to someone who thinks it's a (b) before it dies.

Congrats and good luck!

[Natsu]

Not so sure about incorporating, but that's because I spent my day writing up notes on Camz's talk on incorporation. He gave the three reasons to incorporate:

1. You have significant profits or net income. 2. You have high liability. 3. You're getting funding.

I doubt this meets any of those criteria, unless someone offered to fund him.

Here are the relevant links if you want to see what I'm talking about: HN Story - http://news.ycombinator.com/item?id=2047818 Podcast - http://cameronkeng.com/podcast-ep-1-incorporation/ Summary/Notes - http://pastebin.ca/2033868

[jackowayed]

I'd say it has a high chance of being c. At least where die == level off/decline.

I see a lot of parallels to formspring, whose pagivews/month graph looks like this: http://www.quantcast.com/formspring.me

So you either want to sell as it nears the that peak, or you have to be ready to stick out the trough of sorrow and invest a lot of time into finding a way to grow and monetize it.

So if you have some great vision for how you can turn this into something awesome, super-popular, and monetizable and you want to invest the time in it, you should go for it.

But if not, you're probably going to get some very good offers over the next few days--especially compared to how little time you've invested. If I were you and wasn't totally in love with the site, I'd get out while the offers are good and invest that money into your other businesses. Then when your next site goes viral, you'll just have to wonder how to get it back up ASAP, not how to afford doing so :)

[rmc]

You can scale up databases, set up mysql replication, most of the queries on a page will be reads, just have multiple read only slave machines.

[rbxbx]

http://markbao.com/about

You're 17/18?

Kids these days. Geeze.

Fine work sir, though I don't have much advice to offer beyond what has already been stated.

[mickdarling]

Yeah, and I met Mark 3 or 4 years ago and he had already started at least one and maybe 2 successful startups.

[Skywing]

Well, it's a nifty idea and I know I've seen that trend on Facebook before. But that's also the problem - it's a 1 time use fad type thing. I don't know how much I'd invest in the current incarnation - perhaps just enough to keep the site functioning. Aim for some kind of traffic-based business strategy like ads or something. Maybe it can help pay for the site itself. If you could perhaps take the current idea and evolve it into something with a little more sustaining power then perhaps you could turn it into a nice little side income thing.

[phlux]

let the words become tags linked to the FB account. Such that if users have a tag like "Snowboarder" associated with their FB account - he can build out a tagDB of FB accounts.

This way - you could target ads to FB accounts/users that are all "snowboarders"

Or you can at least search who among your friends have the tag [snowboarder].

Or if you give more categories to the tags, you can ask your friends to fill in their favorite 3 [X] -- so you can simply ask all your friends a question and then build an interesting DB of tags/info/interests of all FB users.

[rapind]

The following until you no longer have issues handling the traffic (which might be right after step 2).

1. Upgrade your linode to something around 2 Gig for now.

2. Start playing with your MySQL or Postgresql config (assuming you're not already using redis or something similar, in which case just throw more RAM at the box). Google performance settings for your DB of choice and look for relevant posts to the memory you have.

3. If there's anything cachable, install memcached and implement it.

4. Purchase a second linode and move your database to it.

5. Upgrade each VPS as needed.

6. At this point you get into fancy scaling and there are loads of options. You should have plenty of breathing room by this point though and can plan it out.

[markbao]

Upgrading to the 4 gig linode as we speak. this is getting kind of ridiculous.

[ElbertF]

This has happened to me a few times and usually the Twitter/social media storm blows over fairly quickly. I suggest getting an extra Linode for now and wait a few days to see what happens. If the numbers do stay up the site will probably be able to pay for itself with ads. Your site is down at the moment so I can't really see what it's about.

Edit: Google Cache got it: http://webcache.googleusercontent.com/search?sclient=psy&...

[jackowayed]

Rather than get another Linode, for the very short term, I'd just crank the current Linode as big as they go (I assume Linode has super-quick resizing) and tell passenger to run a bunch more processes.

[jasonlbaptiste]

There isn't a twitter/social media storm ie- hype cycle. It's genuine growth with no press or even optimization for sharing built in. It can certainly die down, but it's doing it in a highly organic fashion.

[david_shaw]

Adsense. With only about 10k hits a day, my bedtime calculator http://sleepyti.me brings in $10-$20/day. At the rate you're getting users, you could probably bring in a significant load of cash. Good luck!

[KeithMajhor]

"Sleepyti.me is provided as a free service thanks to support from our ads."

I don't know if you read the "Sacked by a Google Algorithm" article from yesterday. http://www.duckworksmagazine.com/11/columns/guest/winter/ind... The comment you make on the results page sounds dangerously close to what got Adsense to boot him.

[csomar]

Depends on the niche and the website. I have made a blogspot blog and brought it to 4% CTR and $20 CPM. That's +$200 a day.

[bemmu]

Curious: What kind of traffic sources are you getting? I notice an FB like button, is that the sole source?

[lakrish]

Hey Shaw, it sounds interesting. Can u guide me about how u started this project and contacted Adsense. I wanna start one like this and make some earning.

[blader]

Advice from a guy who's gone through this a few times:

1. Monitor your retention. Are people coming back or just checking your site out once? You can go viral very quickly but tank just as quickly once the initial interest dies down.

2. Start thinking about a business model. Highly viral sites that don't retain well could really hurt your bank account if you're not monetizing. If you can't can come up with an interesting one, throw up some ads to tide you over.

3. As soon as possible, get your database on dedicated hardware. Your bottleneck is going to be the database: so memcache, add the right db indexes (get a backup db so you can hot-swap while you do this), and add lots of RAM if needed. You don't want to be scrambling to migrate your database once you hit the limit.

[pathik]

I don't suppose this traffic will sustain. Slap an ad right in the center and monetize the hell out of it. Or get a sponsor.

Also, for the short term, Amazon EC2 would be the best option to handle the traffic.

[epi0Bauqu]

I'll sponsor, or more accurately, DDG will.

[catshirt]

the site got twenty two hundred hits within two hours?

[judofyr]

In case you haven't watched it yet: this is a quote from The Social Network - http://www.imdb.com/title/tt1285016/quotes?qt1307141

[mindcrime]

Thousand... twenty-two thousand.

[markbao]

Two hundred twenty thousand. In two hours.

[nanexcool]

Thousand. Twenty two thousand.

[spullara]

My guess is this app doesn't need the resources he is throwing at it. If you look at the pages it is dead simple. I can't imagine what was using all the CPU. Would have been good to get a TOP or something to see what was going on. Since it is on Ruby, I recommend getting it on NewRelic and have it tell you where the bad parts are.

[cmelbye]

Heh, I was wondering what this was when I first saw it a few hours ago. I've seen 5-6 people I follow on Tumblr use this thing already.

I suggest migrating it to Heroku. Push your code, copy over the database, and crank your dynos up. They charge by the hour, so as an example if you have ~10 dynos running (roughly equivalent to 10 thin instances, but slightly better) for 5 days (or however long it takes for the initial traffic surge to subside), you're only going to pay about $50. Then, you can turn down your dyno count and pay a reasonable monthly fee.

[jarin]

There's a bit of magic involved with things like compass/sass and Paperclip on Heroku (you have to use Amazon S3 for storage and there is some other setting I can check on).

Anyway Mark, I have a bunch of apps on Heroku that use all kinds of wacky stuff—including the aforementioned gems—so email me at jarin (at) robotmodehq.com if you decide to try out Heroku and get stuck. I can send over working code snippets that you should be able to just drop in.

[markbao]

I haven't forgotten about this. I need sleep (exhausted and have been in bad health lately). Will get back to this soon.

[rms]

Don't forget to apply for the Thiel Fellowship! Email me for help with your essays (that offer applies to anyone).

[rdl]

Yes, if there is anyone I know who should apply for the Thiel deal, it is Mark. Especially since he has said he doesn't exactly love his college. Thiel should make moving to SF a requirement too.

[auston]

Mark already applied: http://instagr.am/p/vIqz/?ref=nf