HN policy for years has been to allow two kinds of job ad: (1) job posts by YC startups, and (2) the Who's Hiring threads. We kill job ads that are neither of these.
(Incidentally, PG designed the /jobs posts to be the one privilege that YC startups get on HN. Other than that, the site policy is to treat them the same way as everybody else.)
It's not uncommon for posts to be in a grey area between job ad and story. A programming test for an interview process doesn't seem all that grey, though, so I've been killing these. Sorry to be a hard-ass about this, but I don't want to be accused of enforcing the policy erratically.
Edit: I'll unkill the post (but otherwise leave it demoted) so the discussion can remain open.
Edit 2: The argument has been put to me that this is newsworthy because A16Z are doing it. Since that's corroborated by external sources (e.g. Twitter), and the post has gotten a lot of upvotes even while it was buried, that's reasonable—so we'll restore it as news rather than as an ad. Besides which, HN is one of the few places where a programming test counts as interesting.
Can an actual list of policies be posted somewhere ? - I don't ever recall there ever being a discussion of such a policy before. Certainly there have been highly voted non-HN job posts in the past (e.g. Khan Academy)
I wonder if this could act as a negative filter for good programmers. Who worth their salt is going to invest 3 hours of their time just to submit a resume? Wouldn't you want to talk to the employer first to see if they're a good fit?
In a tight labor market, people willing to do this are more desperate than others. So it's going to filter for desperate developers who passed a 3 hour basic skills test.
Old fashioned prior experience and education is probably still a better first filter. This would probably make more sense specifically for junior hires right out of school or with no experience.
Well, empirically we're getting quite a lot of people attempting it! :) Three hours is pretty lenient (you could blaze through it in less time), though the last question is challenging. We think the questions are kind of fun as well.
Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.
one of the best programmers i know refuses to continue with a job application if there is a programming test... its a bit eccentric but valid given that his experience and past achievements say more than a programming test ever could.
3 hours is upper bound and extremely friendly one to allow for all but hopelessly incompetent candidates. I did the test in less than 90 min. First 27 questions should take a competent hacker less than 20 min; rest of the time is for last 3 coding assignments.
I'm saving it for the weekend. Will read about all the virtualization techniques for those spoj like sites and then have a go in 3hours trying to break that one :-)
This doesn't quite make sense to me. Or, it makes sense, but not for the reasons presented on the surface.
If you're Andreesen Horowitz, you have access to a nearly-unlimited supply of capital. You also have incredible access to information and dealflow throughout the industry. In situations in which you are not capital constrained, the most efficient way to establish a workforce is to acquire it from somewhere else; that's why acqui-hires make so much sense, because you've basically had someone else do the hard (and often completely unpredictable) work of figuring out if people can work together in a successful manner (which, I should note, is different from determination of whether the thing they're working on is successful). If I were A16Z, I would arbitrage my information advantage to secure a high quality engineering team by acquiring a company that had smart people but a lame product; going through a traditional hiring process would be a waste of my time (and since everyone is throwing money at my firm, it would also be a waste of time that could be better spent on figuring out where to deploy my capital).
A16Z is rich enough to buy people, if it really wants people, for internal use (whether it is for their own company needs, or for some idea that they want to incubate inside of the firm). And even if they don't want to buy a whole set of people, they have enough wisdom and experience to know that you want to hire within a cohort of successful people rather than randoms (to see this in practice, take a look at Don Melton's post about Eazel alumni ending up at Apple on the front page right now). This tells me that their programming test has more to do with some sort of evaluation of this Hacker Rank service ("tell you what, we will run our own test on the service to see what happens!") or some other sort of analytical adventure.
Or, you know, maybe it is really a programming test after all. Maybe they are hoping to find some sort of diamond-in-the-rough engineers who are socially and career-wise disconnected from an existing network. Some sort of "black swan programmer" or something. But I highly doubt it.
So basically, if you go through this test, you're helping A16Z evaluate something; that something, however, probably isn't you.
Even Andreessen is unlikely to be structured such that spending millions of dollars at a time on things that don't make returns is a sensible thing to do.
You have to understand the way capital inside venture funds is usually allocated - it's not just one big checking account. A common way to describe the economics of a venture fund is "2 and 20" which means that the fund takes 2% management fees per year (across a 10 year fund lifetime) and takes 20% of the returns or upside.
So, in a hypothetical $300m fund with this arrangement, 80% of the money is reserved for making investments that provide returns. Every time an investment is made, a capital call takes place where the LPs are asked for the money.
The other 20% of the fund goes to management fees - 2% per year although it's not uncommon that these fees be non-linearly distributed, IE 3% in the early years and 1% in the out years when (presumably) new funds are also throwing off management fees.
So your $300m fund really generates $500k / month in management fees. (Probably paid annually or quarterly.) Enough for some very comfortable salaries, ample support staff, great office space and a travel budget but not enough to really start throwing dollars around millions at a time.
Now of course Andreessen has a reported ~$4B under management but I'd be surprised if they're really generating north of $6M in management fees per month. As fund size grows, management fees may come down (since it doesn't take 10x the effort to write checks 10x as large) or a fund as confident as Andreessen (and in as strong a position) might make a "1.5 and 30" deal where they take fewer dollars as management fees up front but get a larger percentage of the upside.
Note, I know absolutely nothing about what's really happening inside Andreessen, this is a purely hypothetical story against what is considered "typical" in the industry. (In the US.)
I think you're reading too much into this. Sometimes a programming test is just a programming test.
If I were to guess, they're building an internal dashboard and DWH to crunch numbers on their investments (or perhaps potential investments). This test appears to focus on that skill set.
Yeah, they could probably buy a company, but that would be overkill. They're VCs, they're supposed to be selling companies at a premium, not paying premiums. ;)
Why acquihire someone who knows how much they can ask if you can pay close to nothing to that test website (it's a genetic service) and get a clueless buy competent nerd that will be glad to work alone on a project for peanuts?
That's why we're poor and they have "unlimited capital"
Not to sound offensive , but you're not A16z.
1.<< A16Z is rich enough to buy people>> No one is rich enough to buy people , unless well if you're talking about criminals.
2.Most good engineers will tell you , that money is NOT the primary motivator for them to build great products.( So aqui-hiring won't work)
3. As for Hackerrank--> that's what they're supposed to do - find people with good technical skills.
Remember that VCs have money to spend on acquisitions for their investors, not for themselves. It's very rare to see VCs acquire talent for themselves. It's usually personal network based hiring. What makes this unique is that it's a talent test first - groundbreaking in an industry that normally hires based on personal connections.
The real question is "What problems will these folks be solving?"
An entire page detailing what you want from me, coupled with a 180 minute test. Yet in all of this there are only 4 bullet point sentences describing what you're offering me. That deal is not for me.
So, someone taking Statistics 101 is considered to know machine learning in the startup circus? (Not that this would really surprise me, given the tendency to appropriate terms that had a certain meaning for purposes that have only a passing similarity to the original meaning.)
As an employer, I would seriously hope this does become the norm.
If some employers want to implement recruiting processes that require developers to complete tasks that don't reflect real-world development, it makes it that much easier for other employers to differentiate themselves and attract competent developers who aren't interested or desperate enough to play these games.
Frankly, in today's market I feel sorry for any experienced developer who is going to complete a "programming test" in which referencing a website or book other than language documentation is forbidden. I reference websites and books all the time, and I don't know a single developer who doesn't.
Seems like the obvious (if dishonest) thing would be to use a throwaway phone # to get the test, then spend a day or so solving it, then type it in (with plausible delays) using your real number a day later. Assuming the test is roughly the same each time (even if details are different, it's probably of the same general class).
In grading, the results of 6-8h of work entered in 2h would look more impressive than 3h of work in 3h, I suspect.
(Sometimes I hate that whenever someone gives me a situation, my first thought is "how can this be cheated or gamed", followed by "what controls can I put in place to prevent that, and and what costs"; it's basically automatic.)
Probably to build internal tools. A lot of VCs firms are trying to apply more data to their investment decisions and that requires internal tools to handle it all. Also makes sense given the qualifications they are asking for.
My guess is to have an in-house team that can help out the companies they fund. I know that Netflix's cloud architect, Adrian Cockcroft, left to join a venture capital firm to advise on scaling issues for companies that firm funded[0].
My guess would be a team to provide a data-centric approach to investment. Crunch a whole lot of numbers to identify industry trends and specific firms that are promising, and see if those can replicate human performance.
Or they're simply handling recruitment for a stealth portfolio startup.
Does this mean Mattermark https://mattermark.com/ have started a tech arms war amongst the larger VC firms to create their own toolsets to differentiate their deal flow from others?
Mattermark has gone some way to provide proof that data can make a difference, at least at highlighting opportunities that might not have been visible before. I wondered whether this would result in herding investors towards some companies, which means differentation and opportunity comes from spotting things the others miss... finding the outliers that Mattermark miss.
Interesting that the test is "make a dashboard", well into the standard skillset for most analyst programmers who work for financial institutions (I built a lot of this stuff for the finance sector in London).
The overwhelmingly largest share of returns accrue to capital. It's a good idea to resist being cast in the role of simply "programmer" over time, for this reason.
My problem is I like taking these tests to find out what I don't know (quite a lot usually). To be fair though code reviews usually tell me more than tests these days.
The constraint that gets me is the honor code - I cannot use an IDE with code completion?! but that leaves nano or notepad ...
The test's restriction on consulting material from websites and books, excluding the official reference documentation boggles my mind. Unofficial 3rd party documentation is standard kit for a developer, especially when using community driven programming languages.
Have you guys thought about possibly employing smarterer.com rankings? I took your test, I think smarterer may yield a better evaluation, although you can't write code on a clock.
This is a very difficult question and, thus, illustrates
the transcendent brilliance and astoundingly
"deep domain knowledge" of A16Z. An answer is
available only on the bleeding edge of the
most brilliant insights of research in
machine learning. Ordinary people can't expect
to understand such depth! All bow down to
the incandescent brilliance of A16Z -- after all,
one of their founders used C++ to write
a GUI application to read a text markup
language and display the formatted results!
Maybe this was the first time, I mean after
Word, TeX, and some others?
[+] [-] dang|12 years ago|reply
(Incidentally, PG designed the /jobs posts to be the one privilege that YC startups get on HN. Other than that, the site policy is to treat them the same way as everybody else.)
It's not uncommon for posts to be in a grey area between job ad and story. A programming test for an interview process doesn't seem all that grey, though, so I've been killing these. Sorry to be a hard-ass about this, but I don't want to be accused of enforcing the policy erratically.
Edit: I'll unkill the post (but otherwise leave it demoted) so the discussion can remain open.
Edit 2: The argument has been put to me that this is newsworthy because A16Z are doing it. Since that's corroborated by external sources (e.g. Twitter), and the post has gotten a lot of upvotes even while it was buried, that's reasonable—so we'll restore it as news rather than as an ad. Besides which, HN is one of the few places where a programming test counts as interesting.
[+] [-] ig1|12 years ago|reply
[+] [-] zodiac|12 years ago|reply
I'm quite interested in those tests, could there possibly be a way to see them?
[+] [-] unknown|12 years ago|reply
[deleted]
[+] [-] rvivek|12 years ago|reply
[+] [-] abalone|12 years ago|reply
In a tight labor market, people willing to do this are more desperate than others. So it's going to filter for desperate developers who passed a 3 hour basic skills test.
Old fashioned prior experience and education is probably still a better first filter. This would probably make more sense specifically for junior hires right out of school or with no experience.
[+] [-] balajis|12 years ago|reply
Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.
[+] [-] jheriko|12 years ago|reply
[+] [-] lordmax|12 years ago|reply
[+] [-] gcb0|12 years ago|reply
[+] [-] unknown|12 years ago|reply
[deleted]
[+] [-] numair|12 years ago|reply
If you're Andreesen Horowitz, you have access to a nearly-unlimited supply of capital. You also have incredible access to information and dealflow throughout the industry. In situations in which you are not capital constrained, the most efficient way to establish a workforce is to acquire it from somewhere else; that's why acqui-hires make so much sense, because you've basically had someone else do the hard (and often completely unpredictable) work of figuring out if people can work together in a successful manner (which, I should note, is different from determination of whether the thing they're working on is successful). If I were A16Z, I would arbitrage my information advantage to secure a high quality engineering team by acquiring a company that had smart people but a lame product; going through a traditional hiring process would be a waste of my time (and since everyone is throwing money at my firm, it would also be a waste of time that could be better spent on figuring out where to deploy my capital).
A16Z is rich enough to buy people, if it really wants people, for internal use (whether it is for their own company needs, or for some idea that they want to incubate inside of the firm). And even if they don't want to buy a whole set of people, they have enough wisdom and experience to know that you want to hire within a cohort of successful people rather than randoms (to see this in practice, take a look at Don Melton's post about Eazel alumni ending up at Apple on the front page right now). This tells me that their programming test has more to do with some sort of evaluation of this Hacker Rank service ("tell you what, we will run our own test on the service to see what happens!") or some other sort of analytical adventure.
Or, you know, maybe it is really a programming test after all. Maybe they are hoping to find some sort of diamond-in-the-rough engineers who are socially and career-wise disconnected from an existing network. Some sort of "black swan programmer" or something. But I highly doubt it.
So basically, if you go through this test, you're helping A16Z evaluate something; that something, however, probably isn't you.
[+] [-] oroup|12 years ago|reply
You have to understand the way capital inside venture funds is usually allocated - it's not just one big checking account. A common way to describe the economics of a venture fund is "2 and 20" which means that the fund takes 2% management fees per year (across a 10 year fund lifetime) and takes 20% of the returns or upside.
So, in a hypothetical $300m fund with this arrangement, 80% of the money is reserved for making investments that provide returns. Every time an investment is made, a capital call takes place where the LPs are asked for the money.
The other 20% of the fund goes to management fees - 2% per year although it's not uncommon that these fees be non-linearly distributed, IE 3% in the early years and 1% in the out years when (presumably) new funds are also throwing off management fees.
So your $300m fund really generates $500k / month in management fees. (Probably paid annually or quarterly.) Enough for some very comfortable salaries, ample support staff, great office space and a travel budget but not enough to really start throwing dollars around millions at a time.
Now of course Andreessen has a reported ~$4B under management but I'd be surprised if they're really generating north of $6M in management fees per month. As fund size grows, management fees may come down (since it doesn't take 10x the effort to write checks 10x as large) or a fund as confident as Andreessen (and in as strong a position) might make a "1.5 and 30" deal where they take fewer dollars as management fees up front but get a larger percentage of the upside.
Note, I know absolutely nothing about what's really happening inside Andreessen, this is a purely hypothetical story against what is considered "typical" in the industry. (In the US.)
[+] [-] nostromo|12 years ago|reply
If I were to guess, they're building an internal dashboard and DWH to crunch numbers on their investments (or perhaps potential investments). This test appears to focus on that skill set.
Yeah, they could probably buy a company, but that would be overkill. They're VCs, they're supposed to be selling companies at a premium, not paying premiums. ;)
[+] [-] gcb0|12 years ago|reply
Why acquihire someone who knows how much they can ask if you can pay close to nothing to that test website (it's a genetic service) and get a clueless buy competent nerd that will be glad to work alone on a project for peanuts?
That's why we're poor and they have "unlimited capital"
[+] [-] balajis|12 years ago|reply
[+] [-] pskittle|12 years ago|reply
[+] [-] mathattack|12 years ago|reply
The real question is "What problems will these folks be solving?"
[+] [-] xedarius|12 years ago|reply
[+] [-] eternalban|12 years ago|reply
[+] [-] _pmf_|12 years ago|reply
So, someone taking Statistics 101 is considered to know machine learning in the startup circus? (Not that this would really surprise me, given the tendency to appropriate terms that had a certain meaning for purposes that have only a passing similarity to the original meaning.)
[+] [-] balajis|12 years ago|reply
But it's amazing how far you can get with some simple scatterplots, histograms and linear/logistic regressions.
a) Most of the time, more data beats better algorithms: http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...
b) Even those seemingly simple things get complicated when you have a lot of data: http://komarix.org/ac/lr
[+] [-] tpae|12 years ago|reply
[+] [-] 7Figures2Commas|12 years ago|reply
If some employers want to implement recruiting processes that require developers to complete tasks that don't reflect real-world development, it makes it that much easier for other employers to differentiate themselves and attract competent developers who aren't interested or desperate enough to play these games.
Frankly, in today's market I feel sorry for any experienced developer who is going to complete a "programming test" in which referencing a website or book other than language documentation is forbidden. I reference websites and books all the time, and I don't know a single developer who doesn't.
[+] [-] rdl|12 years ago|reply
In grading, the results of 6-8h of work entered in 2h would look more impressive than 3h of work in 3h, I suspect.
(Sometimes I hate that whenever someone gives me a situation, my first thought is "how can this be cheated or gamed", followed by "what controls can I put in place to prevent that, and and what costs"; it's basically automatic.)
[+] [-] nilsbunger|12 years ago|reply
[+] [-] cnaut|12 years ago|reply
[+] [-] Oculus|12 years ago|reply
0 - http://gigaom.com/2014/01/07/netflixs-cloud-architect-adrian...
[+] [-] neil_s|12 years ago|reply
Or they're simply handling recruitment for a stealth portfolio startup.
[+] [-] grosen|12 years ago|reply
[+] [-] gautamsivakumar|12 years ago|reply
[+] [-] riffraff|12 years ago|reply
[+] [-] buro9|12 years ago|reply
Mattermark has gone some way to provide proof that data can make a difference, at least at highlighting opportunities that might not have been visible before. I wondered whether this would result in herding investors towards some companies, which means differentation and opportunity comes from spotting things the others miss... finding the outliers that Mattermark miss.
Interesting that the test is "make a dashboard", well into the standard skillset for most analyst programmers who work for financial institutions (I built a lot of this stuff for the finance sector in London).
[+] [-] joncooper|12 years ago|reply
The overwhelmingly largest share of returns accrue to capital. It's a good idea to resist being cast in the role of simply "programmer" over time, for this reason.
[+] [-] elnate|12 years ago|reply
[+] [-] rmmw|12 years ago|reply
[+] [-] balajis|12 years ago|reply
[+] [-] lifeisstillgood|12 years ago|reply
The constraint that gets me is the honor code - I cannot use an IDE with code completion?! but that leaves nano or notepad ...
[+] [-] nostromo|12 years ago|reply
"You may use reference language documentation or use an IDE that has code completion features. "
[+] [-] thomaslangston|12 years ago|reply
[+] [-] jqm|12 years ago|reply
Hopefully they send some feedback... I'm not looking for a job, just wanted to know how I would do if I were:)
Besides, I love programming challenges.
[+] [-] jqm|12 years ago|reply
Something to be aware of if you are considering doing this... it looks like it really is for employment and not a challenge.
I guess one of the first question "Are you ready to go to work in Menlo Park CA" (No) should have probably clued me in:)
Oh well... they will "be in contact". Right.
[+] [-] 6thSigma|12 years ago|reply
A more realistic programming test would be a single, complicated problem with an "end of day" time restraint.
[+] [-] dpweb|12 years ago|reply
[+] [-] Oculus|12 years ago|reply
[+] [-] minikomi|12 years ago|reply
[+] [-] ojbyrne|12 years ago|reply
[+] [-] graycat|12 years ago|reply
[+] [-] argonaut|12 years ago|reply