top | item 399863

After Credentials

186 points| mqt | 17 years ago |paulgraham.com | reply

201 comments

order
[+] aristus|17 years ago|reply
I think Paul is treading ideas with some interesting implications for him, personally. He is the head of an organization that funds companies on the basis of a written & oral exam, whose principal selling point is the transmission of "credentials" upon the kids they pick. Part of it is the kids' merit, but part of it is YC's past success.

People go to YC for many of the same reasons people went to Stanford -- past seed stage, it's still largely a credentials game. So you have to ask whether the same potential for "corruption" is present. Are they failing to transmit credentials over generations because it's a revolutionary model, or because it's only been around less than one generation?

What if someone did a sweetheart deal for a YC startup as a personal favor/quid-pro-quo/etc? What does it mean when founders who got rich from a YC company start funding new YC companies because they are YC companies? Is that "corruption" in the same sense?

[+] pg|17 years ago|reply
Some of what you say is true, but it's not true that the principal value of YC is the brand name. The YC brand has little weight to the customers of a startup, and without those it quickly dies. YC's principal value is the advice we (and the other YC startups) offer.

The part you're right about is that "admission" to YC is structurally a lot like college or grad school admissions. The difference, though, is that our choices are immediately tested. College admissions officers do not, as far as I know, follow up on the careers of the people they accept and see how they did. Whereas if we pick a bad group, that fact will immediately be thrust in our face. This causes us to work very hard to try to pick the best people.

It was being in a sense a colleague of college admissions officers that made me think about such topics. We're the next bottleneck downstream from them, and it quickly became clear to us what a bad job they were doing:

http://paulgraham.com/colleges.html

[+] fallentimes|17 years ago|reply
We're (TicketStumbler) a pretty good example of a startup that hasn't really been affected by being a YC company. [1]

I did YC because it seemed interesting and I needed a good excuse to quit my job. [2] Tom was bored and wanted to get out of Ohio.

When we launched on Techcrunch, we were met with widespread apathy. Techcrunch readers, by and large, weren't our target market and didn't give two shits about a ticket site. We expected this to happen, and it was still cool to be featured on Techcrunch anyways :). We weren't creating a new platform or becoming the next facebook - we were just solving a simple problem (finding tickets).

The investors we met with didn't care that we were with YC (many didn't even know), they cared that we were making money in an expanding market.

Also, our most targeted/best traffic has been from places that didn't know/didn't care who YC was (e.g. Ticketnews.com, Thrillist). Same situation for our users.

Finally, as Paul alluded to, the YC name can only do so much for you, if you don't have a good product none of it matters.

[1] - I'm only talking about the YC brand name/credentials, not the program itself, which certainly had a positive impact on TicketStumbler.

[2] - When much of your family lives in Detroit and has been recently or is, unemployed, it's difficult to say your ditching a cushie 60k per year job to do a startup. Having "backing" makes it much easier.

[+] cperciva|17 years ago|reply
PG claims that a shift towards smaller companies results in people being judged on their performance rather than based on factors which do rather poor jobs of predicting said performance; my experience with tarsnap indicates otherwise.

I have a small number of customers -- somewhere between 10% and 20% -- who are using tarsnap based on a recommendation from another tarsnap user; but the vast majority of tarsnap users signed up based on some other factor which is at best predictive of the quality of tarsnap. Reasons I've heard include:

  * I am the FreeBSD Security Officer,
  * They find my blog to be interesting and well-written,
  * I have a doctorate from Oxford,
  * They like the fact that I made the tarsnap client source code available, and
  * I beat them on a mathematics competition 10 years ago.
There are a great many instances of companies succeeding with inferior products as a result of having superior marketing; while I am fortunate that people are interested in using tarsnap for reasons which are quite unrelated to the quality of the tarsnap code or service, the fact remains that the market isn't (in PG's words) "[taking] every organization and [keeping] just the good ones" -- rather, the market takes every organization and keeps just the ones which rank best on a nebulous combination of quality, marketing, and luck.

Yes, credentials are an imperfect way of predicting quality; but are they any worse than the marketing and luck which the market introduces?

[+] pg|17 years ago|reply
Initially the users of a new product will have random motivations. They can't all come from recommendations when you've only just launched. But a year from now the number of users you have will depend mainly on how good tarsnap is.
[+] maurycy|17 years ago|reply
I think that in case of tarsnap it is relatively hard to make the decision if you don't have basic knowledge in cryptography, at least. That's why the information assymetry occurs and thus signalling has much more impact on it.
[+] alexandros|17 years ago|reply
Paul seems to be hitting on the optimization by proxy issues seen in many areas, even in pagerank. Google looks not for quality of a page but for indicators of that quality, namely inbound links by quality websites. Similarly exams and credentials do the same. If you can write about these books, you know literature. If you got a degree there, you must be very capable. And in the first generation it works.

However, as people become aware of the ranking mechanism, they try to get ahead by overoptimizing for the narrow areas inspected. So pages do linkfarms and black-hat SEO, students goto cram schools, etc. etc. essentially producing the exact opposite of what was initially sought. I believe Joel spolsky had a good article on a simillar pattern in sales.

The question is if an algorithm that affects its input data can be made so that it adapts to the changes and gradually reaches some kind of equilibrium with the input data or stabilizes in some other way. As paul suggests, one way would be to make the system more transparent, or even 'social' such that the participants have a way to update it in an agreed way. This would keep the minority from cheating on the majority.

Doing away with such filters though might be a bit harder than we think, especially given the breadth of their applications. As aristus commented, even YCombinator has an examination system. In fact it wouldnt be too hard to imagine cram schools for YC given enough incentive. I fear this is a problem we have to either solve or live with.

[+] netcan|17 years ago|reply
How would you run a cram school for YC?

Say you were given 19 yr olds at the end of Year 1 of Uni. You have 2-3 years part time. How would you get them in? Activities? Interview coaching? Resume building? Portfolio building? Specialised learning?

[+] ryanwaggoner|17 years ago|reply
I fear this is a problem we have to either solve or live with.

Aren't they all? :-)

[+] edw519|17 years ago|reply
One could easily amend this to include "After Perceptions".

As a young geek, I was often passed over because I didn't look like I could do the job. It started with little league and continued on through high school and into the work place. I even quit one of my better jobs because a slicker looking but far less talented peer was promoted into what should have been my job.

No more. As a mild mannered adult geek, I still get the same pseudo-negative first impression reaction until I open my mouth and show them how much they'd benefit from me and technology.

The Performance Economy was a long time coming (and probably taken for granted by anyone under 25), but now that we're there, there's no going back.

[+] davidw|17 years ago|reply
> until I open my mouth and show them

On a complete tangent, this is one of the perks of living in a foreign country if you speak the language well - instant recognition that you have mastered their language, and that you must therefore not be a complete moron.

The downside of this in Europe, for me, has been moving about 4 hours away (absolutely nothing in the US, in terms of distance) and completely losing this advantage.

[+] bayam|17 years ago|reply
Ask someone who works at Google if there was ever a time when in the company's history when a Stanford or Berkeley degree wasn't a major advantage in the hiring process. Even at a small startup, you're probably going to have more faith in an unknown from Stanford than an unknown from Western Kentucky U.

Paul's essay is interesting, but I don't think any of recent academic / sociological studies would confirm his conclusions. In general, the best indicator of one's success is (a) your parent's income and (b) your education. Academic connections open so many more doors and opportunities, even to those who don't have the intelligence and skills to match. An idiot who graduates from an unknown college will probably go nowhere, but one from Stanford may end up as CEO. (Personally, I've seen the latter occur.)

[+] mstoehr|17 years ago|reply
I've definitely seen evidence that the school one goes to correlates with success, but are you sure that the causality flows in the direction: academic credential --> success? It seems plausible at least that if the parent is wealthy, seeing to it that their kid is well-educated is a good way of signaling that their kid is decent material, it's a costly signal so only the wealthiest of parents will probably be able to afford it. And since the kids of wealthier parents tend to do better, the education signal, by some accident ends up being used a great deal to gauge future success.

I actually think that it used to just be a straightforward signal of wealth, but these days we don't want to admit that that's the case, we instead want to justify it as producing some useful economic product.

[+] nostrademons|17 years ago|reply
"Ask someone who works at Google if there was ever a time when in the company's history when a Stanford or Berkeley degree wasn't a major advantage in the hiring process."

All my info is based on what they said (and hence somewhat unreliable), but the factors cited in my case were my startup experience, the interview feedback, and the fact that my skillset happened to fit what they're looking for at this point in time. I also have an Amherst degree, so I can't rule it out as a factor, but nobody once mentioned it. And my GPA sucked, so it certainly couldn't have been all academic.

[+] smanek|17 years ago|reply
I honestly don't think that 'cram schools' are particularly effective. I taught for the Princeton Review for a while ($20+/hr when I was 18 sounded pretty good). In my experience, the average student would only gain about an extra 50 points the SAT (relative to what they would have gained studying on their own). If memory serves, a standard deviation (on the old 1600 point test) was usually in the neighborhood of a 110 points.

Increasing your score by less than half a standard deviation was nice, but it rarely made or broke your college application.

[+] tokenadult|17 years ago|reply
On the one hand, high SAT scores are still rather scarce,

http://talk.collegeconfidential.com/college-admissions/41382...

but on the other hand, any one applicant with perfect SAT scores can still be rejected by plenty of colleges.

http://talk.collegeconfidential.com/parents-forum/377882-how...

Agreed with the point that on a dollar-per-SAT-point basis, there is little rationale for choosing commercial SAT preparation over simply self-studying with the released previous exams. The most decisive thing to do to raise SAT scores is to read widely and avidly in English.

[+] rsheridan6|17 years ago|reply
Applications to top schools are extremely competitive; I could easily see 50 SAT points making or breaking an application. I was under the impression that these days your application has to be more or less perfect to even have a chance to get into an Ivy League school, unless you're a legacy or a URM.
[+] apu|17 years ago|reply
In India they seem to make a huge difference. Friends and relatives who went to "IIT Preparation Courses" (6-12 months, at a boarding school, for getting into one of the IITs, the top technical colleges in the country) usually did vastly superior to others (of similar ability) who did not.
[+] Dilpil|17 years ago|reply
But then again, how many of those students would have studied alone if not for the class?
[+] jackchristopher|17 years ago|reply
Paul, how much of the conclusion did you decide on before starting the essay?

It felt more like an attempt to convince us, rather than a revelation to yourself. Not that that's bad, but you didn't seem excited.

What were your diffs?

[+] pg|17 years ago|reply
Actually there were a lot of surprises. That credentials are seals between generations; the distinction between direct and indirect methods of helping your kids; that sealing off direct methods promotes indirect ones; the emptiness of the explanation schools give of legacy admissions (which now seems to me as damning evidence as the practice itself); the precise connection between credentials, measurement, and organization size; the fact that falsely impressing colleagues in a small organization wouldn't have a lasting effect, because the organization wouldn't last; the reason the word yuppie arose, and the connection between startup founders and yuppies; that the spread of measurement would bring credential granters into line as well; and that the left, because their attachment to equality (of result, not opportunity) makes them dislike markets, have to fall back on a broken method of preventing transmission of power between generations.

I think the reason I didn't sound excited was the way this got written. I started it this summer, didn't work on it for months, and then started again in November. I rewrote it so many times that all that's left of the first version is the beginning, up to "Chinese example," and one paragraph in the middle that used to be in the beginning.

[+] DThrasher|17 years ago|reply
One important factor Paul left out was the ability to terminate bad employees. If an organization can't get rid of poor performers at reasonable cost, due to restrictive labor contracts or tenure for example, the organization will have to put much more effort into screening candidates.
[+] miked|17 years ago|reply
The usefulness of measuring depends on how actionable the measurements are. I think this is one of the big reasons that the startup culture in Europe has never really developed. In most European countries (and US government agencies) it's almost impossible to fire someone after a short probationary period. This distorts labor markets so badly that even left-leaning labor economists (e.g., Brad DeLong) have sharply criticized European labor laws.

This brings up an interesting sorting issue. If it's easy for people to be terminated and/or quit (because they know that there are other jobs, available in part because other employees have been fired or quit), there should, over time, be an improvement in the matching between employees and jobs. This should lead to higher productivity as well, not to mention greater job satisfaction.

[+] pg|17 years ago|reply
This is true. I didn't go into detail about the reasons large organizations can't cheaply measure the performance of recruits, but this is the sort of thing I meant.
[+] BKmke|17 years ago|reply
PG, I enjoy your essays a great deal. Often while reading the content I click on your numerically referenced notes. Clicking on the numbered reference takes you to the details of that note. Please link the number in the notes section back to the location in the essay. This way I can click the note in the text and read the note and then click the note reference to return to the essay content and continue reading. This will prevent me from having to scroll back up to place where I left off. I would appreciate this user interface improvement. Thank you again for your thought provoking work.
[+] msg|17 years ago|reply
use the back button?
[+] rakesh_d|17 years ago|reply
You seem to be overlooking the fact that most of the times the academic credentials of a candidate tend to be directly proportional to his/her performance. In places like South Korea or India (my place), it is damn hard to find qualified people for the posts of professors even in 'Institutes of Excellence' like the IITs. There's an astronomical amount of gap between the amount of funding a normal university receives and the grants that IITs receive. It should be then very easy for a prospective employer to decide between a candidate from IIT and one from a normal university. The candidate from IIT most probably (99.99%) outperforms the other guy.

That's why admission to the IITs becomes the all-important thing not only for the students, but also to the parents whose only hope to get rich is through their wards. The offshoring / outsourcing phenomenon further catalyzes the sexiness of 'Brand IIT' as the students from IIT end up with offers from MNCs, which are far more lucrative than what the Indian Space Research Organization offers to its most senior scientist.

IITs get to chose their students from a vast pool of 500,000 students who cram day and night for more than 3 years during their high school. This further adds to their glamor. Hence it is not uncommon to find companies preferring a chemical engineering graduate from an IIT over a computer science graduate from other universities for the job of a programmer.

Thus third world citizens still live in the age where "Credentials Do Matter".

[+] Shamiq|17 years ago|reply
For those who care, there is a whole branch of labor economics surrounding this topic:

http://is.gd/c0a2

(The wikipedia link had a set of paren that killed the URL)

[+] taw|17 years ago|reply

[deleted]

[+] jbert|17 years ago|reply
If large organisations are less efficient:

- why do they survive?

- if they survive despite all, why do we permit them?

If the market was regulated to set a cap on the size of a single company's workforce (or some other metric, perhaps market capitalisation), we'd also avoid the 'too big to fail' problem we have at the moment.

Not to mention quasi-monopolistic practices like large chains killing off small local competitor with an externally-funded price war (then raising prices once the competition disappears).

Is it that we need the largest companies for some large-scale activities (chip fabs, aircraft manafacture), or that they can be efficient at scale (fast food, wal-mart)?

Would the world/market be a better place if companies were forced to fission at a certain size?

[+] davidw|17 years ago|reply
> why do they survive?

http://en.wikipedia.org/wiki/The_Nature_of_the_Firm

I would really like to see more analysis of what structural differences point towards the lots of smaller firms that pg sees, and whether this is a short blip in time, or a long term trend.

Also, I hate to dredge up that ugly topic "politics", but currently, in the US, if you or your firm fail, that's great in terms of opening up space for others to try, but a little scarier in personal terms, with regards to things like health care. Long term, who knows how things like that will play out.

[+] llimllib|17 years ago|reply
> If the market was regulated to set a cap on the size of a single company's workforce (or some other metric, perhaps market capitalisation), we'd also avoid the 'too big to fail' problem we have at the moment.

I don't think either of the metrics you mention would solve the "too big to fail" phenomenon. The fact is that "too big to fail" is a misnomer, which has actually been used to represent the concept "too connected to fail".

I.e. it is not Bear's/AIG's pure size that prevents us from allowing them to fail, it's the connectedness that they have in our financial system, which depends on a web of millions of transactions per day between every large bank. Then, if a big bank were allowed to fail (not that I'm endorsing this theory), that would cause a cascade effect where banks started charging higher rates for those millions of transactions, which would cause less of them to happen, which would cause even higher rates, etc. etc., ending in a completely broken financial system where each bank wanted too much money for each of the many transactions they depend on to exist in their current form.

So, I don't think that limiting the size of companies in the financial sector would prevent the "too big to fail" phenomenon by itself. Which doesn't mean it's not a good idea; just that I think that particular example doesn't suport the idea that well.

(edit: and read Coase, like davidw's comment suggests :)

[+] retyred|17 years ago|reply
this entire "lots of small firms are the future" meme is probably just wrong. paul graham knows a lot about websites but very little about the rest of the economy. show me the ten person company that can produce the equipment to generate and transmit electricity. or food production. where are all the ma and pa grocery stores? and banking...we have fewer banks now than ever and the number is still dropping. and PCs...making a PC is easy....so what happened to the 10,000 strip-mall PC makers?

show me real examples outside of the web industry that this theory is even remotely correct

[+] netcan|17 years ago|reply
Paul mentions small companies interacting directly with markets as the catalyst to shifting from credentials to performance.

Another potential catalyst is the growth of freelance/contract labour. Essentially, what is being created is a less secure labour environment where there is more incentive & ability to directly measure performance. The assumption is that you are fired at the end of the job. If you can get further work or a referral, it's a bonus.

There is something obscuring the effects of this though. The growth of freelance/contacting partly coincides with & partly is driven by the internationalisation of labour markets. That will obscure some aspects of it. A copywriter in a cushy job in a high wage country won't leave to pursue a $25/hr career as a elence copywriter. They stay away from competition with international labour markets.

I don't think that will go away any time soon. What would be interesting to study would be these kinds of markets in low wage countries where hiring freelancers on international labour markets is not a cost advantage to hiring local employees

In any case, if we are moving to a higher performance/credentials ratio, I think this is a big part of the equation, bigger then startups.

[+] lionheart|17 years ago|reply
"In our country, college entrance exams determine 70 to 80 percent of a person's future."

I'm sure that I can speak for many of us here when I say, thank God that's not the case here anymore.

[+] nostrademons|17 years ago|reply
Could also be because of increasing specialization. When my parents' generation was going to work, the sort of things you'd actually do might include serving beverages on a plane, assembling cars, or shuffling papers around. There's not much room for differentiation in these jobs: you either do them well or you don't do them, without many gradations in between. So the only way to "prove" that you had qualified, top-notch employees was to use their degree as a substitute for their competency.

A similar effect occurs in the financial and management consulting industries today, two of the most degree-conscious industries. It's nearly impossible to measure performance in these industries, because the results of an individual's work don't become apparent until 5-10 years later, and there are a whole host of confounding factors. Did the money manager get rich because he was good, or did he get rich because he was lucky?

[+] gqwu|17 years ago|reply
PG, when you were looking at YC apps, how many +points did you give to candidates who worked at Google?
[+] pg|17 years ago|reply
It counts for something with us-- probably more than school, for example-- because it seems like Google is a pretty good filter. But what we care most about is what people have built.
[+] rsheridan6|17 years ago|reply
"History suggests that, all other things being equal, a society prospers in proportion to its ability to prevent parents from influencing their children's success directly."

Can we assume that pg is in favor of restoring the estate tax?

[+] anamax|17 years ago|reply
> Can we assume that pg is in favor of restoring the estate tax?

Can we see some evidence that the estate tax in the US ever had the characteristics cited by its proponents?

Consider, for example, the possibility that Caroline Kennedy will be appointed to the Senate by the NY Governor who just happens to come from a line of NY politicians. This isn't an isolated case - a huge fraction of current US senators are related to past senators and governors. (Biden's successor, one of his advisors, is seat-warming until his son is ready to run.) The House of Representatives isn't much different. The same thing happens at the state level - IL is mainstream in this.

The estate tax also doesn't hit the rich. The wealth of Gates et al will not be subject to the estate tax. It will, however, work to the benefit of their kids. (The Fords, Rockefellers, and GM heirs worked the same game.)

The only folks who pay estate tax are medium size family-owned biz. Many of them have to sell to pay the tax bill.

Interestingly enough, Buffet makes a lot of money buying such biz. He also makes a lot of money selling insurance to their founders to pay estate taxes.

[+] pg|17 years ago|reply
Actually I don't think inherited money is that dangerous. Unless it's coming from more than a generation back, you don't even get it till you're old yourself. And when people inherit money young, if anything it seems to screw them up rather than giving them an advantage.
[+] mattchew|17 years ago|reply
We've always had entrepreneurs in the U.S., but clever little firms haven't taken over the economy yet.

Big clumsy companies still have the advantage in a lot of areas. I'd like to see those areas eroded, but I suspect some of them are growing.

For example, my impression is that fewer doctors are in private practice than there used to be, with more of them belonging to clinics. There are certainly fewer independent pharmacies than there used to be.

[+] geebee|17 years ago|reply
I agree with much in the essay, but it's interesting that PG used law as an example of an organization that broke from the deferred compensation principle.

Most "elite" law firms actually do pay by seniority. An associate is generally paid according to the number of years he or she has been with the firm. And while the salaries for young lawyers is very high, they are actually (often) working for the potential of becoming partner - a reward for work done earlier in life. This is the essence of deferred compensation.

Law firms also tend to be more impressed with credentials than almost any other segment of the economy. The pecking order of law schools would surprise many people from the engineering world. Of course, some schools do have stronger reputations (for good reason), but in law, Harvard > Columbia > Cornell. Students accepted to a school higher in the chain will prefer that school by a wide margin. Whereas engineering students, especially at the grad level don't tend to have an MIT > Stanford > Berkeley kind of mentality... they'd be more interested in how their research interests match up with faculty in that area.

I think Law, Medicine, and other professions will be the holdouts against the trend away from school-based credentials. It's probably no surprise - these are the few sectors of the economy that can actually ban someone (with the full weight of government behind them) from practicing based on degrees and exams.

The question is, when will the tide overtake them (if ever)? Will the opportunities in a free market eventually become so appealing that only second-rate people will seek out the credentialed industries?

I'm not joking here. Law school is three years, but we all know that people learn at different rates. I suspect that some people could learn the material in a few months, whereas others would need six years, and some would never be able to learn it at all. The requirement of a 3 year degree most definitely sends some of the brightest folks into other fields where they can leverage their exceptional ability to learn quickly (software seems to be a favorite field for these people).

[+] evgen|17 years ago|reply
> I think Law, Medicine, and other professions will be the holdouts against the trend away from school-based credentials. It's probably no surprise - these are the few sectors of the economy that can actually ban someone (with the full weight of government behind them) from practicing based on degrees and exams.

These are also fields where the actions of a single bad/incompetent actor can have very, very serious consequences for their customers. Death and long-term incarceration are pretty heavy negative externalities and most of us do not want to wait for "reputation" to sort out the bad apples. These are also fields with large bodies of knowledge that an applicant in the field needs to master, with relatively easy mechanisms for testing this knowledge.

There is a reason your doctor has copies of his diploma's on the wall, and a reason why your eye searches them out when you are discussing your problem. Sometimes knowing that the other party has passed through several well-regarded and reputable gatekeepers is a good thing. Will the "wisdom of the crowd" ever replace the bar exam, medical boards, and graduation from an elite university? No.