[heliostatic]

I'll make the obvious point that these interviews were not good predictors of success at the companies.

In general, I've found in 15 years of interviews that interviews are not highly correlated with the work at any given company. I wish we would move beyond cargo cult interview styles and base interviews (and screening, for that matter) on the actual work and culture of each company.

[eckesicle]

I ran a largish statistical analysis where I was working on all interviews and employee ratings for the following year (16000 data points).

My conclusion was that:

* some interview problems carried zero signal

* some interviewers are really good at identifying strong candidates, and others correlate negatively! On average interviewers rating is a weak signal (<0.25. Correlation)

* geographic location is a medium signal (0.2)

* a relevant degree is a strong signal (0.4)

* choice of programming language is the strongest signal (>0.8 correlation (or anti-) for some languages, and some.

There is some selection bias though since candidates who were rejected did not get a chance at the job so we didn’t have a rating for them.

[hermannj314]

The choice of language being a signal is interesting. That goes against my intuition.

Can you elaborate?

[zephrx1111]

Didn’t know the language can be a factor. I was always told any language is fine.

[breckenedge]

Do you have a link to this research? This is amazing.

[prennert]

There are a few outliers, but generally, I found most candidates hired are good, if the interview process mimics the actual projects someone is going to work on in the next 3 months or so. But the downside is that interviewing is slow and cannot easily distributed across multiple teams, because the tasks are tailor made.

Due to the downsides, I guess it is easier for companies to rely on standardised tests. Even just for the reason that those are easy for anyone in the company to lead the interview. But likewise it is easy for anyone to game the system.

[bagacrap]

Or they were good predictors of success, just not in the way you expected. The individuals got the job done one way or another, just like they nailed the entry level job position.

How would you base a job interview on actual work? You have very limited time and you're trying to take a representative sample of a candidate's capabilities.

IMO interviewing is just plain hard and unpleasant. Obviously if you perfectly solve interviewing for software engineers you could rule the world, because you could afford to offer all of your hires $1M/yr which is far less than the best engineers are worth, but more than what everyone else offers.

[osigurdson]

It is funny. Let’s say you have an MIT PHD computer science grad with a 4.0 GPA. The 8 years of verifiable performance means nothing. The competitive coder would blow them away in an interview.

[bee_rider]

A PhD is intended to set you up for an R&D type job. It depends on what kind of education the competitive coder has of course, but if they are interviewing for the same types of positions, the scientists isn’t exploiting their competitive advantage.

[hikawaii]

Yes, because the entire reason competition code was adopted in the first place is people graduating with prestigious titles that could not write code.