[p4wnc6]

Maybe I'm missing something, but why isn't the implication going the other direction?

In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for?

Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.

[GoToRO]

:)) Just the other day a manager said "HR is overwhelmed. They have a student that filters the CVs and she is doing all she can." The senior programmer exploded "How is a student any good at filtering programmers, she does not know anything about programming!".

[hnal943]

Based on your comments in this thread, it sounds like you may need a new job.

[rifung]

If you consider how well someone does as how likely they are to be hired, then it makes sense that interviewers are the source of "truth", because it's their grade or opinion which will decide whether you get hired or not.

We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't.

I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be.

After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?

[p4wnc6]

Using the measurement, "how likely are they to be hired" seems like it's precisely the problem. "How likely someone is to be hired" is a totally erratic and unpredictable property of a bureaucratic hiring process. It's exactly not the sort of thing you want to use to define an absolute notion of skill or performance in a job interview.

[in_cahoots]

This is my concern too. Any person who's gone in for many interviews has seen that the interviewer isn't the absolute measure of performance: we've all seen problems that are just trivia questions, are designed to show off the skills of the interviewer, or (worst of all) where a second solution isn't accepted because it's not what the interviewer thought of. All of these issues can result in disagreements on performance, in both directions.

I would like to see a different analysis: for multiple interviewers of the same candidate, how similar are the ratings?