> Out of any group of 1000 applicants [...] Fewer than 3 will pass the on-site interview and receive offers.
> Our ultimate goal is to have the best people here, doing their best work.
I feel like everyone is so afraid of hiring a 0x or -1x programmer that they don’t want to take a chance on a mere 1x programmer.
If I ran a company (I don’t!) on the scale of hundreds or thousands of developers I’d be very choosy about who I let into senior/leadership type roles, but be rather accepting of individual contributors, the hope being that many of them would grow and develop their skills.
Am I just vastly underestimating the overal strength of a random sample of 1000 candidates? Are there really only 3 worthwhile developers?
Basically, there's a ton of completely unqualified (or even under-qualified) applicants who apply to just about every job opening out there. If you've ever been on the other end of the table, posting a job opening and sifting through applicants, the situation becomes clear very quickly.
I'm actually surprised 250 make it past their initial screen. The rest of the numbers seem reasonable to me though.
No, I don't think you're underestimating their strength - I think it's more that you're vastly underestimating how expensive it can be to make a bad hire and how difficult it can be to fire someone in California, even at the individual contributor level.
A few companies do manage to do it, but as far as I can tell you have to set up your corporate culture, HR department etc. from the outset to be prepared and ready for firing people frequently, and that's not something many startups (other than maybe Netflix?) were/are really well prepared for...
It's more likely that there are a lot of false negatives in CircleCI's hiring funnel. The issue isn't "there aren't enough good developers." There are plenty of good developers. It's possible that this sample of 1000 developers came from really bad sources (there's not enough information provided to determine the quality of their sourcing). For example, referrals convert at a rate of 57% to the screen (compared to 13% when sourcing). Recruiting agencies convert at 59% (although you'll be giving away 10-30% of first year salary for that). It could be that this sample is purely inbound applicants, which convert the lowest at 13%. [1]
We have benchmarks to understand how well a company is converting at each step in their funnel. CircleCI is converting well in 2 of their funnels, but the other 2 are underperforming significantly per my other comment in this thread.
Undoubtedly there are many more than that. But they have the luxury of having 1000 people apply when they're only looking to hire 3. There might be 100 good devs, but that doesn't mean they want to or are able to hire that many.
Past a certain level of skill, it really does come down to specific experience, priorities, communication styles, etc. There are many worthwhile devs that simply have a less ideal package for that specific role, at that specific company, at that specific time.
Keep in mind that only 1/4 of those thousand passed an initial screen, which isn't even an interview step, but rather glossing over their resume to see if it fits the position. From there, only 117 passed a quick phone screen.
It still seems high, but I can imagine these average numbers being inflated by some important 'Lead Engineer' type of position, for which the extreme vetting is reasonable.
I think the majority of the candidates are capable, and a lot of them are probably quite excellent. But when you have that many candidates for so few positions, you have the luxury of really trying to go for the best people.
I interviewed at CircleCI many months back. Overall I thought the process was done very well until I got to the stage before the onsite and I was simply sent an email that said that while everybody like me and thought highly of me they were not going to move forward at that time. I followed up asking for more specific information and never heard back.
This process is never perfect, but I think when you don't hire someone it is always nice to provide something solid the interviewee can go off of. Even if the response is simply that we felt we had stronger candidates for whatever reason.
I saw a horrific case the other week where an internal recruiter cut and paste the feedback from an engineer verbatim. The candidate received feedback which said and I quote “You’d have to be a madman to have used X in 2017”.
How would you feel about there being some sort of standardized credential certification, similar to a BAR or trade skill licensing, that proved a candidates technical chops ahead of time? Do you think this would help reduce the number of candidates that gets turned down during the initial screen?
It seems to me like if there was some standard of proving technical skills, both parties could save a ton of time and go back to regular culture-fit style interviews.
Of course, for the highest paying/most demanding jobs interviews would have to be tailored - but I don't really see why junior/mid level roles need interviews that last 4-8 hours on site plus any pre-on-site screening that needs to happen.
The PE exam is traditionally taken by people in the physical engineering disciplines; mechanical, structural, civil, and the like. By passing this exam you earn the right to use the 'Professional Engineer' title. Many (all?) permitting jurisdictions require non-trivial construction projects to be reviewed and approved by licensed PE's - this is not a joke certification.
In the physical engineering world (structural, mechanical, civil) the PE exam is seen as a mark of a competent engineer, and usually comes along with a significant pay increase, even if the engineer is not using their license to review and approve projects.
Seems to me like the PE exam would be an appropriate exam filter for software hiring.
I think the Computer Science degree was supposed to basically be that, in theory. It obviously isn't in practice.
I've overheard some other fields and the rigueur they go through is laughable in comparison. Heard a Marketing guy brought in for an interview in the next room go "I'm the right guy for the job!" basically, his only credentials being one previous job in the field and a Marketing degree, and my boss comes back and goes "Hmm, he seems very confident! Maybe we should hire him!"
Meanwhile in our field, having 15+ years of direct experience means basically nothing other than they will set up a phone screen. The inevitable technical interview seems to be designed to rob us of our confidence as soon as possible, putting us on the defensive when we dare to get one question wrong because we couldn't remember or study for 100% of everything that exists throughout all of computer science and technology and our domain specialty, etc.
Even my S.O., who's in proposal writing, only has to make sure she has her existing portfolio of past work updated and printed, but otherwise does very little to prepare for interviews. She doesn't try to study every possible question in that could possibly be asked about technical writing, marketing, graphic design, RFPs, sales, and whatever else her field touches. She generally only gets background and personality questions, that's about it. Yet she's directly responsible for her companies winning millions of dollars in business in her job, per proposal.
The default of distrust in programmer abilities amongst companies is so bad I've been very tempted to leave the field at times, because I'm sick to death of going through the endless hoops and basically having to refresh my entire degree when I want to start looking for a new job (which I'm overdue to start doing again and I've been dragging my feet partly because I don't have the time right now to refresh the skills that haven't been needed at my job the past few years).
I feel like this is a disingenuous question. Are you asking if they should be? Do you believe that CircleCI has some nefarious plot to increase engineering team productivity by mining spare workcycles out of interview candidates?
If I were betting on this I would wager that something like 95% of the time, the existing employee would be able to solve the project or problem much faster without the candidate's input at all, but the point is to try and approximate a more realistic work experience and environment.
I think that's a reasonable goal and I don't think CircleCI should be vilified for it.
I find this attitude frustrating. It seems like a constant on sites like reddit and HN is a strong disdain for the algorithmic question style technical interview. Some companies take note and switch up the interview process and ask real world questions (like CircleCI using a current bug or problem) rather than how to invert a binary tree and people get upset about not being compensated.
How should companies interview people in a manner that assesses their technical skills but doesn't "exploit" them? Personally I think the algorithm based interviews are great but also see value in these modified approaches of a short take-home assignment.
If you are interviewing X number of people for 1 job, you also have to expect a candidate is applying at ~X number of jobs. Asking them to work on your codebase without compensation, or do any take home work without compensation, is exploitative.
This question is a red herring. I hate this question because the only acceptable answer is to ask candidates about their experience and make a gut feel on that. Which only works if your gut feel is properly calibrated. Which often looks like unconscious bias.
Anything else is "too much work" on the candidate.
A take-home? "I don't have time to work on unpaid stuff!"
A pairing session? "I don't have time to work on unpaid stuff!"
Software engineering trivia? "Why are they asking me to build a linked list when I'd just use stdlib in real life?"
Asking about stuff on the resume? "They've never worked on ${thing}; how could they judge me on it?"
Did you read the full article? Or are you cherry-picking to tailor your criticism?
> Studies have shown that that women are less likely to respond to an ad that has overly masculine or aggressive language (despite being qualified), whereas men will apply regardless.
Having a job description more gender friendly encourages more women to apply and men will apply regardless.
I do worry that this is deeply circular logic. Words like "challenging" are considered strongly masculine by the Gender Decoder they linked to. I don't see how calling that word masculine is doing anything other than entrenching gender stereotypes even further.
> 2. Micro-skills take-home problem. The candidate completes a coding problem and reviews it with the interviewer. Talking through the problem with the candidate helps us assess ability as well as the candidate’s thought process, and how their priorities and values in writing software line up with ours.
This is generally a lousy experience. I've been places that want me to build a 'simple' (single entity) fullstack CRUD app (angular w/ASP.NET core backend.)
So what is it that they want prioritized? UI skills? Backend dev skills? I can do both, but I'm not going to spend a week making it enterprise-production-ready. The hiring engineers all have things they are looking for and these things are never well defined for you up-front, so no matter what you do (unless you spend way too much time on it,) someone may use it to disqualify you (without you ever knowing it was important.)
The fact that they use a pseudo scientific "Gender Decoder" is a red flag. It leads me to think Circle CI has unfair prejudices against men and masculinity in general.
By the way, "Objectives" and "driven" are neither feminine or masculine, they're just words. Can you imagine if every word in the Oxford English Dictionary had a gender classification?
If it's true that using "female-coded language" results in more applications from women, without reducing the number of men who apply, what's the harm?
If it's not true, then I would agree that this is probably a waste of time.
There's at least one peer-reviewed study (Gaucher et al, 2011) which supports the idea that these words make jobs sound less appealing to women. Do you know of a study which challenges these findings?
Wouldn't be surprised if it is a newer policy, that they didn't have before. I've worked in departments that had programming tests they give applicants that their early employees never took, and freely admit they probably wouldn't have been able to complete if it was in place when they applied.
Policies tend to just get stricter and stricter over time, especially with every bad experience they have (only takes one or two usually before these people go off the deep end trying to correct for it).
>Candidates pair with members of the engineering team, and work together on problems/projects in the actual CircleCI codebase.
This seems like it'll bias to those who can quickly ramp up on a new code base in an unfamiliar environment. Which, in my mind, is a distinct skill from being able to contribute quality code to a code base you already know. I wonder if this is known and acknowledged bias or not at CircleCI.
While this is true, one could come up with similar arguments for any task put to the interviewee. Not to demean your point, but the interview process _is_ one of elimination after all, and biasing the task to favour certain traits and skills is but one of the ways you can go about it.
There's a basic problem with posts like this. Finding the right people to associate with is a hard problem.
Finding friends is hard. Finding a significant other is (ideally, anyway) a once-in-a-lifetime challenge. Finding people to hire is even harder: you're still trying to establish chemistry, only now there's a lot of money involved and the actors are big groups of people with complicated internal dynamics.
So I think what's wrong with this post is that CircleCI seem so self-satisfied. They seem to think they have a handle on this problem, when anyone who's been through a job interview in the last year or so will tell you it's a shitty, capricious, random experience even when they take you.
CircleCI’s hiring process had 1000 people enter their funnel as new candidates and 3 emerge as employees. Others have tabulated the rate of candidate->screen, screen->onsite, and onsite->offer. If we make a conservative estimate of 1 hour of company time per considered resume (averaged across all steps, higher for those who get far into the process, lower for those pruned earlier), they burned at least half a head of employee time to hire 3 heads. That’s way too high an interview cost per hired head.
Only 3/1000 will receive offers. Does this strike anyone else as an extremely flawed and stringent filter? They claim to only want to hire the best, but in my opinion, they will end up hiring the lucky, with similar opinions and skillsets to the interviewers.
Reminds me of the joke about throwing away half the candidates arbitrarily because you wouldn't want to hire someone that has bad luck.
It does seem pretty strict, but without some sort of baseline this might be rather normal. I've never worked in recruiting or seen hard numbers on this sort of thing before so I have nothing to compare it to.
The typical stages of a hiring funnel look like this: new candidate to screen, screen to onsite, onsite to offer, offer to acceptance. The respective ratios for each stage are: 17%, 32%, 31%, and 69%. On average, you'll look at about 88 candidates to make 1 hire. [1]
For CircleCI, their new candidate to screen is 25%. This is higher than the 17% average. Their screen stage looks considerably larger than I've seen at most companies: it's 4 different steps. From the 250 candidates, it looks like 7 go into an onsite. That's about 3% which is significantly below the 32% benchmark. Of those 7, 3 will receive offers (it's not mentioned how many accept those offers). So the onsite to offer is 42%, which is higher than the 31% average. We can't determine the offer acceptance from the data.
I think it's great they're looking at the funnel and conversion rates. Most recruiters aren't able to tell you these figures which is a red flag. The only way to improve those metrics is to know them first and then measure how changes impact each stage.
[+] [-] hamandcheese|8 years ago|reply
> Our ultimate goal is to have the best people here, doing their best work.
I feel like everyone is so afraid of hiring a 0x or -1x programmer that they don’t want to take a chance on a mere 1x programmer.
If I ran a company (I don’t!) on the scale of hundreds or thousands of developers I’d be very choosy about who I let into senior/leadership type roles, but be rather accepting of individual contributors, the hope being that many of them would grow and develop their skills.
Am I just vastly underestimating the overal strength of a random sample of 1000 candidates? Are there really only 3 worthwhile developers?
[+] [-] jlaurend|8 years ago|reply
Basically, there's a ton of completely unqualified (or even under-qualified) applicants who apply to just about every job opening out there. If you've ever been on the other end of the table, posting a job opening and sifting through applicants, the situation becomes clear very quickly.
I'm actually surprised 250 make it past their initial screen. The rest of the numbers seem reasonable to me though.
[+] [-] rrdharan|8 years ago|reply
A few companies do manage to do it, but as far as I can tell you have to set up your corporate culture, HR department etc. from the outset to be prepared and ready for firing people frequently, and that's not something many startups (other than maybe Netflix?) were/are really well prepared for...
[+] [-] jmtame|8 years ago|reply
We have benchmarks to understand how well a company is converting at each step in their funnel. CircleCI is converting well in 2 of their funnels, but the other 2 are underperforming significantly per my other comment in this thread.
[1] http://qr.ae/TU1sB2
[+] [-] jamestimmins|8 years ago|reply
Past a certain level of skill, it really does come down to specific experience, priorities, communication styles, etc. There are many worthwhile devs that simply have a less ideal package for that specific role, at that specific company, at that specific time.
[+] [-] alehul|8 years ago|reply
It still seems high, but I can imagine these average numbers being inflated by some important 'Lead Engineer' type of position, for which the extreme vetting is reasonable.
[+] [-] ivanfon|8 years ago|reply
[+] [-] akmiller|8 years ago|reply
This process is never perfect, but I think when you don't hire someone it is always nice to provide something solid the interviewee can go off of. Even if the response is simply that we felt we had stronger candidates for whatever reason.
[+] [-] rrdharan|8 years ago|reply
https://www.quora.com/Why-dont-companies-give-interview-feed...
[+] [-] ilovecars2|8 years ago|reply
[+] [-] protonimitate|8 years ago|reply
It seems to me like if there was some standard of proving technical skills, both parties could save a ton of time and go back to regular culture-fit style interviews.
Of course, for the highest paying/most demanding jobs interviews would have to be tailored - but I don't really see why junior/mid level roles need interviews that last 4-8 hours on site plus any pre-on-site screening that needs to happen.
[+] [-] sgslo|8 years ago|reply
The PE exam is traditionally taken by people in the physical engineering disciplines; mechanical, structural, civil, and the like. By passing this exam you earn the right to use the 'Professional Engineer' title. Many (all?) permitting jurisdictions require non-trivial construction projects to be reviewed and approved by licensed PE's - this is not a joke certification.
In the physical engineering world (structural, mechanical, civil) the PE exam is seen as a mark of a competent engineer, and usually comes along with a significant pay increase, even if the engineer is not using their license to review and approve projects.
Seems to me like the PE exam would be an appropriate exam filter for software hiring.
[+] [-] cableshaft|8 years ago|reply
I've overheard some other fields and the rigueur they go through is laughable in comparison. Heard a Marketing guy brought in for an interview in the next room go "I'm the right guy for the job!" basically, his only credentials being one previous job in the field and a Marketing degree, and my boss comes back and goes "Hmm, he seems very confident! Maybe we should hire him!"
Meanwhile in our field, having 15+ years of direct experience means basically nothing other than they will set up a phone screen. The inevitable technical interview seems to be designed to rob us of our confidence as soon as possible, putting us on the defensive when we dare to get one question wrong because we couldn't remember or study for 100% of everything that exists throughout all of computer science and technology and our domain specialty, etc.
Even my S.O., who's in proposal writing, only has to make sure she has her existing portfolio of past work updated and printed, but otherwise does very little to prepare for interviews. She doesn't try to study every possible question in that could possibly be asked about technical writing, marketing, graphic design, RFPs, sales, and whatever else her field touches. She generally only gets background and personality questions, that's about it. Yet she's directly responsible for her companies winning millions of dollars in business in her job, per proposal.
The default of distrust in programmer abilities amongst companies is so bad I've been very tempted to leave the field at times, because I'm sick to death of going through the endless hoops and basically having to refresh my entire degree when I want to start looking for a new job (which I'm overdue to start doing again and I've been dragging my feet partly because I don't have the time right now to refresh the skills that haven't been needed at my job the past few years).
[+] [-] anonfunction|8 years ago|reply
Are they compensated for this work?
[+] [-] rrdharan|8 years ago|reply
If I were betting on this I would wager that something like 95% of the time, the existing employee would be able to solve the project or problem much faster without the candidate's input at all, but the point is to try and approximate a more realistic work experience and environment.
I think that's a reasonable goal and I don't think CircleCI should be vilified for it.
[+] [-] smiths1999|8 years ago|reply
How should companies interview people in a manner that assesses their technical skills but doesn't "exploit" them? Personally I think the algorithm based interviews are great but also see value in these modified approaches of a short take-home assignment.
[+] [-] lghh|8 years ago|reply
[+] [-] nunez|8 years ago|reply
Anything else is "too much work" on the candidate.
A take-home? "I don't have time to work on unpaid stuff!"
A pairing session? "I don't have time to work on unpaid stuff!"
Software engineering trivia? "Why are they asking me to build a linked list when I'd just use stdlib in real life?"
Asking about stuff on the resume? "They've never worked on ${thing}; how could they judge me on it?"
[+] [-] fein|8 years ago|reply
Yet you have to tailor your ad language so more women apply because they get scared by words like "objectives"?
I don't even know where to start here.
[+] [-] ictoan|8 years ago|reply
> Studies have shown that that women are less likely to respond to an ad that has overly masculine or aggressive language (despite being qualified), whereas men will apply regardless.
Having a job description more gender friendly encourages more women to apply and men will apply regardless.
You should read the article liked to this article - https://hbr.org/2014/08/why-women-dont-apply-for-jobs-unless...
Women apply to jobs only if they are 100% qualified whereas men doesn't get deterred by lack of qualification.
[+] [-] nilkn|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] lghh|8 years ago|reply
[+] [-] GiorgioG|8 years ago|reply
This is generally a lousy experience. I've been places that want me to build a 'simple' (single entity) fullstack CRUD app (angular w/ASP.NET core backend.)
So what is it that they want prioritized? UI skills? Backend dev skills? I can do both, but I'm not going to spend a week making it enterprise-production-ready. The hiring engineers all have things they are looking for and these things are never well defined for you up-front, so no matter what you do (unless you spend way too much time on it,) someone may use it to disqualify you (without you ever knowing it was important.)
[+] [-] paxys|8 years ago|reply
[+] [-] mychael|8 years ago|reply
By the way, "Objectives" and "driven" are neither feminine or masculine, they're just words. Can you imagine if every word in the Oxford English Dictionary had a gender classification?
[+] [-] titanomachy|8 years ago|reply
If it's not true, then I would agree that this is probably a waste of time.
There's at least one peer-reviewed study (Gaucher et al, 2011) which supports the idea that these words make jobs sound less appealing to women. Do you know of a study which challenges these findings?
[+] [-] morogop|8 years ago|reply
[deleted]
[+] [-] ionforce|8 years ago|reply
How did you come up upon this conclusion?
[+] [-] Willson50|8 years ago|reply
[+] [-] daenz|8 years ago|reply
[+] [-] cableshaft|8 years ago|reply
Policies tend to just get stricter and stricter over time, especially with every bad experience they have (only takes one or two usually before these people go off the deep end trying to correct for it).
[+] [-] cdpolyme|8 years ago|reply
[+] [-] marcinzm|8 years ago|reply
This seems like it'll bias to those who can quickly ramp up on a new code base in an unfamiliar environment. Which, in my mind, is a distinct skill from being able to contribute quality code to a code base you already know. I wonder if this is known and acknowledged bias or not at CircleCI.
[+] [-] bhasi|8 years ago|reply
[+] [-] paxy|8 years ago|reply
[+] [-] mnemonicsloth|8 years ago|reply
Finding friends is hard. Finding a significant other is (ideally, anyway) a once-in-a-lifetime challenge. Finding people to hire is even harder: you're still trying to establish chemistry, only now there's a lot of money involved and the actors are big groups of people with complicated internal dynamics.
So I think what's wrong with this post is that CircleCI seem so self-satisfied. They seem to think they have a handle on this problem, when anyone who's been through a job interview in the last year or so will tell you it's a shitty, capricious, random experience even when they take you.
[+] [-] yodon|8 years ago|reply
[+] [-] daenz|8 years ago|reply
[+] [-] phinnaeus|8 years ago|reply
It does seem pretty strict, but without some sort of baseline this might be rather normal. I've never worked in recruiting or seen hard numbers on this sort of thing before so I have nothing to compare it to.
[+] [-] MarkSweep|8 years ago|reply
[+] [-] jmtame|8 years ago|reply
For CircleCI, their new candidate to screen is 25%. This is higher than the 17% average. Their screen stage looks considerably larger than I've seen at most companies: it's 4 different steps. From the 250 candidates, it looks like 7 go into an onsite. That's about 3% which is significantly below the 32% benchmark. Of those 7, 3 will receive offers (it's not mentioned how many accept those offers). So the onsite to offer is 42%, which is higher than the 31% average. We can't determine the offer acceptance from the data.
I think it's great they're looking at the funnel and conversion rates. Most recruiters aren't able to tell you these figures which is a red flag. The only way to improve those metrics is to know them first and then measure how changes impact each stage.
[1] https://medium.com/@jmtame/surprising-insights-from-talking-...
[+] [-] nartz|8 years ago|reply
[+] [-] titanomachy|8 years ago|reply
[+] [-] AboutTheWhisles|8 years ago|reply
[+] [-] nartz|8 years ago|reply