top | item 26401081

The technical interview practice gap

146 points| leeny | 5 years ago |blog.interviewing.io | reply

219 comments

order
[+] bidirectional|5 years ago|reply
Here's a serious question, does any job paying over 250k have an easier hiring process than FAANG? The thought of going through an investment banking, white-shoe law firm or medical residency pipeline seems utterly terrifying compared to 'show up in jeans and take this algorithms test which has near-infinite practice material available'.
[+] SavageBeast|5 years ago|reply
When there is a cottage industry focused on training people to pass your interview then you have a problem.

I think Allen Iverson said it well:

"We're talking about practice, man. [laughter from the media crowd] We're talking about practice. We're talking about practice. We ain't talking about the game. [more laughter] We're talking about practice, man."

Fun fact:

I had a recruiter reach out about interviewing at a non-FAANG company that is private. Being private there are no RSUs on offer but instead some kind of Company Funny Bucks doled out based on "performance" etc. Whatever, all well and good so far. Competitive salary but TC wont be anywhere in the same solar system as some of the FAANG total comps Im aware of for a position of similar level.

Then the kicker comes - I get an email about "interview preparation" and I kid you not it was every bit as intensive as AMZN. Do they really expect me to put up with this hazing for the same money I can go make without it? For Company Funny Bucks (told to expect 10-15% of my salary at best with a whole lot of qualifying "if's" in there indicating this was mostly fiction). Needless to say I performed a quick analysis of the value of my time and opted to politely pass on the interview.

[+] PragmaticPulp|5 years ago|reply
> When there is a cottage industry focused on training people to pass your interview then you have a problem.

Doctors have board exams. Lawyers have bar exams. Financial professionals have their own set of exams. Electrical and mechanical engineers have professional engineer exams.

The difference is that they're standardized, so the prep industry caters to passing those exams more so than passing the interviews.

Software engineering is far from standardized because the subject matter is far too varied, so instead companies handle their own type of exams customized to their needs.

There is also a cottage industry around interview prep for lawyers, doctors, and other professions. It's just not well known because it's extremely expensive relative to spending a couple hundred bucks (if that) on Leetcode. It often takes the form of expensive 1:1 coaching that starts in the hundreds of dollars and can easily enter five figures. Students from wealthy families frequently have private coaches for this.

[+] 120bits|5 years ago|reply
I haven't cleared a single technical interview after applying to FAANG companies in last 5 years. I'm currently employed and I have been working for than 8 years. I'm good at what I do. I might not the greatest programmer but I can come up with a approach to solve a problem.

My past experiences have been terrible, not a single person who interviewed me wanted to know what I worked on or what I have achieved or how I solved a given problem. Few introductory questions and jump into coding questions. Some of them are medium and few of them were hard. But at most I was able to find a way to solve(not completely code a solution).

It seems lot easier when you are in college and are surround by students who are practicing coding interviews. You can talk, discuss and gain more experience. I don't know how I can do that, when I'm busy maintaining and coding for my current job. Its just so demotivating.

[+] arebop|5 years ago|reply
Practice definitely helps.

There's a lot luck involved because there are so many candidates, most of whom cannot come up with an approach to solve a problem (let alone to solve it well enough to serve billions of users well). So even most candidates who are qualified can't convincingly demonstrate that given the time constraints and pressure etc.

It is hard to get into FANG for college new grads, it is even harder if you're trying to break into the tech industry from outside without the schedule flexibility and focused community support of a top tier university. It is possible but you need a combination of high ability, strong drive, and luck.

At my FANG, we've made some interviewing changes over the past 5 years such that these days you'd spend at least 20% of your day interviewing talking about what you worked on and achieved and how you solved problems, whereas in the past it would be entirely at your interviewer's discretion and typically only be a few minutes of unscored small talk.

FANG is an interesting life experience and it pays well, but plenty of people find it is not all it is cracked up to be. Just as in every job, a lot depends on the particular people you work with and whether you enjoy the particular problems you are solving. So maybe also consider whether you really need to go FANG (right now) or whether you could be just as happy at a place that's willing to take a more individual approach on hiring. That place could even be a stepping stone to reach FANG later on if that's what you want to do later.

[+] MaxBarraclough|5 years ago|reply
The usual HackerNews advice here is something like find a way to commit a few hours a week to LeetCode.
[+] pavas|5 years ago|reply
The interview process for FAANG companies is primarily focused on preventing false positives because of the damage that can be done by individuals when working at scale. Unless you have a really large sample size I wouldn't take any of that personally because for each interview, luck is involved.

Furthermore, the process is partly about how good you are at being able to learn, and partly about how well you are able to keep yourself motivated to grind something out, or a combination of the two. Given enough time, FAANG interviews are 100% something you can study for, and furthermore, it is expected that you spend the time to study for them as that is also a signal of something in itself.

On the job, you'll either have to be smart enough to pick stuff up quickly or spend the time to grind out the work (working overtime, etc). You also can't fuck things up too much because the fallout can be really expensive, so it helps to be able to do things right the first time. I think what is generally tested in the interview correlates with these requirements, and prevents false positives (and probably lots of false negatives too but you can just study more and try again at that point).

[+] BossingAround|5 years ago|reply
An honest question; do you need to work for FAANGs if this is the case? I think it's fairly accepted that you need X amount of hours per week to designate only to studying for your FAANG interview if you want to pass. This might be 10h a week for the next 3-6 months, or 15 for the next 10 months (or any other number of hours a week for any other number of months/weeks), as it wholly depends on you.

You can either dedicate the time and practice, or decide you don't want to do it since your situation does not enable you to do so (e.g. due to kids, family, or simply lack of motivation).

Of course, there are people who do not have to study. It seems you are not one of them. (Neither am I to be honest, this is just an observation, not an offensive remark). What else is there to do other than studying or not passing the interview in such a case?

[+] MrMan|5 years ago|reply
Start or find a company that takes a "value" approach to investing in humans. There are a lot of undervalued people out there, in North America many if not most people's potential is being wasted. SV / FAANG is a land of, in some ways, artificial scarcity. The reason jobs are hard to get is not just that the work is so demanding, it is not true. The rewards are high, and this is part of the reason that not everyone is allowed to partake.

I am oversimplifying it, but the same forces that created the FAANGs can allow you to avoid them and companies like them, avoid SV. The world has become more evenly distributed in some ways, but HN and channels like it can make you think you have to pass through these certain gates.

[+] jeffbee|5 years ago|reply
This is pretty explicit in interview training at least at Google. Nobody cares how you solved some problem because inside the Google none of your experience applies. What are needed are people who can read and write code, who know algorithms and can discuss the complexity of them.
[+] 908B64B197|5 years ago|reply
> I won’t name any specific companies here, but tech giants (many with good intentions and with marching orders to boost their diversity numbers) often do a considerable amount of outreach and pre-interview engagement with candidates from underrepresented backgrounds. This outreach is usually an info session where one engineer speaks to a virtual room of candidates from underrepresented backgrounds. The engineer tells them what to expect in technical interviews, encourages them to learn how to articulate their thought process out loud while solving a problem, and recommends resources like Cracking the Coding Interview. Some companies even go so far as to offer their underrepresented candidates a mock interview or two. [...] Unfortunately, for candidates who are unfamiliar with the process, neither of these interventions is nearly enough.

At some point how much hand-holding is enough?

They pretty much told candidates where to get sample problems, gave a session showing what the interview is like, did one mock interview... Pretty much all what prospective applicants have to do is pair up and do a few more practice rounds.

How is it the evil tech company's fault? They gave them the toolbox with all the tools in it and instruction on what to do. And the material to practice building whatever they said they will test them on.

[+] brandall10|5 years ago|reply
I interviewed with Google last summer, and the recruiter personally did all of this (aside from a mock, but did mention interviewing.io) and included a pretty lengthy email with resources. I’m a white male.

Still, the vast majority of applicants aren’t going to pass. It’s not like it’s just some minor brushing up for most - it takes real work and dedication, for some that could do the job well over 100 hours... an infinite amount of time probably won’t be enough for those who can’t.

[+] claudiulodro|5 years ago|reply
I haven't participated in too many FAANG interviews and even with the recruiter handholding it's still super confusing if you're not familiar with the scene:

What difficulty of questions will be asked? Easy? Hard? Do people actually want to "hear the thinking process" or do they just want an optimum answer regurgitated quickly? Do I need to practice "regular" interview questions also ("What's your greatest weakness?")?

All of this with a life-changing, potential millionaire amount of money at stake, which isn't great for the nerves.

[+] maerF0x0|5 years ago|reply
No specific amount of "hand holding" is enough until the diversity numbers are up, as that is now the metric.

This is not a specific judgement about diversity as a metric, but an observation of making any something a metric.

[+] aphextron|5 years ago|reply
I'll preempt the usual discussion on these posts with this:

Everything in life is a game. You can choose to learn the rules, and play to win it, or you can call foul and take what's given to you easily.

Either one is valid. But don't act entitled to the gold medal if you choose to prioritize other things over practice.

[+] bidirectional|5 years ago|reply
Yep. Quant finance has a very similar culture to FAANG companies when it comes to interviewing style. Yet practically every one in the field just realises that's how the game works, does some practice and gets on with it. No one likes going through this process, but once you realise that well-paying, prestigious jobs just require you to take a roughly standardised, undergrad-level test, it almost feels like cheating.
[+] blacktriangle|5 years ago|reply
And in fairness to the FAANGs, it seems like we actually know the rules: review your fundamental CS algorithms and grind leetcode. Compare this to any other company who is looking for whatever the fuck culture fit means and how to get a job there is anybody's guess.
[+] dominotw|5 years ago|reply
Agreed 100%. FB has the right to conduct interviews however they see fit.
[+] gkoberger|5 years ago|reply
This blog post is selling an agenda (as is what i'm about to say :) ), and that agenda is that you should pay Interviewing.io to help you get through interviews.

I think the bigger problem is that technical interviews are broken. I ask everyone to bring their own project to work on, since I hate the traditional tech interview process so much. And it goes incredibly well.

It could be adding a feature to a side project, starting something new, or contributing to open source. (Or, we have some stock ideas if people need inspiration.) Rather than asking them to solve a problem they had never heard of before or use a codebase they're not familiar with, I get so much more out of watching them work in their own environment.

I can ask them questions about why they're doing something, and they tend to have much more detailed answers because they've been thinking about it for weeks. They're solving a problem they care about in a codebase they know, which mimics how working with them will be a few months in.

We could all just buy into this notion that tech interviews suck but we have to deal with them. And then, like SAT Prep classes, there's a whole industry around how to pass them. Or, we can spend more time talking about how there's definitely a better way.

[+] josephg|5 years ago|reply
I know this is controversial around here, but I think technical interviews are a fine tool when done well. I interviewed for triplebyte for a year (interviewing about 400 candidates). The triplebyte interview has a series of different parts assessing different skills (live coding, knowledge, debugging, etc). There’s plenty of room for candidates to be weak in some sections of the test but still pass with flying colours.

We had a hypothesis internally that some parts of the interview were redundant. The interview was certainly tweaked over time, but the data science team found otherwise. All scores are positively correlated. And removing any major part of our interview would have made the interview worse. Given all my experience with it, I believe the triplebyte interview to be a fair assessment of candidates, and respectful to everyone involved. (Though I’d be very interested to know how much 5+ practice interviews would move the needle on scores, as this article claims.)

The hardest part of my job was nervous candidates. Stress is designed to help us survive running away from a lion. When we get stressed, we shut down our ability for creative thought and our memory gets weaker - which is literally the opposite of what you want in a job interview. Most of my energy as an interviewer went into helping candidates stay calm throughout the process, and I wasn’t always successful.

I don’t think there’s anything fundamentally wrong with whiteboard interviews. You’re right - we could probably rearrange the professional world to avoid all stressful assessments but I tend to agree with this article. Just do a bunch of them and you’ll get used to them. Being interviewed is just talking for an hour or two with someone smart about something you care about. Usually you get to talk to someone you would have no access to in your daily life. If you can get past your terror of being judged, they can be a lot of fun.

[+] tomatohs|5 years ago|reply
A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review.

I'm building a developer screens sharing app that, among other things, automates non-whiteboard interviews. Typically we can review a 60 minute take-home in 10 minutes, without cloning the repo. http://paircast.io

If you're an engineering manager that wants to move away from the leetcode challenges, please get in touch with me. [email protected]

Also see http://they.whiteboarded.me/

[+] swagonomixxx|5 years ago|reply
> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review.

Literally this. You can pretty much do all of the interviews required for FANG without talking to a human being, which from the FANG perspective, is absolutely GREAT, because it takes all of the existing social biases (e.g preferring a certain race, preferring a certain gender, etc.) out of the window and focuses ONLY on the algorithmic knowledge of the applicant.

Of course, there might be some interview here and there about soft questions, but for the most part, they don't really give a shit about that, and it's unlikely that you'd be rejected based on that, unless you _really_ suck at bull-shitting success/failure stories (which I am - can't stand doing it).

I think the last time I applied to Amazon (over a year ago at this point) I didn't even get phone screened. It was just a link to 2 graph algorithm problems that were the equivalent of a medium difficultly on Leetcode, and I was immediately graded. It's so fricking easy for FANG nowadays.

[+] novok|5 years ago|reply
The issue with take homes is they tend to be time unbounded for the interviewee, not the amount of time it take to review them AND there is still a whiteboard algo interview to do.

When your doing whiteboard interviews the interviewer is there the entire time interacting with you, so it's taking up 1.5hrs of engineer time at least for every 1hr interacting with the interviewee. Reviewing a take home takes less time.

[+] anon_tor_12345|5 years ago|reply
>A big reason we use leetcode style challenges is because they can be automatically scored

but they're not? every screen i've been through has been on something that doesn't even compile the code (e.g. coderpad.io for FB or docs at G) and the interviewer then writes a summary afterwards (ala hiring packet at G) with ratings along some axes.

[+] PragmaticPulp|5 years ago|reply
> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review.

Take-home problems are automatically scored as well. The company defines an API or interface to the code and creates a test suite to run against it.

The benefit of take-home problems is that you have the option of reviewing the code in-depth if you need more perspective. If a candidate only passes 7/10 tests, I'll still dive into the code to see what went wrong. If it's an honest mistake, I ignore the failure and amend the instructions to make it more clear for the next candidate.

[+] dominotw|5 years ago|reply
> Typically we can review a 60 minute take-home in 10 minutes

So a candidate a invests 60 minutes of their time and a multi million dollar company invests 10 minutes.

We should all boycott these asymmetrical "take home" tests.

[+] mLuby|5 years ago|reply
Forget for a minute who won't get through the "broken tech interviews" and instead consider the types of bad hires that could get through these filters:

- Take-home LeetCode: poor communication skills, bad at workflows, unpleasant personality, poor architecture, write-only, cheaters.

- Live LeetCode: same as take-home LeetCode but at least they have communication skills and aren't cheating so you know they can write efficient functions.

- Resume: all the issues with take-home LeetCode.

- Culture fit/behavioral: all the issues of take-home LeetCode, but at least they're good at communication and have a good personality.

- Take-home project: same as take-home Leetcode but instead of efficient functions you know they can at least architect and implement a small, standard application (if they didn't cheat).

- Show me your side project: Same as take-home project, plus they might work extremely slowly, and might only be passionate about that one side project, but you know they have communication skills.

- Architecture whiteboarding: can't actually code to implement the systems they're drawing, but you know they have communication skills.

So it's no surprise that companies wanting to reduce the risk of a costly bad hire (a bad fit, not necessarily a bad person) evaluate the candidate from different angles to check for red flags.

I sympathize with the frustration of those who present false negatives and are filtered out despite actually being a good fit, for example working parents who can't spend a whole weekend polishing a take-home project.

[+] 908B64B197|5 years ago|reply
with take-home there's also the fact that the great candidates will probably have an offer before they even get to start working on your take-home, unless you are Google/SpaceX or pays extremely well.

Students sort companies they are interested in in descending order so your take-home will get done whenever they have some time available.

[+] 11thEarlOfMar|5 years ago|reply
We pair the take home with a code review the candidate hosts with the SW team.
[+] pandeiro|5 years ago|reply
Great summary of the problematic aspects of each of these approaches!
[+] faeyanpiraat|5 years ago|reply
It is very hard to parse your comment, a bit better formatting may help I guess?
[+] high_derivative|5 years ago|reply
I have mixed feelings about this. I despise white board and leet code. I hate it when we (at a FAANG) cannot hire some otherwise excellent candidate.

One interesting thing though is that from a market perspective, it still ends up making sense for our org (ML research lab). There are simply so many applicants, that we can afford to pass on anyone. All the candidates hired tend to still be excellent on all other fronts AND they can do these quizzes. And it does result in some hires not being the most credentialed ones with the longest CVs but the ones who outperformed during the interview for whatever reasons.

So..I can see it makes sense for the most prestigious orgs, but I still hate it, and I especially do not see why startups who are struggling to hire feel the need to copy this when they clearly do not have the same options.

Another thing that really irritates me when people in hiring committees actually pretend passing them means something about candidates' innate skill. Like, I get it, these are the rules we made up, but do we have to pretend they are that meaningful?

[+] treis|5 years ago|reply
Company that sells interview practice says interview practice is important... more at 11
[+] posharma|5 years ago|reply
I said this yesterday and will say it again. We discuss interviewing a lot here. Despite all the proposals things don't really change. FA(A)NG style interviews are just that - whiteboarding solutions to N algorithmic problems in M minutes (or minor variants of this). Either we accept this and move on with solid preparation or modify our priorities such that we're fine with not wanting the money and/or the quality of work (debatable) offered by these companies. At the end of the day, it's a choice and it's still in our hands.
[+] trap_chateau|5 years ago|reply
I'm not entirely sure I'm getting what you're conveying here. Are you just reminding everyone that change is slow to come, and to be pragmatic about these applications? (I agree with that.) Are you also claiming it's futile to attempt to change or push back against this interviewing process, at least through discussion about it?

With enough push back, I do think it's possible that those in charge of hiring in the world might reconsider how valuable leetcode tests are to judging potential employees.

[+] s17n|5 years ago|reply
I went to a bootcamp (one of the more reputable ones at the the at least - they didn't charge the attendees) and it was basically 75% interview prep, 20% working on a project designed for the job search (highest possible ratio of buzzwords/effort) and 5% everything else. If bootcamps aren't doing a good job of interview prep I doubt it's for a lack of trying.
[+] spiderPig|5 years ago|reply
Very clever post encouraging companies to spend more on this trite style of interviewing perpetuated by Aline, Gayle and the leetcode gang. Who offers said mock interviews at $100 a pop? Well interviewing.io of course!
[+] scott_codie|5 years ago|reply
Predicting future productivity is hard and there is no magic set of questions that will predict it. It seems tech companies worry too much about the questions they ask rather than interpreting the answers that they were given. The solution seems to be to not worry about it too much and just hire (and fire) liberally.
[+] jdlyga|5 years ago|reply
I get contacted by a ton of recruiters, as I'm sure most of you do. But do I have the time to practice for programming interviews? No.
[+] endisneigh|5 years ago|reply
I wonder what percentage of people at their current job would pass if they had to re-interview without preparation.

In general, companies with more money than they know what to do with can offer increasingly larger wages to make people jump through arbitrary hoops. I know people get frustrated with this, but it's really the rational thing to do.

When you consider the expected value, siphoning up 10 to 25% of your employers time to practice leetcode at work will benefit you more than doing most things at work.

[+] bluefirebrand|5 years ago|reply
I can say with certainty that I would pass the interview if I had to do it again without prep.

Mostly because when I was interviewed the company liked me enough to skip half the process, including their code challenge stuff.

[+] mihaitodor|5 years ago|reply
After numerous attempts at these software engineer interviews, I added the following disclaimer to my LinkedIn https://www.linkedin.com/in/mtodor/ and cover letters:

"I have a personal policy against any type of live coding or online coding tests during interviews and I don't enjoy or engage in any form of competitive programming. Otherwise, I am happy to work offline on coding assignments with reasonable goals and deadlines and to have in-depth technical discussions about software architecture and design as well as relevant technologies."

I'm still perfectly able to find suitable jobs with good pay as a software engineer and being upfront about the above policy prevents a lot of frustration and wasted time on my side. Yes, I do have to invest a lot more effort into looking for a suitable job, but, then again, I'd very much rather spend my free time doing that than going again through one of these interviews.

PS: I have 14 years of work experience, currently working as a senior engineer / tech lead on cloud infrastructure and this is my 8th job.

[+] twp|5 years ago|reply
This is simultaneously so wrong and so right.

So wrong because it equates passing the interview with being good at the job. These two things have almost nothing in common.

So right because practicing jumping though artificial hoops is exactly what you need to do to succeed at a large corporate. This skill is orthogonal to contributing actual value.

[+] flashgordon|5 years ago|reply
This article (whether we like the current practises or not) highlights the value of practice. What is interesting is the (anecdotal) trend of "reverse abuse"? I know several folks who routinely take time out of their schedule each month to go through one or two interviews with real companies just to not get rusty. This way when the faang time comes they are ready and don't have to block out 2-3 months leetcoding again. Clearly if you cannot beat them take advantage of them!
[+] tharne|5 years ago|reply
These posts are getting tiresome. There's way too much focus on FAANGs and SV in general.

Most of us work in hum-drum industries for companies no one's heard of, in towns no one's heard of, and have reasonable hours, good enough pay, and have never logged onto leetcode for anything other than to briefly see what all the fuss is about.

I will guarantee that at least half the developers I work with have never logged onto leetcode and many have not even heard of it. They're too busy working, raising kids, and coaching little league to be wringing their hands about the latest nonsense in SV.

[+] Swizec|5 years ago|reply
As a lucratively employed Silicon Valley engineer, I’ve never seen leetcode. Not even sure what it looks like or how it works.

And that’s with being an immigrant from eastern-ish europe.

But it sure is fun to flog on it as a meme. I doubt more than 30% of engineers working in SV startups ever actually used leetcode. And I certainly wouldn’t hire anyone above absolute junior based on their leeting skillz.

[+] boatsie|5 years ago|reply
I can understand your annoyance at the FAANG obsession, but when the total comp is literally 2-3 times that of the hum drum industries, doing roughly the same work, it makes sense that people would want to consider them.
[+] tidepod12|5 years ago|reply
A significant portion of developers (and especially developers on this website) have experience with, or desire experience with, these FAANGs and SV companies, and these posts are relevant to them. If you feel like you aren't the target audience, then you are welcome to ignore the post and click on something else. But it's a bit absurd to condescend others just because they have different experiences than you.

And with that said, you too should care, because I guarantee it's only a matter of time before those "hum-drum industries" take a look at the way SV does things and says "hey, we should do that too". This is already happening in many places. Your company could be next. You should be interested, if only to learn to spot the warning signs and nip it in the bud when you can.

[+] kappi|5 years ago|reply
Preparing for tech interview has become a cottage industry by itself. We sent people to Moon before people were subjected to these theoretical tests.