top | item 16355109

Guide to Take-home Coding Challenges

99 points| janephilipps | 8 years ago |fullstackinterviewing.com

147 comments

order
[+] songzme|8 years ago|reply
Good article, but I personally hate Take-Home coding challenge:

Some people spend an entire week on the coding challenge, which really set the bar unrealistically high. Once I turned in a project 2 hours after receiving the assignment and got rejected because they think I didn't even try. Every functionality worked flawlessly, but they want tests and for me to treat it like a real production work. But how do you motivate yourself to treat such a trivial take-home (which will never be maintained because its a take-home assignment) with the perfection you would in production code? In production you have much better insights like how your component impact others, how users will be using your app, etc.

[+] PeptoKelpto|8 years ago|reply
This is my gripe with take homes. I have spent a week on a take home before and got glowing reviews on how awesome the tests were etc. only to be rejected down the line without any feedback whatsoever. If I apply for a job now and they give me a takehome I kindly ask for an in person interview or decline to proceed with the process.
[+] on_and_off|8 years ago|reply
This is why we use a timer on our take home exercise.

You are only supposed to spend 6 hours on it, we give a 24 hours timer so you are not too pressured for time (you can take breaks, have diner, a walk, etc) .

So far it works pretty well.

I have not yet seen any applicant writing tests. Just delivering the project with the requested features and acceptable code (no memory leaks, crashes, unreadable mess, etc) is more than enough to go to the next step.

If you demonstrate enough proficiency during this step, we will probably skip most tech questions and focus on fit in the next step.

[+] stevenjohns|8 years ago|reply
> but they want tests and for me to treat it like a real production work

When hiring back-end devs, I explicitly make it clear I want at least some tests -- and the more the better. Of course I don't expect 100% test coverage but writing tests shows that 1) you understand best practices 2) you know how to write testable (readable, maintainable, clean and clear) code. If candidates chose just one module and tested that well it would be enough to get the point across.

Seeing how people write their tests is also quite important for me -- if instead of using factories (or Faker or some other strong dummy data app) they instead decide to hardcode usernames and passwords, it's a huge red flag.

[+] drewbuschhorn|8 years ago|reply
Our strategy is to save the take-home for after the in person interview, and to pay you for the time (up to the limit we expect), then a follow up to talk about the work.

I'm about to start applying to new positions and it sounds like I won't get the same courtesy, but only you can decide if your enthusiasm for the position matches the opportunity cost of the interview process.

[+] davidbanham|8 years ago|reply
I completely agree with your criticism but I disagree with your conclusion.

The fact that some applicants will spend ridiculous amounts of time on a challenge is a critical flaw. It doesn't mean that take-home challenges are worthless, though. It just means they need a functional mechanism to enforce time limits equally on all candidates.

[+] qudat|8 years ago|reply
> Some people spend an entire week on the coding challenge, which really set the bar unrealistically high.

Simple, cater the challenge so that's not possible.

Some companies are just bad at conducting interviews, that's the real tell and not take-home challenges.

I've had terrible in-person technical interviews, but I'm not going to reject the entire concept of in-person technical interviews because I've had a couple bad experiences.

If someone doesn't want to participate in a take-home coding challenge then that's their prerogative, but not everyone thinks they are a bad idea. If I were requesting a take-home challenge and a candidate I liked said they will not proceed if they have to do one, then I would probably make an exception. But that's me working at a startup where I have full interview control.

[+] janephilipps|8 years ago|reply
Thanks! I definitely understand that many people are opposed to take-home challenges because they can be very time consuming. For me, I use them as a learning opportunity, so even though they are not necessarily going to be used or maintained in the future, they can be a great way to practice or get better at a specific language or tech stack.
[+] BurningFrog|8 years ago|reply
The motivation is that you're showing off your skills with this code. You want to deliver the code that, in your opinion, is the best possible for solving the given task.

> which will never be maintained because its a take-home assignment

You will most likely be asked to add a feature or two to it in the followup in person interview.

[+] shill|8 years ago|reply
There are several possible reasons why you didn't include any tests in a take home exercise, and they are all bad.

1) You ran out of time to write tests.

This might be the least bad reason. You can probably talk your way out of this by explaining what tests you would have written if you had the time.

2) You don't know how to write tests.

This shows that you have only worked at places the don't do testing and that you haven't taken the initiative to learn about testing on your own.

3) You know how to write tests, but you don't think they are needed.

You could probably tell the interviewer why tests were not important for the particular exercise. But I have never seen a take home exercise that doesn't require a basic algorithm and some tricky conditional business logic. Both of which would benefit from unit tests.

4) The code you wrote isn't testable.

This is probably the worst reason. Everything is in one or two giant functions and you can't inject dependencies for testing without refactoring everything. This will frighten your interviewer.

[+] busterarm|8 years ago|reply
In general I find them to be completely unrealistic if you're trying to find anyone good.

Someone really good will already have a full time job, a limited number of "sick days" to use for interviewing and is probably somewhere in the interview process with at least 3 other companies -- probably closer to 5 or 7. Then they might have family responsibilities on top of that.

I budget 4, maybe 6 hours if I really like you, on any company in the application process -- tops. Almost 100% of this time is taken up by interviews. You want me to give your company a whole weekend of my time on some chance that you might hire me? In addition to hours of interviews? That already tells me that I don't want to work for you -- especially if I haven't at least interviewed with 3 technical people already. It's one of the clearest negative signals I know of.

If I work 13 days in a row without a day off, I'm going to be a crabby asshole at my job and probably blow all of my other interviews until I recuperate. Your company isn't worth that.

[+] djtriptych|8 years ago|reply
This is a little silly. If you’re “really good” you may be talking about signing on for a 4-5 expected commitment at $1M+ Total compensation over the course of your tenure. Million dollar contracts require due diligence. 4-5 man days between both companies is totally reasonable.

If you’re interviewing with 7 (!) companies at a time, maybe be a little more discerning about where you’re trying to work. Everybody wins if you’re specific on that point.

[+] busterarm|8 years ago|reply
Just for some color: The only company that's given me a weekend challenge that ever gave me an offer was 8th Light. I've done live coding and whiteboarding with other companies and received offers, but the take home work has only been productive that one time. I've had code I submitted to these looked over by better programmers than myself and heard very few negative comments, all minor.

After some years of experience, I can't see myself doing one of these again pretty much ever.

[+] davidbanham|8 years ago|reply
I think the only humane way to administer a take-home coding challenge is with an enforced time limit. That way everyone is on a level playing field and you're not advantaging those candidates that happen to have oodles of free time.

The author raises that point:

https://www.fullstackinterviewing.com/2018/02/02/the-ultimat...

But takes the wrong view on it in my opinion. The answer simply can't be "Just spend as long as it takes".

That point is the thing that led me to build a tool that enforces time limits on challenges via a custom git server. I used it to interview around 100 candidates at a company I was running at the time and it was an excellent approach.

I've since productised it as https://takehome.io/

If you want the benefits that take-home coding challenges can bring to your hiring process, but are concerned about treating your candidates humanely, I urge you to check it out.

[+] deepaksurti|8 years ago|reply
>> enforces time limits on challenges

But there is a problem. In the past when I have worked on take home challenges which usually take 3 hours or thereabouts, I have done them in 3 sessions of 1 hour each in the evening after work:

1 hour: think about the problem in detail making some notes for scenarios, edge cases, high level API design etc 2nd hour: first rough cut working draft with some test coverage 3rd hour: polish it up with more tests, docs, a README and sending it out.

I think `takehome.io` should account for time boxing across multiple sessions, else it might be pretty much useless for working engineers who do it across time boxed sessions :-)

PS: I apologize in advance if you already have this feature, but is not so readily evident on your product home page.

[+] vanadium|8 years ago|reply
This is how I do it, as a Director dn hiring manager. 4 hours max, and the exercise is entirely freeform against a set of 10 typical requirements; as a Front End Engineer candidate for my team, I want to see your opinion and execution matched to the problem at hand.

It can be done in 2 hours in the most simplistic, undeliberated way using legacy practices, but 8 if you're a perfectionist. Come in somewhere in the middle, but stop at 4 hours no matter where you're at.

I don't even require that the exercise is completed, but I want you to speak to your thought process and where you went/were going when you hit the 4 hour mark. The ability to understand our space, make good decisions, and show progress toward executing on them is ultimately what matters in the end.

[+] janephilipps|8 years ago|reply
This is really cool, thanks for sharing! I don't necessarily disagree with your point, though I do think an unrealistic time limit is even worse than no time limit. I also emphasize the importance of not taking on a challenge that is outside the scope of your ability here: https://www.fullstackinterviewing.com/2018/02/02/the-ultimat...

What are your thoughts on timed coding problems like Codility or Hackerrank?

[+] vanderreeah|8 years ago|reply
As someone who's just entered the employment market after being a freelancer for a while, I have no idea how people who already have jobs can physically participate in the lengthy interview processes most jobs require these days. Ok they can take sick days, but if you're a bad liar or have an overactive conscience that's not ideal.

I totally understand the value of coding challenges, but I've been endlessly surprised at companies that expect me to expend more than 4 hours of my life interviewing for a chance at a job that will be paying maximum 50k. These are small companies, hardly the big 4. It's really an imposition.

[+] PacifyFish|8 years ago|reply
There's a lot of contempt for take-home coding tests in this thread. They aren't a perfect solution by any means (1), but much closer to an engineer's real responsibilities than a whiteboard session.

For this reason and this reason alone, I will continue to advocate for take-home coding challenges.

(1) IMHO, the best practical solution is a pair-programming interview using a real bug or feature the company has already solved.

[+] davidbanham|8 years ago|reply
Challenges administered via pair programming can be useful if your team/company operates predominantly in that mode. If most of your actual day-to-day work is done individually, though, I think your challenge should mirror that reality.

Interviewing all candidates in a pair programming setting risks weeding out all the people who are great at the job, but get anxious in a paired interview scenario.

A job interview is already pretty stressful and anxiety inducing. Adding somebody looking over your shoulder while you type into that mix can be really rough on some people.

[+] carlmr|8 years ago|reply
I also prefer take homes. I find whiteboard interviews are much worse, because I have to relearn a lot of things in detail that aren't useful in my previous job or my next, just because these are classical whiteboard problems. Take homes are actually about job skills, not some how good are you at memorizing 100s of algorithms you should never program metric.

Whiteboarding is only good for those who are fresh out of college.

[+] Clubber|8 years ago|reply
No. Talk to them, get a feel, if they are a fit, hire them for a 3 month contract. If they aren't any good, don't hire them. If they are really bad, cut the contract short.

You will never know what someone is capable of in a few hours. People who think that works are just fooling themselves and turning away good talent.

[+] qudat|8 years ago|reply
I agree. I think take-home coding challenges are great. I have conducted take-home challenges and taken them. To me this does a better job measuring actual competency than a white board challenge.

I get that other developers get paid well to program so a lot of them are put-off by doing their job for "free." However, There's a simple solution of which has already been mentioned: cater the challenge to be completed within a few hours. Time-lock it and urge the applicants to only spend X number of hours on the challenge.

Like I have mentioned in another comment, I've been part of in-person interviews that lasted all day on top of an HR and technical phone interview. If I could cut down the in-person interview to just "personality fit" then that would be my preference.

[+] gtirloni|8 years ago|reply
Time suggestions are usualy completely off the mark. Four hours turn into 20 for anything resembling production code (which is what companies are interested in).

I like the idea of take home projects but it shouldn't come right after a phone screen or even the first real interview. More like as close to the last step as possible.

And then there's the question of what other professions ask candidates to work X hours for free. Let's assume our field is too special and not get too deep into the rabbit role ;)

[+] fizx|8 years ago|reply
A take-home coding challenge will bias your pool of applicants towards relatively junior overachievers who are barely skilled enough to complete your challenge.

More senior and skilled people will decline to participate, because they know they can get a better return using the time otherwise spent on the coding challenge to network for more and easier opportunities.

So now you've done a great job of raising the floor of your interviews for little direct cost, but you've also lowered the ceiling.

[+] thesmallestcat|8 years ago|reply
There's a good chance somebody on the other end's going to run your program without really vetting it. Especially if you include a convenient "test runner." Not a bad attack vector, just get your alias past the first screen and you might get arbitrary execution inside their firewall on a host with access keys and the like.
[+] kirbypineapple|8 years ago|reply
I failed a take home challenge from Netflix and was a sad with how it went down. I was given 24 hours to complete the assignment and told that I 1.) shouldn't try to be clever, and 2.) that it shouldn't take more me more than 2 to 3 hours to complete. Early on I realized that the only way to get optimal performance was to use an interval tree to store my data, but thought that an interval tree was being "too clever". I implemented a more naive solution and submitted it with comments indicating that I believed that an interval tree was the optimal data structure to use in this case.

The response was that an interval tree was the correct data structure to use in order to pass the performance component of the assignment and that they were not interested in continuing with me any further. In hindsight I should have reached out to validate whether an interval tree was "too clever", and I think that was a more important factor in my failure than anything.

[+] q3k|8 years ago|reply
If you're thinking of interviewing your candidates with such methods, it's worth noting that their current employment agreement might prohibit them from participating due to IP clauses (more exactly, that they require permission to release any code that they've written, even in their spare time).

This can restrict your pool of candidates quite dramatically, as most US companies seem to have their engineers sign such agreements.

[+] mighty_bander|8 years ago|reply
I was hired by my current employer following a take-home coding challenge. I consider the experience very positive, and it has allowed us to hire skilled developers. However, I notice a few differences between my experience and those of people who complain about these challenges:

1. The challenge was timed, and limited to 3 hours. This is critical, because it respects the applicant's time. 2. Similarly, other interviewing was limited to a very brief "are you actually a developer at all" phone screen and a 30 minute personality interview. 3. I was allowed to choose the framework and language I was most comfortable with, demonstrating that the employer was interested in how well I worked, not in how well I knew the particular framework chosen for their product. I think it probably helped that I used the same stack advertised in the job description, but I appreciate the flexibility being there.

Interviewing is a separate skill from software development. A take-home challenge isn't necessarily going to tell you everything about a person, but it seems to sort candidates into high/medium/low skill levels with a higher degree of accuracy than a paper or whiteboard tech interview.

[+] booleandilemma|8 years ago|reply
Please don’t normalize “take-home coding challenges”.
[+] closeparen|8 years ago|reply
At least make them instead of, rather than in addition to, whiteboard coding sessions.
[+] Clubber|8 years ago|reply
I don't understand why people put up with all the bullshit that goes into tech hiring these days. I mean are companies that offer "market rates," putting people through this? No wonder they can't find anyone good.
[+] megous|8 years ago|reply
So if take-homes are to become the best practice for interviewing, perhaps there should be some service for re-using old ones that companies can trust that the candidate did for some other company and under which constraints. Otherwise it's just too much of a time waste for candidates, repeating take-homes for every single candidate company.

It would be like personal projects but with a few guarantees on top, like that the candidate got the work assigned by someone else, and s/he did it under some time/other constraints. And there would perhaps be a public free-form feedback from the other comapny (no rating). So the next comapny has to judge the work itself without ability to pre-filter based on whatever rating the previous one would have given (to avoid bias, because the scoring would be non-objective/non-standardized).

But I would still not like the take-homes. :)

[+] dawnerd|8 years ago|reply
I dunno, I’ve just walked from interviews that want take home tests. Plenty of other offers out there that don’t require it. If my Github and past experience are not enough then we’re not a match. Companies are going to put you on a probationary period anyways so if they don’t like your work they’d fire you then.
[+] eee6e6e666ee|8 years ago|reply
If there is a guide or a book for taking an interview, the interview has already failed and is measuring the wrong thing.
[+] Arainach|8 years ago|reply
I disagree. Even if you had a hypothetical perfect interview that tested exactly the right combination of raw logical ability/concept familiarity/effective communication, there would always be room for a guide that reminded people exactly how to best prepare.

Interviews need to test for certain characteristics in a limited amount of time. Even if there was some way to say "here's the code I've wrote, e-mails I've sent, and videos of me leading team meetings in the last year", no company would have the time to look at those materials.

As such, we're stuck with the common question formats trying to determine if you communicate with your interviewer well (understanding what's being asked, explaining your thought process and what you're trying to accomplish, talking about anecdotes and building rapport) while also showing off some technical understanding of implementation and design. No one's yet found a better system that's more respectful of both the company's time and the candidate's time.

These "take home projects" are an interesting variation that I'm not going to dismiss out of hand, but it's hard enough to take 8 hours out of my day for an interview loop - adding more hours in the evenings isn't something that scales well to interviewing with a bunch of places.

[+] rboyd|8 years ago|reply
don't do unpaid take-home coding challenges. particularly if you have no idea what the company is offering for compensation should you get the offer.
[+] janephilipps|8 years ago|reply
For a well established developer, this may be true, but for newer developers, they often have to jump through a lot of hoops to prove their skills first, especially if they come from a non-traditional background and do not have a CS degree.

Do you think this issue is less pronounced with the move toward companies providing compensation guidelines up front?

[+] qudat|8 years ago|reply
I only agree so far as the take-home challenge has no time restrictions.

I would choose a take-home coding challenge over any white boarding challenges.

[+] PeptoKelpto|8 years ago|reply
I think it's awesome that you can become a software engineer without any engineering schooling. Where I live you have to take software engineering in University to be a software engineer. Bootcamps and certs usually only get you dev jobs.
[+] grim-jokes|8 years ago|reply
Instead of take home challenges, I think I would rather give a potential employee some code snippet and ask them to explain to me in their own way as to what is going on and if they see/find any particular issues.

This will theoretically let me gauge their attention to detail. Ideally I think it would show me how familiar they are with the programming language the snippet was written in as well.

[+] zerr|8 years ago|reply
Flagged. Since no payment is mentioned, so it is assumed that a candidate will do it for free... Lets not encourage such deeds.