When I had less than 5yrs experience I used to ask these types of questions, thinking they showed the ability of a developer to reason out critical software issues. Then I met a guy who had practically memorized everything I could ask. He spit out answers so fast it was appallingly clear he was regurgitating answers from a book. This proved nothing about him and everything about me.
Today, I recognize that good software development owes nothing to data structure knowledge or obscure algorithms. This knowledge can't hurt but very few jobs or languages require you build a skip list from scratch. Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.
For functional purposes, I consider myself to know something if I could page in the details well enough to implement it in 5 minutes with Wikipedia or less.
How this plays in interviews is interesting for just how inconsistent it is. Sometimes I win the algorithm lottery and have had reason to implement the obvious solution myself at least once, and the interviewer learns my memory works. Sometimes I don't know what they're going for but come up with a reasonable answer anyway and the interviewer is thrilled I managed to have an independent thought. Sometimes I lose and the interviewer is clearly trying to guide me to the answer they want to see and I have no idea what they're getting at.
Though not quite as bad as a friend failing an interview for not using a hash table, even though the answer he described was a hash table—just with the identity function as the hash.
(Though maybe about as absurd as one interviewer digging for any formal training I might have had in C++. Like you can teach that language in a semester—please. There are maybe 4 proper experts on the planet.)
I think there might be a happy medium. Knowing how to implement a specific algorithm is not necessary. Knowing of algorithms and picking the right one is important. For example, a junior developer apparently forgot about hash sets. He used a list for lookup. Worked fine in his local tests. Fail miserably when we had to load millions of records. A more seasoned programmer should be able to explain why a hash would work well here.
I used to do interviews in one company I worked for and we had this online test, which consisted of four tasks very similar to those in the list.
There was this one candidate who, by mistake, was given the test twice. Apparently he took offence to that. He found that his friend did the exact same tasks so he copied them.
The designers of that test tout that their system is able to detect plagiarism, but that was not the case here.
Anyway eventually we hired him because even though his first attempt wasn't particularly impressive and he cheated a face-to-face interview revealed that he was a skillful programmer.
One thing I learned from this experience is most of the time these test simply confirm that the candidate went to college - sometimes a particular one.
>Today, I recognize that good software development owes nothing to data structure knowledge or obscure algorithms. This knowledge can't hurt but very few jobs or languages require you build a skip list from scratch. Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.
While I don't think that the current state of interviewing is perfect, this statement is pretty out there. Good software development owes NOTHING to data structure knowledge? Choosing the right data structure is critical for software that performs well (or performs at all once you get to a certain scale), and demonstrating knowledge of various data structures shows at least that a candidate knows a bit about which tools they have available to them on the job.
Of course not many people are implementing skip lists from scratch in their everyday job, that's why in the list of 500 questions none of them are "Implement a skip list". Data structure questions rarely ask you to implement some data structure from scratch, they ask you to look at a problem and figure out which data structure best lends itself to the solution.
I won't argue that asking questions about obscure algorithms isn't a problem, but good interviewers aren't asking questions that need obscure algorithms, they are asking questions that need algorithms like DFS, basic variations of sorting, and basic DP problems.
The best way to memorise hundreds of different algorithms (or anything else for that matter) is to understand the few underlying principles beneath them. I would be surprised if he knew them all and couldn't come up with intelligent ways of mixing them for achieving a new task.
- Programmers are increasingly more "API plumbers", not "pathfinders". The change being you have to more often create an "easy to maintain" glue layer between lots of existing code (most of which your company wrote a bunch of years ago, or is external to your code in a framework or library) instead of solving a problem from scratch using CS basics.
Interview questions haven't shifted to "this library has such and such function calls, how would you expect it to report an error?" or "describe two distinct UI frameworks you have used and explain their differences". They're still very much in the era of OK we have two rectangles, how do we find the overlapping pixels.
- Software is increasingly built in teams. We don't ask about "how you handled a difficult person on your code review" or "how would you break this change in to two logical commits" or even "what's a good name for this function", all of which are more important than knowing the big-o for some obscure interval tree.
- CS 101 algorithms have very little to do with how they are implemented. e.g, in a recent discussion with a co-worker I discovered libstdc++ hashtable's iteration orders depend not only on hash code but also on insertion order. Looking at how libstdc++ and MSVC implemented hashtables was both very difficult and exhausting to follow and had many many more details than the typical CS 101 approach of just hash the key and direct lookup the bucket! Turns out writing code that is really fast needs more than knowing the big-o.
- Big O is such a garbage simplification of all the "fundamental" knowledge. Why not just ask some fundamental graph or combinatoric counting questions? Why nothing about what is computable and what is not, or which languages are regular? What about information theory? How come no information recall or database questions? or even these days with all the ML, why not some basic linear algebra?
Here's my gripe with this interview process. It encourages engineers to spend months and years into learning and remembering optimal solutions to textbook problems.
A REAL engineer would spend the same time doing something productive....things like:
- Writing a sql optimization for their current company that saves resources
- Creating shared memory interfaces for faster processing of logs
- Modifying filesystems or kernel to create their own log processing
- Modifying JVM to optimize critical code
- Rewriting parts of a server to set up security alerts
- Fast encryption and decryption of in-memory cache records
- Writing an interpreter for faster generation of low level code for high speed processing
- And on and on
My point is, today's interview process treats engineers with such complicated experience as shit because they spent time working on complex projects and couldn't memorize these questions. But then these same companies will hire some mediocre engineer who cannot think out of the box but is an expert at memorization.
Said mediocre engineers then hire other mediocre engineers using the same process, because these mediocre engineers do not even have the ability to grasp what kind of engineering would lead to above points.
To top it off, these same mediocre engineers sweep this problem under the rug claiming they are trying to reduce false negatives. Lol!
Google style interviews don't screen for the best engineers, but they screen strongly for subservience. The fact that a candidate has to go out of their way to prepare for these interviews indicates that the candidate is likely willing to jump through whatever hoops they're told to jump through.
It's similar to how investment banking jobs place such a high premium on Ivy league degrees with high GPAs despite the actual job itself being so simple a high school student could do it. Someone who's willing to jump through all those hoops to get into a top school and graduate with a high GPA has a strong track record of following orders and is less likely to complain about 100 hours workweeks.
If this materially impacts the bottom lines of companies that are practicing this approach eventually companies that are not using this process will prevail.
> Here's my gripe with this interview process. It encourages engineers to spend months and years into learning and remembering optimal solutions to textbook problems.
Wrong. Questions aren't designed to be memorized. It shouldn't take months, or years, to prepare for this stuff if you already have a CS background unless you're really slow.
It's a tragedy of our industry that this super FUN list of basic algo problems, is connected in everyone's mind to lame-ass whiteboard interviews. These problems should be like playing scales: engineers should just work through these, practice them till they're no-brainers. But not to land that sweet entry-level SWE job at Google, but just to grease your own wheels in day-to-day coding and problem-solving.
I've never been to an interview that would ask this kind of crap, and would be very surprised if a future interview did. So yeah, I'm just looking at it as a list of possibly interesting problems, which makes it a way better link than "interview help".
I agree -- in addition, such problems can also be seen as a springboard into research - tweak any of them or ask if you can prove that the solution is optimal, and you are quickly in the realm of open questions.
As a fun warm up-question, I always ask the candidate what editor and keyboard they use. The candidate who geeks out about their plugins and cherry switches inevitably passes all 5 rounds; the candidate who uses eclipse with maybe a vim plugin will go on to fail about 75% of the time. It's a good filter too; I'd say 1 out of every 20 candidates expresses strong convictions.
I'm seriously considering using it as a dog whistle and skipping the experience deep-dive (which tests if you are a good story teller), the systems design questions (which tests if you can sync to the interviewer's sense of judgement), and the algorithm hazing (which tests social anxiety). But I'm worried there is some kind of discriminatory correlation baked into that metric.
I worked at a company that tried that for a while and eventually decided to completely abandon it, because it was correlating so strongly to a particular set of experiences shared by the (mostly male, CS degreed) interviewers. Particularly with more junior engineers, where having a sexy setup is more likely to correlate who you hung out with in college than whether you actually coded enough to find a need for it yourself. So if you're going to play with that as a signal, I'd advise you to be very careful.
I have exactly the opposite bias. I think people who get really into keyswitches, alternative keyboard layouts, and .vimrcs are prone to getting distracted by unimportant details. Of course, we all want to use the most comfortable and productive tools, but a skilled person will attach more weight to things that have more impact.
Still, i'm glad i know people like that, because it's really easy to get a knowledgeable recommendation when i need a new keyboard!
As someone who totally geeks out about my editor (vim, what else!?), I can understand that.
However, I'm pretty sure that out of the many, many developers I've worked with, including some brilliant ones, I'm one of barely a handful who gives much serious thought to editors. This could also reflect a bias of location - HN'ers are more likely to geek out over editors, but I think in general people in Silicon Valley tend to care more about this, as opposed to people in Israel, where I'm from.
I do sometimes ask similar questions about more technical topics that the candidate should be familiar with, e.g. which DB do you use (backend engineer), which JS framework you use, etc. This is something that the strong candidates must have brushed up against, and should have a strong opinion about.
Just out of curiosity to see how I do, how would you rate using stock Intellij IDEA and a Macbook? Additionally am interested because some top algorithm competitors (of which I am not one) have this very setup.
I often ask people about their editor setup too - I tend to find that the level of enthusiasm they have about their editor is a moderately good indicator of their enthusiasm in general (though not, necessarily, their competence - I think that's stretching it too far). If they give me a look as if to say, "Why on earth is he asking this?" or they reply with something like, "Oh, whatever to get the job done... what do you use?" then it doesn't suggest to me that this is a person who gets much pleasure out of programming.
The other nice thing I find about asking this is that it isn't a question they've prepared for, so you tend to get an honest answer.
I have a friend who memorized all of these types of questions and answers. It worked because he got offers from Google, Facebook and Uber, amongst others. The key is to pretend like you're figuring it out on the spot and not just regurgitating it from memory. I plan on doing the exact same thing when I look for my next job. It's unfortunate, but the reality of the state of our interviews.
I have two kinds of questions: (1) tell me about a problem you've solved, and (2) let me tell you about a problem I'm facing. I'll never ask candidates about made up problems. Why would I waste our time talking about fake stuff, when there's so many actual real problems in front of us that we could dive deep on?
The best, most fair interviews I've been through follow this pattern. I remember one of them told me about a current problem they had, and told me "You can google anything you want, you have 30 minutes. After that, you should tell me how would you solve this problem, including any tradeoffs you've made".
This is by far the most realistic scenario employees of most companies face. It doesn't matter so much that one doesn't remember the specific details of tree traversal or how linked lists are implemented. It matters much more that one is able to find it out quickly, understand it and apply it. Those are the skills you're looking for.
In my opinion, it is great having people trying to "memorize" those questions. If they understand them, it would be like one half of formal computer science studies. What is a shame is people graduated from Computer Science university courses not remembering a clue about algorithms, because they "memorized" just to pass the exam. BTW, some of the interviews may be are too much hard, requiring being a Donald Knuth, having a "intelligence" index of 160, and being a nice team player (if you target very smart people you'll deal with crazy jerks most of the time -until those jerks become tamed, somehow-). Super-smart people should target working for the NASA or similar, not for writing APIs in Python 50 hours a week with just 3 week per year holidays.
Nice list, but a side-note: all this focus on interview-preparation may give the false impression that algorithms and data structures is a closed body of work, when in fact it is an active and lively field of research with many basic questions not yet understood.
Someone could go through these 500 questions and for (almost) all of them formulate variants/extensions that would be open research questions. Or alternatively, ask for each of them: is this the best possible solution? Can I prove it? What if I restrict what the algorithm can do? What if the data comes online? What if the input is noisy? etc. etc. etc.
Please stop telling companies not to use this method of interview.
Sincerely, Michael.
I use it to filter myself out before wasting anyone's time.
What I see here is an AND gate, and their use of this process itself triggers the false in my own filters-for-job-positions system.
I see it as an uniform filter. The company is filtering me out, or in. And, if I understand the reality of their options for filtering methods, I can use that information to filter them back. Interestingly enough, this one is pretty uniform for me, if they use the method of out-of-a-book problems in either an on-the-whiteboard or pair-programming session, they have solved the problem of whether I want to work in that position without my having to trouble myself with memorizing problems to answer, nor waste their time screening me.
The filter I am applying here is against the out-of-the-book part of this. If a company wanted to provide a previously solved bug in a section of their own code, with enough information to debug but perhaps not enough testing written into the system, to me this would not filter that position out. As long as the job would be similar in setup to the methods used during the interview this would be fine by me. Difficult, as the level of chaos internally to any interviewee is very high, and most positions would not be nearly as chaotic as the induced level this hiring process creates.
I think asking on-the-board questions, or running some pair programming, is a filter for perspective, and attempts to identify how an applicant handles chaos.
To put it another way, the company wants people who when they hear an alarm will file out of the business in an orderly fashion, doing that job that has out-of-a-book interviews during hiring. I personally walk towards alarms, not away, and am plenty happy not to be stuck in a position that isn't supposed to have someone like me in it.
My system is tragically flawed, of course. A company could easily use an interview process they consider arbitrary to filter the lowest tier of applicants (non-tech people fishing for a few months paid internet browsing), and I could, not recognizing the insignificance of it, preselect them false when they would have stopped that interview after ten minutes and hired me on a handshake.
If you are just asking Algo and probably pseudo code, these questions might help, but if you ask to write the proper code with boundary conditions etc, that makes it difficult even if you know the Algo. Of course I am assuming that no one memorizes the code ;).
I used to give people a description of how an algorithm was implemented, and ask them to write it out in code. Specifically we'd take the conversation to boundary conditions, failure conditions, testing, etc, from there. I was reasonably happy with the outcome.
The solutions here make the common mistake of ignoring the cost of stack space, and thereby incorrectly claiming that recursive solutions have lower space complexity than iterative solutions.
Not if you point out that whatever platform you're coding in employs tail recursion I guess? But yeah the intermediate results will still have to be stored somewhere.
[+] [-] wheaties|9 years ago|reply
Today, I recognize that good software development owes nothing to data structure knowledge or obscure algorithms. This knowledge can't hurt but very few jobs or languages require you build a skip list from scratch. Instead, writing good software requires the ability to maintain a system, to debug problems, and to give constructive code review. That's it.
[+] [-] LnxPrgr3|9 years ago|reply
How this plays in interviews is interesting for just how inconsistent it is. Sometimes I win the algorithm lottery and have had reason to implement the obvious solution myself at least once, and the interviewer learns my memory works. Sometimes I don't know what they're going for but come up with a reasonable answer anyway and the interviewer is thrilled I managed to have an independent thought. Sometimes I lose and the interviewer is clearly trying to guide me to the answer they want to see and I have no idea what they're getting at.
Though not quite as bad as a friend failing an interview for not using a hash table, even though the answer he described was a hash table—just with the identity function as the hash.
(Though maybe about as absurd as one interviewer digging for any formal training I might have had in C++. Like you can teach that language in a semester—please. There are maybe 4 proper experts on the planet.)
[+] [-] virmundi|9 years ago|reply
[+] [-] Tade0|9 years ago|reply
There was this one candidate who, by mistake, was given the test twice. Apparently he took offence to that. He found that his friend did the exact same tasks so he copied them.
The designers of that test tout that their system is able to detect plagiarism, but that was not the case here.
Anyway eventually we hired him because even though his first attempt wasn't particularly impressive and he cheated a face-to-face interview revealed that he was a skillful programmer.
One thing I learned from this experience is most of the time these test simply confirm that the candidate went to college - sometimes a particular one.
[+] [-] adamnemecek|9 years ago|reply
[+] [-] sumeno|9 years ago|reply
While I don't think that the current state of interviewing is perfect, this statement is pretty out there. Good software development owes NOTHING to data structure knowledge? Choosing the right data structure is critical for software that performs well (or performs at all once you get to a certain scale), and demonstrating knowledge of various data structures shows at least that a candidate knows a bit about which tools they have available to them on the job.
Of course not many people are implementing skip lists from scratch in their everyday job, that's why in the list of 500 questions none of them are "Implement a skip list". Data structure questions rarely ask you to implement some data structure from scratch, they ask you to look at a problem and figure out which data structure best lends itself to the solution.
I won't argue that asking questions about obscure algorithms isn't a problem, but good interviewers aren't asking questions that need obscure algorithms, they are asking questions that need algorithms like DFS, basic variations of sorting, and basic DP problems.
[+] [-] skybrian|9 years ago|reply
[+] [-] lefstathiou|9 years ago|reply
[+] [-] jpiabrantes|9 years ago|reply
[+] [-] tzs|9 years ago|reply
What do you mean by "maintain a system"?
[+] [-] nanospeck|9 years ago|reply
[+] [-] maxaf|9 years ago|reply
[+] [-] ziikutv|9 years ago|reply
[+] [-] lefstathiou|9 years ago|reply
[deleted]
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] darioush|9 years ago|reply
- Software is increasingly built in teams. We don't ask about "how you handled a difficult person on your code review" or "how would you break this change in to two logical commits" or even "what's a good name for this function", all of which are more important than knowing the big-o for some obscure interval tree.
- CS 101 algorithms have very little to do with how they are implemented. e.g, in a recent discussion with a co-worker I discovered libstdc++ hashtable's iteration orders depend not only on hash code but also on insertion order. Looking at how libstdc++ and MSVC implemented hashtables was both very difficult and exhausting to follow and had many many more details than the typical CS 101 approach of just hash the key and direct lookup the bucket! Turns out writing code that is really fast needs more than knowing the big-o.
- Big O is such a garbage simplification of all the "fundamental" knowledge. Why not just ask some fundamental graph or combinatoric counting questions? Why nothing about what is computable and what is not, or which languages are regular? What about information theory? How come no information recall or database questions? or even these days with all the ML, why not some basic linear algebra?
[+] [-] NTDF9|9 years ago|reply
A REAL engineer would spend the same time doing something productive....things like:
- Writing a sql optimization for their current company that saves resources
- Creating shared memory interfaces for faster processing of logs
- Modifying filesystems or kernel to create their own log processing
- Modifying JVM to optimize critical code
- Rewriting parts of a server to set up security alerts
- Fast encryption and decryption of in-memory cache records
- Writing an interpreter for faster generation of low level code for high speed processing
- And on and on
My point is, today's interview process treats engineers with such complicated experience as shit because they spent time working on complex projects and couldn't memorize these questions. But then these same companies will hire some mediocre engineer who cannot think out of the box but is an expert at memorization.
Said mediocre engineers then hire other mediocre engineers using the same process, because these mediocre engineers do not even have the ability to grasp what kind of engineering would lead to above points.
To top it off, these same mediocre engineers sweep this problem under the rug claiming they are trying to reduce false negatives. Lol!
Sad state of the industry!
[+] [-] JDiculous|9 years ago|reply
It's similar to how investment banking jobs place such a high premium on Ivy league degrees with high GPAs despite the actual job itself being so simple a high school student could do it. Someone who's willing to jump through all those hoops to get into a top school and graduate with a high GPA has a strong track record of following orders and is less likely to complain about 100 hours workweeks.
[+] [-] qaq|9 years ago|reply
[+] [-] pkaye|9 years ago|reply
[+] [-] SamReidHughes|9 years ago|reply
Wrong. Questions aren't designed to be memorized. It shouldn't take months, or years, to prepare for this stuff if you already have a CS background unless you're really slow.
[+] [-] khazhoux|9 years ago|reply
[+] [-] acheron|9 years ago|reply
[+] [-] lkozma|9 years ago|reply
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] xaybey|9 years ago|reply
I'm seriously considering using it as a dog whistle and skipping the experience deep-dive (which tests if you are a good story teller), the systems design questions (which tests if you can sync to the interviewer's sense of judgement), and the algorithm hazing (which tests social anxiety). But I'm worried there is some kind of discriminatory correlation baked into that metric.
[+] [-] sbisker|9 years ago|reply
[+] [-] twic|9 years ago|reply
Still, i'm glad i know people like that, because it's really easy to get a knowledgeable recommendation when i need a new keyboard!
[+] [-] edanm|9 years ago|reply
However, I'm pretty sure that out of the many, many developers I've worked with, including some brilliant ones, I'm one of barely a handful who gives much serious thought to editors. This could also reflect a bias of location - HN'ers are more likely to geek out over editors, but I think in general people in Silicon Valley tend to care more about this, as opposed to people in Israel, where I'm from.
I do sometimes ask similar questions about more technical topics that the candidate should be familiar with, e.g. which DB do you use (backend engineer), which JS framework you use, etc. This is something that the strong candidates must have brushed up against, and should have a strong opinion about.
[+] [-] poikniok|9 years ago|reply
[+] [-] bshimmin|9 years ago|reply
The other nice thing I find about asking this is that it isn't a question they've prepared for, so you tend to get an honest answer.
[+] [-] rachelbythebay|9 years ago|reply
[+] [-] commentai|9 years ago|reply
[+] [-] pfarnsworth|9 years ago|reply
[+] [-] khazhoux|9 years ago|reply
[+] [-] alansammarone|9 years ago|reply
This is by far the most realistic scenario employees of most companies face. It doesn't matter so much that one doesn't remember the specific details of tree traversal or how linked lists are implemented. It matters much more that one is able to find it out quickly, understand it and apply it. Those are the skills you're looking for.
[+] [-] inetknght|9 years ago|reply
[+] [-] faragon|9 years ago|reply
Anyway, algorithms matter.
[+] [-] lkozma|9 years ago|reply
Someone could go through these 500 questions and for (almost) all of them formulate variants/extensions that would be open research questions. Or alternatively, ask for each of them: is this the best possible solution? Can I prove it? What if I restrict what the algorithm can do? What if the data comes online? What if the input is noisy? etc. etc. etc.
[+] [-] mtreis86|9 years ago|reply
Please stop telling companies not to use this method of interview.
Sincerely, Michael.
I use it to filter myself out before wasting anyone's time.
What I see here is an AND gate, and their use of this process itself triggers the false in my own filters-for-job-positions system.
I see it as an uniform filter. The company is filtering me out, or in. And, if I understand the reality of their options for filtering methods, I can use that information to filter them back. Interestingly enough, this one is pretty uniform for me, if they use the method of out-of-a-book problems in either an on-the-whiteboard or pair-programming session, they have solved the problem of whether I want to work in that position without my having to trouble myself with memorizing problems to answer, nor waste their time screening me.
The filter I am applying here is against the out-of-the-book part of this. If a company wanted to provide a previously solved bug in a section of their own code, with enough information to debug but perhaps not enough testing written into the system, to me this would not filter that position out. As long as the job would be similar in setup to the methods used during the interview this would be fine by me. Difficult, as the level of chaos internally to any interviewee is very high, and most positions would not be nearly as chaotic as the induced level this hiring process creates.
I think asking on-the-board questions, or running some pair programming, is a filter for perspective, and attempts to identify how an applicant handles chaos.
To put it another way, the company wants people who when they hear an alarm will file out of the business in an orderly fashion, doing that job that has out-of-a-book interviews during hiring. I personally walk towards alarms, not away, and am plenty happy not to be stuck in a position that isn't supposed to have someone like me in it.
My system is tragically flawed, of course. A company could easily use an interview process they consider arbitrary to filter the lowest tier of applicants (non-tech people fishing for a few months paid internet browsing), and I could, not recognizing the insignificance of it, preselect them false when they would have stopped that interview after ten minutes and hired me on a handshake.
[+] [-] peteretep|9 years ago|reply
[+] [-] rv11|9 years ago|reply
[+] [-] peteretep|9 years ago|reply
[+] [-] fjdlwlv|9 years ago|reply
[+] [-] curiousDog|9 years ago|reply
[+] [-] signa11|9 years ago|reply
[+] [-] gragas|9 years ago|reply
[deleted]