Yeah I remember the Google interview a few years back. I did it mostly for fun, and because their recruiters kept contacting me.
So there we were, building a tree out of a forest of existing smaller trees or such. That was after they couldn't find my resume, and used a 5 year old one they found some place. They seemed annoyed and tired. By that time I felt nothing short of me proving P!=NP would have help changed their mood. I mean, they didn't even ask me what I knew, what I worked on lately. Heck, I could have been herding goats for the previous 5 years, but if I could have solved that tree problem, well, Googleplex here I come, I could have been herding goats there, maybe as part of some a green eco-initiative project...
I guess half way through I just kind of decided that Google must be a pretty bad place to work at and kind of gave up. I was tempted to ask them how often they had to build the tree out of subtrees at Google. Or in this case how many times did they really have to invert the binary tree, but I am nicer that than, maybe they really did have a pretty shitty day.
Ever since then I've declined to interview at Google. I get contacted every year like clockwork "Wanna try again?", "No thanks, good luck inverting binary trees though..."
Oh God this brings back memories. I was actually offered a job at Google (by some miracle) but turned it down. The interviews (7 in total) were an awful experience.
I had to solve problems that I had not looked at since my masters and were mostly theoretical CS things which nobody outside of academia ever has to actually use.
I had a brief read through of their coding style a few days before so I solved it in a very Google way which is what most likely got me the job. Even though doing it their way was just awful (in terms of implementation).
There is no doubt that Google's search algorithms are the best but I can't help but think the rest of their services are only high performing because they can through their enormous data centre resources at the problem.
I also hated the attitude of the interviewers and engineers I met. They were so up their own asses. Their way was the best possible way and if you didn't agree well fuck you!
I have only turned down two jobs in my career that I 100% do not regret and Google is one of them.
I actually once had to build a tree from subtrees for work (I was modelling some kind of process as a tree, given pairs of data). It was good fun, and a break from the mundane CRUD-based work I usually do. I did it with a whiteboard initially, and ended up producing a fairly robust solution.
The difference was that I was able and to try various approaches out without pressure. I had access to my old algorithms textbook as a safety blanket. Some friends dropped by and we went out for supper, and I chewed over my ideas with them.
I also had two weeks to solve the problem, and even though it took one Friday night to solve the problem, being given 2 weeks to produce a working solution put me at ease.
Doing something like that in a high-stakes interview would be nearly impossible for me.
I was told by someone many years ago to always take a copy of my resume with me to an interview. That advice has paid off a couple of times. These days I also keep an email draft ready to send.
Yes, the thing I hated the most about my Google interview experience was that there was no discussion whatsoever of my previous projects. But there was the question "What would you like to do?" and I realized later that I should have hijacked that question to talk about my past projects I enjoyed the most.
They don't start looking at your resume or asking what you have been up to until you pass the technical interviews. Which means that if you resume or experience isn't up to snuff, the interviews are a complete waste of time for you because of a decision that takes them five minutes to make.
The last interview I did was for a node.js position. I was told to bring my laptop. When I got there, the put me in an office and handed me a packet with instructions on checking out a nearly empty git repo. Then they said "Write an API that meets as many of the following requirements as possible and push the code. We'll be back in 3 hours."
After 3 hours, I was taken to a conference room where I demonstrated the API. I also talked about a bug I had that had hung me up, but I fixed it right there in the conference room.
Got the offer. It was a great experience. I enjoyed the challenge, but it felt useful and related to the work rather than just esoteric problem that would not likely ever be required by the job, and if it was I'd have time to research it and figure it out.
That actually sounds like a decent way to do things. The only thing I'd change is making it shorter and trying to incorporate some pair programming at least to some minor degree to see how well they communicate along with developing.
This, especially if the conference room is filled with a reasonable representation of their programmers, rather than just one or two people, and they get to ask questions and we get to talk about the code.
Hearing someone talk about their own code and explain their approach and the up and down sides of it, is a lot more valuable than simply having the correct answer roll out.
The interview process took about two weeks and on the whole was pretty reasonable for both sides.
There was a short phone screen (~30 minutes)
Then I had two technical exercises to do. For each, I was given a reasonable time period (4 hours) that started when I visited a special link to the get the problem description, and then used whatever tools I wanted to get it done, then email in the completed exercise.
The two exercises were not completely trivial, but were relatively straightforward and actually involved skills relevant to the position and not arcane puzzles or comp-sci theory. (Basically, the first was to write a Postgres schema from scratch (total 5 or 6 tables and maybe 30 columns across all the tables). The second task was to write a CSV parser (including reasonable error checking/recovery) to load data into the schema from the first problem.
Finally, there was the main interview (conducted via Facetime, this is a remote position). That ran for close to 3 hours.
This worked well, since it gave me a feel for what the work would actually be like, and I was given, as the candidate, every oppurtunity to put my best foot forward - no whiteboard exercises, no surprises.
>and then used whatever tools I wanted to get it done, then email in the completed exercise.
This is a much more realistic way of testing someone's skills. Asking someone to write code while they watch is not.
I am personally horrible at coding while someone is looking over my shoulder. I am just too preoccupied with their presence and the fact that they are watching. And, unless a company is still into pair programming (is anyone these days?), it's not a valid test.
Give me a real problem, reasonable time to solve it, and the tools I'd have in the real world. Then, I can show you what I can do in that same real world vs. how well I interview in some contrived format.
I'll be the contrarian and claim that inverting a binary tree is a perfectly reasonable interview question. (Although not anymore now that it's famous). It combines basic background knowledge (what is a binary tree and how is it structured?), ability to reason logically (binary tree algorithms are often recursive, "inverting" a tree with no children is a no-op, otherwise you want to swap the children, and the children's children, and...), and enough familiarity in one language to write simple constructs without an IDE or stackoverflow holding your hand. None of this requires you to be Jeff Dean.
I will agree that the guy who kicked all this off has a legitimate complaint if the recruiter failed to adequately explain the interview process, and I'm sure there are lots of bad interviewers that focus on meaningless details or obscure trivia. But I don't agree that the concept of having candidates write code is fundamentally awful.
I also think it's an extremely simple question, but only if you've ever taken a data structures class.
Turns out a lot of people in our industry don't! I have a Computer Engineering degree, and know this stuff mainly due to this being my hobby (and having tried out for the IOI in high school). But there are former classmates who are only vaguely aware of these data structures, because they never got any exposure to them despite writing real programs.
I don't think there's a super strong argument that most computer engineers need to know much of anything about base data structures. There are so many other concepts that are more important to writing and deploying useful software (especially in the enterprise world) that at no point involve writing complex data structures. Data structures (or rather, knowledge of the internals) are much more domain specific than some of us would like to admit, especially in the age of RoR and Unity. There are many more design patters more important to know about than being able to implement quicksort (it took 6 years for researchers to write the first bug-free implementation!)
Google does it because they can. They're probably filtering out a lot of people who treat code as prose rather than a technical manual though. Which is probably why Google libraries look the way they look.
The biggest surprise for me is how many game programmers don't know about this stuff. It seems like the sort of stuff you'd run into pretty quickly
I don't get the obsession with binary trees questions on interviews. I've had several in a couple different interviews. You know how many times I've coded a binary tree algorithmn, in 15 years as a professional programmer? Zeeeeero. They just aren't that common in actual code.
Why not ask questions about, say, working with a general tree data structure, like XML or an HTML DOM? That's something that comes up all the time in many fields. Why the obsession with low-level pointer juggling?
I'm more than happy to participate in a technical interview and I always prepare the best I can. However, I find it odd that technical interviews don't work both ways. The interviewer can ask you to solve problem x, but if you ask the interviewer to solve problem y (fair play, right?) I suspect you would be shown the door.
Interviewer: "Can you show me on the whiteboard how you would invert a binary tree?"
You: "Of course." Writes on whiteboard.
Interviewer: "Excellent!"
You: "Your turn. Can you show me on the whiteboard how you would implement a priority queue first with O(1) insert and O(n) remove complexity, and then with O(n log n) insert and remove complexity."
I think even better would be to solve a problem on the whiteboard WITH the interviewer: one which the interviewer did not know or prepare an answer to.
- First, it would reflect the type of problems you may have to solve in the role you're interviewing for.
- Second, it demonstrates your interpersonal skills, your ability to de-construct, understand and then solve the problem at hand.
- Third, it demonstrates how you work with others to solve a technical problem.
- Finally, it puts both the interviewer and interviewee on the same level. It's not so much of "you v.s. me", but a "we". Hopefully, by the end the interviewer thinks: "Hey, I'd really like to work with this person. They're really smart, solved the problem faster than I could, they were very easy to work with, etc."
I think this would make technical interviews more fair, more fun, and at last, representative of real work you would do in the role as I doubt the interviewer will want to solve the 8 queens problem under pressure either.
The rules are pretty clear on what the interview will cover. The recruiter tells you months before the interview and they (Google) even send you a list of what books and papers to read before you consider yourself ready. That is more than fair. I just joined a start up. It consists of 3 non-minorities and 3 minorities. All three non-minorities were recruited early on because they all knew someone within the company (and they all were competent). All three minorities came in through a recruiter - expensive and the bar definitely was a little higher than the internal referral-hire process.
So In summary, I like Google's process for its fairness. They also seem to hire quite a few good engineers and a few great ones too.
I had an interview (and got rejected) at Google in 2012. Nobody sent me a list of papers and books to read.
I applied for a software engineer position. I was rejected when I couldn't answer system administration questions (the guy asking the questions was a system administrator at Google)
> The recruiter tells you months before the interview and they (Google) even send you a list of what books and papers to read before you consider yourself ready.
Right, because I have nothing better do in those months than to rehash CS 101 and waste my time inverting trees and reversing strings.
I'm only a rising junior in CS on my second internship and I completely agree. I did really well in my Data Structs & Algorithms class (A+...) and I just hate these types of problems. I bombed Google and Dropbox's interviews, and am ready to never go through another process like that again if I can.
I've been exceedingly lucky landing gigs at great companies that don't filter with these questions. I worked for LeadGenius (YC S11!) and the technical portion of the interview was super fun and effective; I was asked to write a useful piece of software in python which I later applied to school projects! No whiteboarding, no data structures. I'm currently interning at General Motors and their interview was similarly sans whiteboarding, and I'm surrounded by brilliant interns and coworkers. Hmm. At least for me and a few of my peers, being asked to solve the type of problems presented in the article raises some red flags about a company's culture.
Fact is, it just isn't true. As a Googler, I am surrounded by enormously talented engineers, the likes of which I rarely encountered on the outside. They love what they do, and it shows.
I have no doubt that there are false negatives/positives, but I am convinced that Google is doing something right.
The OPower guy said they had a ton of problems where they
will be using Scalding, so I asked him what they are
doing in its absence. He said Oh we pojo it. Then he said
pojo this and pojo that, and soon I was drowning in
pojos, so I asked, Sorry, what exactly is a pojo ? Now,
bear in mind I am a Scala programmer and haven't touched
Java in ages, and they knew that. Their whole pitch was
they wanted to inject some new Scala blood into their
tired Java veins, and that's why I interviewed there. So
the guy is agape, and says, you don't know what a pojo is?
When was the last time you wrote Java?
When interviewing a candidate, use some wacky made-up term like "pojo" a lot in offhand conversation, and reject anyone who doesn't ask what it is.
[speaking for myself only, not my employers past or present]
[I am guessing this will be an unpopular opinion, so posting anonymously]
I feel that interviews are essential. Every alternative I've heard seems to not work. I've tried.
0. Review their contributions to open source
--a. Some brilliant people have none - they prefer to get paid for their work & not disseminate it openly for free - that does not mean they should be disqualified
--b. I've personally seen people with stellar resumes claiming to have contributed to many projects who could not even tell me when they'd use a hashtable in any scenario of their choice
1. Ask about their last project
--a. It is easy to lie, to any level of detail. Anyone who tells you otherwise has never watched a politician speak.
--b. Speaking about engineering (even well) does not mean you can actually engineer well.
2. Pair program with them
--a. This tends to waste a lot of time, since either you use an abstract problem (and you're back to a normal interview) or you use a real problem and you spend 10000 hours explaining it.
3. Hire them on a trial period
--a. This is insulting to the brilliant people
--b. This is a waste of company time on the not brilliant people
Trial period: It's not just "insulting" - I wouldn't do it. If I have a job I'm very satisfied with and you dragged me to your place in order to interview after contacting me. I won't be willing to spend a week in a trial.
This is a sellers market anyway.
What we found works is asking practical questions and not theoretical ones. You can ask people to code with a computer and internet and whatever editor they want simulating a real environment.
Thanks for going ahead and expressing what you think may be an unpopular point of view. Your points are fair, and are a bit part of why I feel so ambivalent about technical interviews. While I don't really like how they're handled, I think they may be necessary, and they may be an unfortunate byproduct of a good thing.
For instance, I majored in math, not CS. I took some basic computer science, but much of my programming was done in math classes, sometimes informally, sometimes as part of the course itself, and sometimes as part of those optional 1-2 unit pass/fail labs.
When I first opened a data structures and algorithms book, I was slightly amused by what I'd already covered and what I hadn't. For instance, I'd written DFS and BFS code for my graph theory class in college, and I'd done a lot of the linear programming at the end of the algorithms book. Various numerical programming exercises had touched on a lotos other things, recursion, lists, and so forth. But there were gaps, and I've studied a lot on my own for them.
Now, in most fields that wouldn't be possible. You can't study something that overlaps quite a bit with law, nursing, or medicine, and then fill in the gaps on your own, demonstrate knowledge to an employer's satisfaction, and go off to work as a lawyer, nurse or physician. I think this is a pretty wonderful thing about software development, and things like the google "entrance exams" are actually part of why it is possible.
I didn't get an offer from google, and reading these threads, I'm realizing that I should have taken months, not weeks, to review. However, I didn't encounter any of the arrogance or hostility, I found that the interviewers did a good job keeping things friendly and collaborative.
That said, it's stressful. People in other fields are often kind of astounded to hear what we go through. You have nowhere to hide, you are at the whiteboard, with a pen, getting grilled technically, and you only have 45 minutes to solve the problem. You fear looking like an idiot, even if the interviewer tries to be positive. You may feel like an idiot, even if the interviewer had no intention of making you feel this way and made an effort to make sure you didn't. And of course, there are plenty of interviewers who may actually kind of enjoy making someone squirm a bit.
So while I may have a somewhat more positive view of the tech "exam" than a lot of people here, I still think something is broken here. I've discussed this a few times with people here on HN, and I honestly do think that three big interviews may be roughly equivalent to taking the bar exam. Seriously. I could have studied for several months (though like I said, I have bigger gaps to plug than a CS major probably would). I read a blog post about a guy who passed the California bar with 100 hours of study. People talk about the "grueling" three days of exams. Well, interview at google, amazon, and microsoft. Between phone interviews, in person interviews, and so forth, sure it can get to that level. Even if it falls short of "bar exam" level effort, the fact that we're starting to talk about it reasonably in the same breath shows just how much effort goes into it.
And here's the problem - it's just an interview. You get no feedback, no credential, nothing. You might do well enough to "pass" under a reasonable set of scoring criteria, but all you get is a "no hire" with very minimal information about how you did.
I've heard people say (again here on HN) that they'd happily take a bar exam for software if it meant that they could be done, for once and for all, with the never-ending series of technical exams that we have to do over and over. I'd be ok with studying three months, six, hell, even a year. Because the exam would be consistent, rigorous, and would provide me with a lasting credential. I'm just not willing to put this sort of effort into a single interview that may be decided very capriciously.
So, all in all, I see merit in these difficult technical exams (first step, let's stop calling them interviews). I think the whole thing could be handled vastly better, though.
IMHO, I feel like people who are not up to it don't agree with coding interviews.
Also, in my experience, the employers that didn't check coding skills at the interview had the worst engineers I met (surprise!).
Now I can see why it's easy to disagree with interviews that check your algorithms 101 knowledge: that stuff is really really rarely used in real-life and you can just google it if you ever need it. But! Keep these in mind:
- Can you come up with a better process that scales with the number of interviewers in your company, but also maintains reasonable consistency and keeps reasonable costs? Maybe you think you can, but keep in mind that the tech giants have data-crunched their interview stats over and over and this process is what they stuck with. (Of course, with scale there's also the problem you occasionally have arrogant interviewers - but I think that problem should be decoupled from the coding/non-coding interviews problem).
- In places where they look for A* engineers, the point of the interview is often not to test what you know best, but how you get along with problems you have never seen before. An algorithm or data structure question often fits the bill.
- Geeks love geeky puzzles (like inverting binary trees). Companies often look for geeks in love with abstract stuff.
Also, IMHO, knowing only one language is a red flag for me too at 10+ years experience level.
I love interviewing and being interviewed. Don't know why but I do. However, I hate these types of interviews. I mean, sure if I am a new grad or you suspect I might not know my shit, ask me some probing questions. However, almost nobody these days needs to invert a binary tree or compute the hash of a string by hand.
Here is my secret go to question when interviewing someone: "can you describe how an AJAX request works, from start to finish?". The answer involves knowing that AJAX works over HTTP, same as regular page requests, knowing what IP, TCP, and HTTP are, and how client and server interact using them. It is a question most competent people can answer, yet it has lots of room for depth of detail. By discussing technical subjects, challenges and solutions I think you get a much better idea of how good someone is vs some predefined set of puzzles.
I enjoy the challenge of programming interviews. When I interviewed in Mountain View, I thought it was just super cool to have been invited. It was like the mother ship had summoned me home!
I was mega-underwhelmed when I botched the last session, though. And after doing so poorly, no one escorted me out or summed things up. Efficient, I suppose -- I'd met with the recruiter at the beginning of the day and there was nothing left to discuss. But there was no denouement. A big build up all day and all of a sudden -- nothing.
That last session I was asked to implement a particular matrix manipulation. IMO a bit more utility than tree inversion. And I tried to talk through my thought process but I'll admit it just wasn't getting there. After it was over, I went back to the hotel, wrote a test case and kept coding until I made it work. I'm a little ashamed to admit that it took 1.5-2hrs to get it to work -- so probably not just intimidation/stage fright. After writing the code, I don't think I understood the design well enough to explain it -- but I did do it on my own w/o any reference. I guess I wasn't surprised when I didn't get an offer.
So I've been contacted again as a part of this latest hiring campaign. I'll give it another try, I suppose. My ego's buoyed by being contacted again and doing well on phone screens, so I suppose I can take another bruising. :)
I don't mean to personalize this to you, but your story kinda illustrates one of the problems. There's so much ego and self-worth tied up in both sides of this style of "interview". Both sides have this overwhelming need to show how smart they are.
It's understandable in some ways. It's part of what attracts people to the field, showing off intelligence. I'm guilty of it myself.
But it makes the "interview" process have an adversarial nature most of the time. That's not helpful to the actual goal of hiring people who will make your team successful.
One thing that might work along these lines- Bring in a hard problem that the interviewer doesn't know how to solve and spend some time trying to solve it with the candidate. As a team, discussing different options and problems with those options. You know, like people do in actual work situations...
>You are then ushered into a room with the programmer's worst nightmare - a blank whiteboard.
Am I the only one who enjoys programming interviews? Even if you screw it up, it's still fun to try your hand at whatever problem they give you. They also don't expect you to do it perfectly; you're allowed to have sub-superhuman performance.
I interviewed at Microsoft and it was about 10 hours of this stuff and I loved it. Had a great time, and I think I did pretty well. One guy wanted me to write a shuffling program. So I came up with one which I later discovered was Fisher-Yates and optimal. But he kept saying "how can it be faster" so I'm not sure what I was missing. Most of the other stuff was pretty fun and straightforward - pointer stuff, string copying, synchronization. It felt like if you had gone through Sedgewick's algorithm book you'd be set for the most part.
Unfortunately by 7pm or so I was sorta exhausted (didn't sleep the night before 'cause I was so excited). I met the hiring manager, and he asked something fun like make a ring buffer and also write a non-recursive inorder traversal function of a binary tree. I apparently had some off-by-one bug in the ring buffer code, and then I just blanked out on the traversal question. No hire :\.
(OTOH, that team wasn't sure if they were going to be using VB6, or C++, or maybe .NET, but probably not because they wanted to ship with Office and Office severely limits dependencies. So maybe it was for the better.)
I believe that was the point - are you expected to do perfectly or not. There are different interviewers - some even just want to show off, probably like the strong hashing one.
It's great if you can have a reasonable discussion that actually shows your knowledge.
I think the idea of solving a whiteboard problem can be fun, providing it's unique. Quite a few times while looking for an internship I was asked mostly standard problems out of "Cracking the Coding Interview" or similar. Under pressure, even problems I've practiced or just understood from a contextual standpoint can be tough. Personally I have issues with the usual situation of it - interviewer staring, which is intimidating, and/or not even trying to have a mutual conversation and bounce ideas around to ease some of the pressure.
Most posts complaining about interview process are being made by people who failed those interviews. There is a ton of reverse-survivorship bias here. It's certainly not the case that companies like GOOG/FB/MS are filled with all bozzos. If anything their interview process have been rather successful considering above average quality of talent at these companies and non-trivial products they work on.
I also sense lot of "entitlement" in OP's post and comments on this thread. It's like "oh I can't answer your interview questions but I'm so good that if I don't get the job, it would be only because your interview process sucks". Most company's HR would let you know what to expect at these interviews. If you are not comfortable with CS whiteboarding questions then you should just decline at that point. It's unprofessional to blame their process after you accepted to go through it, failed and then shit all over it because you didn't get the job.
This is not to say all interviewers are good and many could be downright assholes. But that feedback should be between you and their HR. It's just professional courtesy considering that these companies don't post on Internet how badly you performed on interviews compared to their other candidates to tell other side of your story.
Personally I like solving computational challenges whether I get job or not. As a programmers we are supposed to be loving these kind of CS puzzles. If nothing else, you walk out with few CS things you didn't knew before which would have taken same amount of time to learn anyway. I'm not saying interview questions shouldn't be job related but the fact is that many of these companies are doing LOTs of things and they need to hire more generally because they give you relatively more freedom to move around once you are in. So large companies have to keep things general at some level unlike startups with one project. In any case, if you complain about having to write code and design algorithms at developer interviews then you are probably applying for the wrong job.
Funny, I thought this was going to be "inverting binary trees considered harmful" because the sensible way to do it is to change your traversal algorithm to be right-before-left, and leave the data alone. Silly me!
Yeah I don't think OP was defending Twitter I think it was just more of a story about how crappy the interview process is at most tech companies.
My experience with Twitter was really odd. I knew someone internal and had them directly submit my resume with a recommendation. The resume was tailored for exactly the position they wanted showing me besting all of their requirements. I got an email 4 months later telling me my resume was not good enough for even a phone screening. I've never had that happen before and it's still perplexing.
There's not so much a shortage of talented programmers as there is a shortage of people able to identify talented programmers.
If there's anything in need of serious disruption, it's the adversarial hiring process in technology. It's all mindless cargo-culting, and it doesn't actually work.
I don't think inverting binary trees is the problem per se. I mean, algorithms are the bread and butter of programming. You should be able to implement simple ones like that even if you'd never actually do that in practice. The real problem in my opinion is the whiteboard and time pressure.
I gave this example on Reddit this morning, but I'll repeat it. If a programmer was given an array of elements [a1, a2, a3, a4, ..., an-1, an] and asked to produce the array [a2, a1, a4, a3, ..., an, an-1] and couldn't do it, he'd be the laughing stock of /r/programming for a couple of days and we'd hear about this problem for years to come, the same way we hear about FizzBuzz. Certainly a programmer should know how to traverse an array and manipulate its elements!
For the case of inverting a binary tree, we haven't gotten a clear answer to what it means, but the concensus seems to be to swap every left and right nodes in the tree. Well, that's exactly like the array problem above: traverse the structure and manipulate its elements! The traversal is not a simple loop, you use recursion (or if you're fancier, use an explicit stack to avoid causing an overflow of the call stack), but I'd argue that recursion is a fundamental notion for a programmer to know.
A commenter on Reddit mentioned that there's a difference between programmers and computer scientists and that the programmer is responsible for writing software and the computer scientists are responsible for coming up with algorithms and data structures. But then, what do the programmers use? If we cannot expect them to know about trees, what about linked lists? Resizable arrays? Hash tables? Graphs? These data structures are extremely important tools for creating solutions, shouldn't programmers be aware of them and how to use them?
A lot of these posts don't actually address the problem of hiring and filtering people. If you think whiteboard tests or logic puzzles are not effective, why do you think this and what is a better alternative?
I personally think they are ok, but should more be a means to test if the candidate can reason intelligently and it shouldn't matter much if the end answer they get is correct or not.
[+] [-] rdtsc|10 years ago|reply
So there we were, building a tree out of a forest of existing smaller trees or such. That was after they couldn't find my resume, and used a 5 year old one they found some place. They seemed annoyed and tired. By that time I felt nothing short of me proving P!=NP would have help changed their mood. I mean, they didn't even ask me what I knew, what I worked on lately. Heck, I could have been herding goats for the previous 5 years, but if I could have solved that tree problem, well, Googleplex here I come, I could have been herding goats there, maybe as part of some a green eco-initiative project...
I guess half way through I just kind of decided that Google must be a pretty bad place to work at and kind of gave up. I was tempted to ask them how often they had to build the tree out of subtrees at Google. Or in this case how many times did they really have to invert the binary tree, but I am nicer that than, maybe they really did have a pretty shitty day.
Ever since then I've declined to interview at Google. I get contacted every year like clockwork "Wanna try again?", "No thanks, good luck inverting binary trees though..."
[+] [-] kozukumi|10 years ago|reply
I had to solve problems that I had not looked at since my masters and were mostly theoretical CS things which nobody outside of academia ever has to actually use.
I had a brief read through of their coding style a few days before so I solved it in a very Google way which is what most likely got me the job. Even though doing it their way was just awful (in terms of implementation).
There is no doubt that Google's search algorithms are the best but I can't help but think the rest of their services are only high performing because they can through their enormous data centre resources at the problem.
I also hated the attitude of the interviewers and engineers I met. They were so up their own asses. Their way was the best possible way and if you didn't agree well fuck you!
I have only turned down two jobs in my career that I 100% do not regret and Google is one of them.
[+] [-] buyx|10 years ago|reply
The difference was that I was able and to try various approaches out without pressure. I had access to my old algorithms textbook as a safety blanket. Some friends dropped by and we went out for supper, and I chewed over my ideas with them.
I also had two weeks to solve the problem, and even though it took one Friday night to solve the problem, being given 2 weeks to produce a working solution put me at ease.
Doing something like that in a high-stakes interview would be nearly impossible for me.
[+] [-] melling|10 years ago|reply
[+] [-] desdiv|10 years ago|reply
Truth is stranger than fiction: http://googleblog.blogspot.sg/2009/05/mowing-with-goats.html
[+] [-] _pmf_|10 years ago|reply
Well, searching is not really Google's core competency.
[+] [-] meshko|10 years ago|reply
[+] [-] strathmeyer|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] Osiris|10 years ago|reply
After 3 hours, I was taken to a conference room where I demonstrated the API. I also talked about a bug I had that had hung me up, but I fixed it right there in the conference room.
Got the offer. It was a great experience. I enjoyed the challenge, but it felt useful and related to the work rather than just esoteric problem that would not likely ever be required by the job, and if it was I'd have time to research it and figure it out.
[+] [-] BinaryIdiot|10 years ago|reply
[+] [-] mcv|10 years ago|reply
Hearing someone talk about their own code and explain their approach and the up and down sides of it, is a lot more valuable than simply having the correct answer roll out.
[+] [-] TylerE|10 years ago|reply
The interview process took about two weeks and on the whole was pretty reasonable for both sides.
There was a short phone screen (~30 minutes)
Then I had two technical exercises to do. For each, I was given a reasonable time period (4 hours) that started when I visited a special link to the get the problem description, and then used whatever tools I wanted to get it done, then email in the completed exercise.
The two exercises were not completely trivial, but were relatively straightforward and actually involved skills relevant to the position and not arcane puzzles or comp-sci theory. (Basically, the first was to write a Postgres schema from scratch (total 5 or 6 tables and maybe 30 columns across all the tables). The second task was to write a CSV parser (including reasonable error checking/recovery) to load data into the schema from the first problem.
Finally, there was the main interview (conducted via Facetime, this is a remote position). That ran for close to 3 hours.
This worked well, since it gave me a feel for what the work would actually be like, and I was given, as the candidate, every oppurtunity to put my best foot forward - no whiteboard exercises, no surprises.
[+] [-] zbyte64|10 years ago|reply
[+] [-] unclebucknasty|10 years ago|reply
This is a much more realistic way of testing someone's skills. Asking someone to write code while they watch is not.
I am personally horrible at coding while someone is looking over my shoulder. I am just too preoccupied with their presence and the fact that they are watching. And, unless a company is still into pair programming (is anyone these days?), it's not a valid test.
Give me a real problem, reasonable time to solve it, and the tools I'd have in the real world. Then, I can show you what I can do in that same real world vs. how well I interview in some contrived format.
[+] [-] orangecat|10 years ago|reply
I will agree that the guy who kicked all this off has a legitimate complaint if the recruiter failed to adequately explain the interview process, and I'm sure there are lots of bad interviewers that focus on meaningless details or obscure trivia. But I don't agree that the concept of having candidates write code is fundamentally awful.
[+] [-] rtpg|10 years ago|reply
Turns out a lot of people in our industry don't! I have a Computer Engineering degree, and know this stuff mainly due to this being my hobby (and having tried out for the IOI in high school). But there are former classmates who are only vaguely aware of these data structures, because they never got any exposure to them despite writing real programs.
I don't think there's a super strong argument that most computer engineers need to know much of anything about base data structures. There are so many other concepts that are more important to writing and deploying useful software (especially in the enterprise world) that at no point involve writing complex data structures. Data structures (or rather, knowledge of the internals) are much more domain specific than some of us would like to admit, especially in the age of RoR and Unity. There are many more design patters more important to know about than being able to implement quicksort (it took 6 years for researchers to write the first bug-free implementation!)
Google does it because they can. They're probably filtering out a lot of people who treat code as prose rather than a technical manual though. Which is probably why Google libraries look the way they look.
The biggest surprise for me is how many game programmers don't know about this stuff. It seems like the sort of stuff you'd run into pretty quickly
[+] [-] TylerE|10 years ago|reply
Why not ask questions about, say, working with a general tree data structure, like XML or an HTML DOM? That's something that comes up all the time in many fields. Why the obsession with low-level pointer juggling?
[+] [-] discardorama|10 years ago|reply
The fact that so many people in the previous discussion could not even tell what that means is a very bad sign.
[+] [-] bliti|10 years ago|reply
[+] [-] georgerobinson|10 years ago|reply
Interviewer: "Can you show me on the whiteboard how you would invert a binary tree?" You: "Of course." Writes on whiteboard. Interviewer: "Excellent!" You: "Your turn. Can you show me on the whiteboard how you would implement a priority queue first with O(1) insert and O(n) remove complexity, and then with O(n log n) insert and remove complexity."
I think even better would be to solve a problem on the whiteboard WITH the interviewer: one which the interviewer did not know or prepare an answer to.
- First, it would reflect the type of problems you may have to solve in the role you're interviewing for.
- Second, it demonstrates your interpersonal skills, your ability to de-construct, understand and then solve the problem at hand.
- Third, it demonstrates how you work with others to solve a technical problem.
- Finally, it puts both the interviewer and interviewee on the same level. It's not so much of "you v.s. me", but a "we". Hopefully, by the end the interviewer thinks: "Hey, I'd really like to work with this person. They're really smart, solved the problem faster than I could, they were very easy to work with, etc."
I think this would make technical interviews more fair, more fun, and at last, representative of real work you would do in the role as I doubt the interviewer will want to solve the 8 queens problem under pressure either.
[+] [-] sriram_sun|10 years ago|reply
So In summary, I like Google's process for its fairness. They also seem to hire quite a few good engineers and a few great ones too.
[+] [-] kkapelon|10 years ago|reply
I applied for a software engineer position. I was rejected when I couldn't answer system administration questions (the guy asking the questions was a system administrator at Google)
[+] [-] discardorama|10 years ago|reply
Right, because I have nothing better do in those months than to rehash CS 101 and waste my time inverting trees and reversing strings.
[+] [-] DanFeldman|10 years ago|reply
I've been exceedingly lucky landing gigs at great companies that don't filter with these questions. I worked for LeadGenius (YC S11!) and the technical portion of the interview was super fun and effective; I was asked to write a useful piece of software in python which I later applied to school projects! No whiteboarding, no data structures. I'm currently interning at General Motors and their interview was similarly sans whiteboarding, and I'm surrounded by brilliant interns and coworkers. Hmm. At least for me and a few of my peers, being asked to solve the type of problems presented in the article raises some red flags about a company's culture.
[+] [-] scotth|10 years ago|reply
I have no doubt that there are false negatives/positives, but I am convinced that Google is doing something right.
[+] [-] CamperBob2|10 years ago|reply
[+] [-] dasdasd|10 years ago|reply
I feel that interviews are essential. Every alternative I've heard seems to not work. I've tried.
0. Review their contributions to open source
--a. Some brilliant people have none - they prefer to get paid for their work & not disseminate it openly for free - that does not mean they should be disqualified
--b. I've personally seen people with stellar resumes claiming to have contributed to many projects who could not even tell me when they'd use a hashtable in any scenario of their choice
1. Ask about their last project
--a. It is easy to lie, to any level of detail. Anyone who tells you otherwise has never watched a politician speak.
--b. Speaking about engineering (even well) does not mean you can actually engineer well.
2. Pair program with them
--a. This tends to waste a lot of time, since either you use an abstract problem (and you're back to a normal interview) or you use a real problem and you spend 10000 hours explaining it.
3. Hire them on a trial period
--a. This is insulting to the brilliant people
--b. This is a waste of company time on the not brilliant people
[+] [-] inglor|10 years ago|reply
This is a sellers market anyway.
What we found works is asking practical questions and not theoretical ones. You can ask people to code with a computer and internet and whatever editor they want simulating a real environment.
[+] [-] geebee|10 years ago|reply
For instance, I majored in math, not CS. I took some basic computer science, but much of my programming was done in math classes, sometimes informally, sometimes as part of the course itself, and sometimes as part of those optional 1-2 unit pass/fail labs.
When I first opened a data structures and algorithms book, I was slightly amused by what I'd already covered and what I hadn't. For instance, I'd written DFS and BFS code for my graph theory class in college, and I'd done a lot of the linear programming at the end of the algorithms book. Various numerical programming exercises had touched on a lotos other things, recursion, lists, and so forth. But there were gaps, and I've studied a lot on my own for them.
Now, in most fields that wouldn't be possible. You can't study something that overlaps quite a bit with law, nursing, or medicine, and then fill in the gaps on your own, demonstrate knowledge to an employer's satisfaction, and go off to work as a lawyer, nurse or physician. I think this is a pretty wonderful thing about software development, and things like the google "entrance exams" are actually part of why it is possible.
I didn't get an offer from google, and reading these threads, I'm realizing that I should have taken months, not weeks, to review. However, I didn't encounter any of the arrogance or hostility, I found that the interviewers did a good job keeping things friendly and collaborative.
That said, it's stressful. People in other fields are often kind of astounded to hear what we go through. You have nowhere to hide, you are at the whiteboard, with a pen, getting grilled technically, and you only have 45 minutes to solve the problem. You fear looking like an idiot, even if the interviewer tries to be positive. You may feel like an idiot, even if the interviewer had no intention of making you feel this way and made an effort to make sure you didn't. And of course, there are plenty of interviewers who may actually kind of enjoy making someone squirm a bit.
So while I may have a somewhat more positive view of the tech "exam" than a lot of people here, I still think something is broken here. I've discussed this a few times with people here on HN, and I honestly do think that three big interviews may be roughly equivalent to taking the bar exam. Seriously. I could have studied for several months (though like I said, I have bigger gaps to plug than a CS major probably would). I read a blog post about a guy who passed the California bar with 100 hours of study. People talk about the "grueling" three days of exams. Well, interview at google, amazon, and microsoft. Between phone interviews, in person interviews, and so forth, sure it can get to that level. Even if it falls short of "bar exam" level effort, the fact that we're starting to talk about it reasonably in the same breath shows just how much effort goes into it.
And here's the problem - it's just an interview. You get no feedback, no credential, nothing. You might do well enough to "pass" under a reasonable set of scoring criteria, but all you get is a "no hire" with very minimal information about how you did.
I've heard people say (again here on HN) that they'd happily take a bar exam for software if it meant that they could be done, for once and for all, with the never-ending series of technical exams that we have to do over and over. I'd be ok with studying three months, six, hell, even a year. Because the exam would be consistent, rigorous, and would provide me with a lasting credential. I'm just not willing to put this sort of effort into a single interview that may be decided very capriciously.
So, all in all, I see merit in these difficult technical exams (first step, let's stop calling them interviews). I think the whole thing could be handled vastly better, though.
[+] [-] vladaionescu|10 years ago|reply
Now I can see why it's easy to disagree with interviews that check your algorithms 101 knowledge: that stuff is really really rarely used in real-life and you can just google it if you ever need it. But! Keep these in mind:
- Can you come up with a better process that scales with the number of interviewers in your company, but also maintains reasonable consistency and keeps reasonable costs? Maybe you think you can, but keep in mind that the tech giants have data-crunched their interview stats over and over and this process is what they stuck with. (Of course, with scale there's also the problem you occasionally have arrogant interviewers - but I think that problem should be decoupled from the coding/non-coding interviews problem).
- In places where they look for A* engineers, the point of the interview is often not to test what you know best, but how you get along with problems you have never seen before. An algorithm or data structure question often fits the bill.
- Geeks love geeky puzzles (like inverting binary trees). Companies often look for geeks in love with abstract stuff.
Also, IMHO, knowing only one language is a red flag for me too at 10+ years experience level.
[+] [-] IgorPartola|10 years ago|reply
Here is my secret go to question when interviewing someone: "can you describe how an AJAX request works, from start to finish?". The answer involves knowing that AJAX works over HTTP, same as regular page requests, knowing what IP, TCP, and HTTP are, and how client and server interact using them. It is a question most competent people can answer, yet it has lots of room for depth of detail. By discussing technical subjects, challenges and solutions I think you get a much better idea of how good someone is vs some predefined set of puzzles.
[+] [-] wyldfire|10 years ago|reply
I was mega-underwhelmed when I botched the last session, though. And after doing so poorly, no one escorted me out or summed things up. Efficient, I suppose -- I'd met with the recruiter at the beginning of the day and there was nothing left to discuss. But there was no denouement. A big build up all day and all of a sudden -- nothing.
That last session I was asked to implement a particular matrix manipulation. IMO a bit more utility than tree inversion. And I tried to talk through my thought process but I'll admit it just wasn't getting there. After it was over, I went back to the hotel, wrote a test case and kept coding until I made it work. I'm a little ashamed to admit that it took 1.5-2hrs to get it to work -- so probably not just intimidation/stage fright. After writing the code, I don't think I understood the design well enough to explain it -- but I did do it on my own w/o any reference. I guess I wasn't surprised when I didn't get an offer.
So I've been contacted again as a part of this latest hiring campaign. I'll give it another try, I suppose. My ego's buoyed by being contacted again and doing well on phone screens, so I suppose I can take another bruising. :)
[+] [-] waterlesscloud|10 years ago|reply
It's understandable in some ways. It's part of what attracts people to the field, showing off intelligence. I'm guilty of it myself.
But it makes the "interview" process have an adversarial nature most of the time. That's not helpful to the actual goal of hiring people who will make your team successful.
One thing that might work along these lines- Bring in a hard problem that the interviewer doesn't know how to solve and spend some time trying to solve it with the candidate. As a team, discussing different options and problems with those options. You know, like people do in actual work situations...
[+] [-] wyager|10 years ago|reply
Am I the only one who enjoys programming interviews? Even if you screw it up, it's still fun to try your hand at whatever problem they give you. They also don't expect you to do it perfectly; you're allowed to have sub-superhuman performance.
[+] [-] adamkochanowicz|10 years ago|reply
[+] [-] MichaelGG|10 years ago|reply
Unfortunately by 7pm or so I was sorta exhausted (didn't sleep the night before 'cause I was so excited). I met the hiring manager, and he asked something fun like make a ring buffer and also write a non-recursive inorder traversal function of a binary tree. I apparently had some off-by-one bug in the ring buffer code, and then I just blanked out on the traversal question. No hire :\.
(OTOH, that team wasn't sure if they were going to be using VB6, or C++, or maybe .NET, but probably not because they wanted to ship with Office and Office severely limits dependencies. So maybe it was for the better.)
[+] [-] viraptor|10 years ago|reply
It's great if you can have a reasonable discussion that actually shows your knowledge.
[+] [-] spike021|10 years ago|reply
[+] [-] mrcsparker|10 years ago|reply
Part of the interview process is trying to find out what type of person you are dealing with.
[+] [-] sytelus|10 years ago|reply
I also sense lot of "entitlement" in OP's post and comments on this thread. It's like "oh I can't answer your interview questions but I'm so good that if I don't get the job, it would be only because your interview process sucks". Most company's HR would let you know what to expect at these interviews. If you are not comfortable with CS whiteboarding questions then you should just decline at that point. It's unprofessional to blame their process after you accepted to go through it, failed and then shit all over it because you didn't get the job.
This is not to say all interviewers are good and many could be downright assholes. But that feedback should be between you and their HR. It's just professional courtesy considering that these companies don't post on Internet how badly you performed on interviews compared to their other candidates to tell other side of your story.
Personally I like solving computational challenges whether I get job or not. As a programmers we are supposed to be loving these kind of CS puzzles. If nothing else, you walk out with few CS things you didn't knew before which would have taken same amount of time to learn anyway. I'm not saying interview questions shouldn't be job related but the fact is that many of these companies are doing LOTs of things and they need to hire more generally because they give you relatively more freedom to move around once you are in. So large companies have to keep things general at some level unlike startups with one project. In any case, if you complain about having to write code and design algorithms at developer interviews then you are probably applying for the wrong job.
[+] [-] unoti|10 years ago|reply
[+] [-] swang|10 years ago|reply
Sorry my op sounded like I was accusing the author of something but this whiteboard approach is what I did at Twitter.
[+] [-] BinaryIdiot|10 years ago|reply
My experience with Twitter was really odd. I knew someone internal and had them directly submit my resume with a recommendation. The resume was tailored for exactly the position they wanted showing me besting all of their requirements. I got an email 4 months later telling me my resume was not good enough for even a phone screening. I've never had that happen before and it's still perplexing.
[+] [-] ww520|10 years ago|reply
It's a good advice to go out to interviews from time to time. Just to see what's out there.
[+] [-] waterlesscloud|10 years ago|reply
If there's anything in need of serious disruption, it's the adversarial hiring process in technology. It's all mindless cargo-culting, and it doesn't actually work.
[+] [-] jheriko|10 years ago|reply
amen.
[+] [-] rayiner|10 years ago|reply
[+] [-] gnuvince|10 years ago|reply
For the case of inverting a binary tree, we haven't gotten a clear answer to what it means, but the concensus seems to be to swap every left and right nodes in the tree. Well, that's exactly like the array problem above: traverse the structure and manipulate its elements! The traversal is not a simple loop, you use recursion (or if you're fancier, use an explicit stack to avoid causing an overflow of the call stack), but I'd argue that recursion is a fundamental notion for a programmer to know.
A commenter on Reddit mentioned that there's a difference between programmers and computer scientists and that the programmer is responsible for writing software and the computer scientists are responsible for coming up with algorithms and data structures. But then, what do the programmers use? If we cannot expect them to know about trees, what about linked lists? Resizable arrays? Hash tables? Graphs? These data structures are extremely important tools for creating solutions, shouldn't programmers be aware of them and how to use them?
[+] [-] BobTheCoder|10 years ago|reply
I personally think they are ok, but should more be a means to test if the candidate can reason intelligently and it shouldn't matter much if the end answer they get is correct or not.