The perspective here is subtly baised - look at the diagram down the bottom and realise that they were always only going to hire 1 person, so all the reasons that they give for not hiring the person are, in fact, not reasons that the person filtered out wasn't hired. They are processes to rank the applicants. If there were more candidates they'd add more reasons not to hire most of them, if there were less candidates the reasons not to hire would start to disappear.
In particular, companies are in some sense bluffing with the "Didn't Qualify" category. I've seen hiring situations where nobody qualifies but they actually need to fill the position - they hired someone who didn't qualify and trained them up. They did a great job. "Didn't qualify" is only a real category for the most demanding jobs. Software is just not one of them, nobody has any idea if dev is going to be good or not before they hire them. Companies often have a hard time picking which devs are the productive ones when they've already hired the dev.
So we've got an article about a process used to rank devs, and no particular evidence of whether the dev hired is actually very good. Which is fine, still an interesting read. But it is good to keep a clear perspective. This is one of those situations where doing big parts of the process by fair dice roll is not necessarily an inferior approach.
I think the article was more about the gates in the applicant 'funnel' rather than describing their method of ranking applicants as suitable developers.
The best dev for the job may have been 'unqualified' given that they were looking for "a generalist who can do both Unity and services coding."
this is a very keen observation about most hiring processes. even places that make noises about “hiring good people and figuring fit out later” seem to mostly, anecdotally, reduce the process to “hire the person least likely to fail at following the exact job description”.
Yeah I'm reminded of the military on this Veteran's Day. You join and you aren't qualified for anything at that point. But they take a chance, they train you, and most rise to the challenge.
> companies are in some sense bluffing with the "Didn't Qualify" category.
This is to the point that it's not "bluffing" but simply "how the world works." What's unfortunate is that many new grads (and some veterans) live with the impression that they need to meet (or lie about) all these "requirements". When the real world never operated like that.
Still, some job ads are written to show both "essentials" and "nice to have".
Eh, kind of. Right now I'm hiring for a role where nobody remaining in my applicant pool is qualified (all have only some of the experience I need) but I'm probably still going to hire one of them. Does that "qualify" them? No. It just means I'm probably going to hire despite it.
> If we see potential, the first step is always asking the candidate upfront for their expected salary, availability, and whether they want full-time or part-time. Since we are focused on efficiency, we need to respect people’s time as much as our own. Most candidates appreciate it; for example, it immediately filtered out a very qualified candidate whose salary ask was 4 times our budget.
I feel like the "respectful of applicants' time" thing to do would be to state a salary range in the posting.
In fact, for companies of 15 or more employees, this is legally required in job postings in Washington state, where the company from the article is based.
This sounds like an incredibly toxic hiring process and not a company I'd ever want to work for. So you apply for a job and in response they (maybe) email you back asking for your expected salary (great way to filter out anyone worth hiring btw) and if you're cheap enough they then ask you to do work on a take home assignment. Everyone here thinks that this is okay and they want to be interviewed this way?
"Since we are focused on efficiency, we need to respect people’s time as much as our own". How exactly does this process respect the candidates time?
For the game industry this is practically amazing.
Firstly, for 99% of appointments they usually don't care how good of a developer you are. You may have invented 10 new technologies and have revolutionized the field, if you can't show them a portfolio of games you have shipped then they don't care. They don't hire you for a developer/code role because you're a great developer, they hire you because you've shipped some games before (which is totally a different metric). For whatever reason the whole industry is stuck in this mentality, they can't differentiate between the metrics of appointing a great developer vs trying to find someone that can ship titles.
Asking for expected salary is a pretty quick way to filter because no one ever lists their requirement in their cv. If the job listing included their range then they might have gone with just assuming that the applicants would be within that range, but it doesn't hurt to check.
The test itself is quite easy and straightforward, if anything the real gotchas around it would be to stand out significantly from everyone else.
A few paragraphs below, the article answers that the company can't afford to pay as much as some others; I assume if you are being already being paid way over the market rate you should keep working at the place you're at.
As for the take home, I'd take it or any other kind of non-conventional question that allows me to show me my skills, rather than the usual interview where your interviewer gives you an algorithm or system design question they couldn't solve themselves, with the occasional smirk as they watch you fumble through that question.
> Hi, I’d like to work for your company
> Great, state your price and availability
If match:
> create a simple project(<1 hour of work) that demonstrates familiarity with technologies that will be used on the job.
If done, check for team fit.
I assume you have never seen the German software recruitment process. 6+ rounds spread over 3 months (with no ghosting in the middle if you are lucky). Here is the current process -
- Apply online
- Initial screening with recruiter if they like your resume (book a 45 mins slot)
- Take home assignment or online assessment (2 -4 hours)
- First technical screening interview (1-1.5 hours)
- Second technical interview (system level, deep dive, 1.5 hours)
- Product manager interview (1 hour)
- Senior leadership interview (1 hour)
- Final offer
Between all these rounds, you need to book meetings and it usually takes 1-2 weeks between rounds.
This seems to imply that there is a significant causal link between a developer’s salary and the quality and quantity of their output, and I just don’t think that’s true in the general case.
I met this guy who became a game dev later in life, as in his 30s. He has quite the non-traditional resume as far as devs go, with no formal CS education.
In any case, he went on to work on a game, and kept doing so for years. He hired artists etc. but did the core development himself. Released the game, which has now sold over 150k copies since the release last year. Obviously not crazy numbers up in the millions or tens of millions, but impressive for a first release, and from a dev basically learning as he's moving along, with a regular 9-5 job and family - doing it purely for the love of the game.
Makes me wonder if he'd ever have gotten the chance, had he first tried to join some small indie studio, rather than the DIY route.
The thing is - would he have prospered within a small indie studio?
My experience is that a lot of the traits highlighted above make for brilliant innovators and creators, but actually end up being stifled/stopped by leads within a company. Having this kind of vision and passion is brilliant until it collides within what the founders vision is.
> Work with Indies — no formal connection, but the results were great! –. Our listing went live at 11:30 AM on October 15th, and we had to ask them to shut it down by 6:30 PM on October 17th because we had too many candidates. In the end, we collected 159 applications.
Is this normal in Seattle? That’s a tonne of applicants especially on what seems like a niche job site. Are they mostly junk offshore applications or bots?
Interesting that out of that it looks like 90% were late applications or not qualified and only 17 total completed the take home.
There are an incredible amount of CS graduates, far more than the available jobs, so the number of under-employed CS grads only goes up with time. If anything I'd have expected somewhere around 200-250 apps for that time period.
One note that might be good to highlight in the article is that the take-home is expected to be 2 hours long. From my experience, they are much longer so I was initially surprised to see take-home's being given before an initial call until I looked at the assignment itself.
I still consider this a red flag. The company wants me to put time into the hiring process, but they can’t be bothered to do the same.
If there is at least a recruiter screening first, I’ll apply and ask about “Bring Your Own Code Examples”, mostly when their daily work would use tools that I have some code published.
I can see both perspectives. If you are a skilled hiree, this seems like a waste of time. But if you are hiring online, you will inevitably get a lot of terrible candidates and you need to filter them out. If you are a small team, you can't spend weeks interviewing randos with little to none coding experience for a SE role. The problem is that online hiring is full of noise, but both sides suffer from the expenses this creates.
> always asking the candidate upfront for their expected salary [etc - to respect and save everyone time]
Isn't that a dark pattern (in addition to being a time saver for everyone)? It's a damned if you do and damned if you don't kind of thing.
This is an adversarial question in a process which needs to be ruthless against the time-wasting applications, but also needs to be cooperative with future co-workers.
If you are open to a broad range of salary in exchange for perhaps working at a super exciting place, then there is no good answer. Even "I'm open to a broad range of salary if that's needed to work at a super exciting place" is not a great answer.
Leaving it to the legal requirement of posting an expected salary range and negotiating from there might be a better way.
I am impressed with the extent of the effort here. I struggle with the notion of working with just one other person on a game. Building an entire hiring pipeline and documenting it seems like something that would immediately kill the dream for me at this scale.
There is something that feels very cursed to me about a team of size 2~10 for game dev. At this point I'd much rather go solo or join a team of 100+. Zero structure or a lot of structure. A medium amount of structure seems to bring the maximum amount of entropy.
> There is something that feels very cursed to me about a team of size 2~10 for game dev.
Very small teams can be great - so long as you are able to focus on building a game, not building a company. You need to treat it more like a game jam, at least in the early stages of a project. Just make things, see what works, rapidly prototype and iterate.
But you probably all need to be equal partners in the project from the start for it to work well. It's not so good for creative work when there's a management/employees 'them and us' divide from the start.
Having worked on games in giant teams and small teams my own preference is actually on the smaller end. Ten people feels like a comfortable limit. I'm very much a generalist though and love contributing to all the different bits and pieces of a game. As such I generally find larger teams overly bureaucratic, slow to move and stifling. Solo dev requires a lot of mental fortitude because there is no one else to carry the momentum.
If you’ve never run a hiring process, it’s hard to get a feel for just how difficult and time-consuming it is.
And risky - hiring someone wrong for the role is very expensive and disruptive. And yet more likely to happen that you’d think, even with a rigorous selection process.
> You must ask candidates to solve problems directly related to the role. If you’re hiring a game programmer, knowing how to detect fraud in bank transactions is irrelevant knowledge if that task never appears on the job.
I wish more companies understood this. In all my years of interviews I never got a coding interview or take home programming assignment that even remotely resembled the work they needed.
"Unfortunately, this time, we had 46 late applicants we didn’t even look at"
You couldn't even do the INITIAL / quick triage for those 46? You couldn't even just spend 20 seconds on each to see if they were remotely qualified to be considered or not? It would have taken one person 15 minutes. You instead just threw away all those applications without even glancing at them? In addition, these were applications that people got in before the deadline, before you closed the application process, they applied correctly, and you just threw them away without even glancing at them?
The article includes actual numbers of the filtering funnel! Excellent!
"Late applications" though: Wouldn't it be very wasteful though, to dismiss a third of candidates simply because late application when the posting was live just two days? As admitted, the hiring process is time consuming. It seems wasteful then to filter on "available during the 2 day posting window"! That availability was not a serious job requirement.
This aligns closely with the hiring practices I learned in Industrial and Organizational Psychology. The only thing missing is to have structured interviews to reduce interviewer bias.
The best predictors of job performance are a simulation of the job and past performance. This is not new research or a secret.
Curious about "no-AI" policy. From my experience interviewing lately AI is allowed,but the task is usually big in a small time window. So you quickly spin up the project but I see most of work in QA
A friend at a nearly-FAANG said not using AI tooling in the interview is now an automatic fail. “Not that you’d be able to complete the task on time without it anyway”
We’re using AI tools heavily but also ask candidates to code without it in interviews, and I think it’s reasonable.
Even if you are primarily using codegen, your own coding ability, taste, problem solving, etc. are still deciding inputs to the quality of the final result. And it’s much easier to assess these things in an hour when a human is writing and debugging a relatively small amount of code.
AI tools just produce too much code in a short time. It’s hard to assess what the candidate’s quality bar and attention to detail are really like when there’s so much code to wade through. Anyone can vibe code, but not many can do it without creating mountains of tech debt… the ones who can are usually good programmers with or without AI.
Does it also mean no-Googling? Because the first result is usually LLM-generated. Should you skip to the next page and read only Experts Exchange pages?
I wish we stopped playing games during hiring processes. I get you're trying to weed out the bad ones (can't take pressure, not pro-active, can't ask questions, etc.), but the entire process sucks on both ends.
Let's say we care about the potential employee's needs, most people want to feel like they're making a difference at work and build something that matters. I have never had a job interview where I was able to discuss this topic. It's just "are you good enough to work for us?" while the entire company is falling apart in the background.
Do you actually want to improve your company, or are you just looking to share more workload? Because those are two different things. I'm not looking to join your bike-shedding business
"Our take-home lets candidates code." - As a dev, I absolutely hate the practice of such assignments.
Every non-junior dev/coder should already have at least some indicators out there showing how they code - GitHub, a personal site or any other resources.
For juniors or CS graduates there might be bit of a grey zone, but even then, with how widely available web space is nowadays, there’s really no excuse not to have something out there if you are serious about the "love for coding".
So the sentence “we need to respect people’s time as much as our own” seems flawed to me, because you obviously don’t respect the time of the candidates who coded for nothing for you.
To me, that is also a huge red flag when considering a position.
Important should be assessing someone’s theoretical knowledge of software patterns, principles and architectures ..just getting a feel for their nerd level. Seeing how much they actually care about code and details, whether they can really express themselves and if they could communicate a problem clearly.
This (particular) take home assignment looks absolutely fine to me. If you can't do it under 2 hours (maybe I couldn't? I've never used Unity) then you shouldn't _want_ to work there.
Lol you clearly don't have much experience. Most of the best developers have all of their code locked up in corporate repositories and have nothing they can legally share.
Results from outsourcing can vary. You might end-up with totally unmatched 5 candidates and complain that there is no good people on the market. How would asses that the agency did good job (or any job at all)?
IMO the take home assignment is trivial, even for non-game developers with zero Unity experience; maybe you don't need a take home assignment at all. I also think that asking people to "not use AI" when you have absolutely no way of enforcing it is just self-deception.
This is very consuming. If chief tier people start doing this thjs then this will start draining “em more.
Chief operated people should focus and be picky about whom they want to WORK WITH.
roenxi|3 months ago
In particular, companies are in some sense bluffing with the "Didn't Qualify" category. I've seen hiring situations where nobody qualifies but they actually need to fill the position - they hired someone who didn't qualify and trained them up. They did a great job. "Didn't qualify" is only a real category for the most demanding jobs. Software is just not one of them, nobody has any idea if dev is going to be good or not before they hire them. Companies often have a hard time picking which devs are the productive ones when they've already hired the dev.
So we've got an article about a process used to rank devs, and no particular evidence of whether the dev hired is actually very good. Which is fine, still an interesting read. But it is good to keep a clear perspective. This is one of those situations where doing big parts of the process by fair dice roll is not necessarily an inferior approach.
ludicrousdispla|3 months ago
The best dev for the job may have been 'unqualified' given that they were looking for "a generalist who can do both Unity and services coding."
keeganpoppen|3 months ago
gedy|3 months ago
k__|3 months ago
creer|3 months ago
This is to the point that it's not "bluffing" but simply "how the world works." What's unfortunate is that many new grads (and some veterans) live with the impression that they need to meet (or lie about) all these "requirements". When the real world never operated like that.
Still, some job ads are written to show both "essentials" and "nice to have".
staticautomatic|3 months ago
zahlman|3 months ago
I feel like the "respectful of applicants' time" thing to do would be to state a salary range in the posting.
chihuahua|3 months ago
lone-cloud|3 months ago
"Since we are focused on efficiency, we need to respect people’s time as much as our own". How exactly does this process respect the candidates time?
animal531|3 months ago
Firstly, for 99% of appointments they usually don't care how good of a developer you are. You may have invented 10 new technologies and have revolutionized the field, if you can't show them a portfolio of games you have shipped then they don't care. They don't hire you for a developer/code role because you're a great developer, they hire you because you've shipped some games before (which is totally a different metric). For whatever reason the whole industry is stuck in this mentality, they can't differentiate between the metrics of appointing a great developer vs trying to find someone that can ship titles.
Asking for expected salary is a pretty quick way to filter because no one ever lists their requirement in their cv. If the job listing included their range then they might have gone with just assuming that the applicants would be within that range, but it doesn't hurt to check.
The test itself is quite easy and straightforward, if anything the real gotchas around it would be to stand out significantly from everyone else.
supriyo-biswas|3 months ago
As for the take home, I'd take it or any other kind of non-conventional question that allows me to show me my skills, rather than the usual interview where your interviewer gives you an algorithm or system design question they couldn't solve themselves, with the occasional smirk as they watch you fumble through that question.
preciousoo|3 months ago
This is toxic to you???
rockyj|3 months ago
- Apply online
- Initial screening with recruiter if they like your resume (book a 45 mins slot)
- Take home assignment or online assessment (2 -4 hours)
- First technical screening interview (1-1.5 hours)
- Second technical interview (system level, deep dive, 1.5 hours)
- Product manager interview (1 hour)
- Senior leadership interview (1 hour)
- Final offer
Between all these rounds, you need to book meetings and it usually takes 1-2 weeks between rounds.
andoando|3 months ago
yakshaving_jgt|3 months ago
This seems to imply that there is a significant causal link between a developer’s salary and the quality and quantity of their output, and I just don’t think that’s true in the general case.
TrackerFF|3 months ago
In any case, he went on to work on a game, and kept doing so for years. He hired artists etc. but did the core development himself. Released the game, which has now sold over 150k copies since the release last year. Obviously not crazy numbers up in the millions or tens of millions, but impressive for a first release, and from a dev basically learning as he's moving along, with a regular 9-5 job and family - doing it purely for the love of the game.
Makes me wonder if he'd ever have gotten the chance, had he first tried to join some small indie studio, rather than the DIY route.
endymion-light|3 months ago
My experience is that a lot of the traits highlighted above make for brilliant innovators and creators, but actually end up being stifled/stopped by leads within a company. Having this kind of vision and passion is brilliant until it collides within what the founders vision is.
doppp|3 months ago
brulard|3 months ago
arresin|3 months ago
Is this normal in Seattle? That’s a tonne of applicants especially on what seems like a niche job site. Are they mostly junk offshore applications or bots?
Interesting that out of that it looks like 90% were late applications or not qualified and only 17 total completed the take home.
zipy124|3 months ago
npinsker|3 months ago
ro_bit|3 months ago
kassner|3 months ago
If there is at least a recruiter screening first, I’ll apply and ask about “Bring Your Own Code Examples”, mostly when their daily work would use tools that I have some code published.
matusp|3 months ago
preciousoo|3 months ago
creer|3 months ago
Isn't that a dark pattern (in addition to being a time saver for everyone)? It's a damned if you do and damned if you don't kind of thing.
This is an adversarial question in a process which needs to be ruthless against the time-wasting applications, but also needs to be cooperative with future co-workers.
If you are open to a broad range of salary in exchange for perhaps working at a super exciting place, then there is no good answer. Even "I'm open to a broad range of salary if that's needed to work at a super exciting place" is not a great answer.
Leaving it to the legal requirement of posting an expected salary range and negotiating from there might be a better way.
edent|3 months ago
FinnKuhn|3 months ago
patja|3 months ago
ludicrousdispla|3 months ago
bob1029|3 months ago
There is something that feels very cursed to me about a team of size 2~10 for game dev. At this point I'd much rather go solo or join a team of 100+. Zero structure or a lot of structure. A medium amount of structure seems to bring the maximum amount of entropy.
bluescrn|3 months ago
Very small teams can be great - so long as you are able to focus on building a game, not building a company. You need to treat it more like a game jam, at least in the early stages of a project. Just make things, see what works, rapidly prototype and iterate.
But you probably all need to be equal partners in the project from the start for it to work well. It's not so good for creative work when there's a management/employees 'them and us' divide from the start.
meheleventyone|3 months ago
stevoski|3 months ago
If you’ve never run a hiring process, it’s hard to get a feel for just how difficult and time-consuming it is.
And risky - hiring someone wrong for the role is very expensive and disruptive. And yet more likely to happen that you’d think, even with a rigorous selection process.
cs02rm0|3 months ago
I get icky feelings about these. Clearly, 18 out of 20 submitted something so I guess most people go along with it and perhaps I'm an outlier.
overgard|3 months ago
murkt|3 months ago
sentrysapper|3 months ago
I wish more companies understood this. In all my years of interviews I never got a coding interview or take home programming assignment that even remotely resembled the work they needed.
unknown|3 months ago
[deleted]
rnewme|3 months ago
thegrim33|3 months ago
You couldn't even do the INITIAL / quick triage for those 46? You couldn't even just spend 20 seconds on each to see if they were remotely qualified to be considered or not? It would have taken one person 15 minutes. You instead just threw away all those applications without even glancing at them? In addition, these were applications that people got in before the deadline, before you closed the application process, they applied correctly, and you just threw them away without even glancing at them?
creer|3 months ago
"Late applications" though: Wouldn't it be very wasteful though, to dismiss a third of candidates simply because late application when the posting was live just two days? As admitted, the hiring process is time consuming. It seems wasteful then to filter on "available during the 2 day posting window"! That availability was not a serious job requirement.
pvillano|3 months ago
The best predictors of job performance are a simulation of the job and past performance. This is not new research or a secret.
codr7|3 months ago
Chances are, you're going to see applicants with combinations of skills you couldn't even have imagined.
Humans are complicated.
dainiusse|3 months ago
testing22321|3 months ago
danenania|3 months ago
Even if you are primarily using codegen, your own coding ability, taste, problem solving, etc. are still deciding inputs to the quality of the final result. And it’s much easier to assess these things in an hour when a human is writing and debugging a relatively small amount of code.
AI tools just produce too much code in a short time. It’s hard to assess what the candidate’s quality bar and attention to detail are really like when there’s so much code to wade through. Anyone can vibe code, but not many can do it without creating mountains of tech debt… the ones who can are usually good programmers with or without AI.
zerr|3 months ago
ramon156|3 months ago
Let's say we care about the potential employee's needs, most people want to feel like they're making a difference at work and build something that matters. I have never had a job interview where I was able to discuss this topic. It's just "are you good enough to work for us?" while the entire company is falling apart in the background.
Do you actually want to improve your company, or are you just looking to share more workload? Because those are two different things. I'm not looking to join your bike-shedding business
chr1ss_code|3 months ago
Every non-junior dev/coder should already have at least some indicators out there showing how they code - GitHub, a personal site or any other resources. For juniors or CS graduates there might be bit of a grey zone, but even then, with how widely available web space is nowadays, there’s really no excuse not to have something out there if you are serious about the "love for coding".
So the sentence “we need to respect people’s time as much as our own” seems flawed to me, because you obviously don’t respect the time of the candidates who coded for nothing for you.
To me, that is also a huge red flag when considering a position.
Important should be assessing someone’s theoretical knowledge of software patterns, principles and architectures ..just getting a feel for their nerd level. Seeing how much they actually care about code and details, whether they can really express themselves and if they could communicate a problem clearly.
automatic6131|3 months ago
nradov|3 months ago
DeathArrow|3 months ago
For a 3 people team this 4 weeks hiring process is too tedious.
aeonflux|3 months ago
lunias|3 months ago
imsurajkadam|3 months ago
ro_bit|3 months ago