I do think running the code would be a tiny bit faster, even if it's merely seconds either way. Opening a python REPL and pasting that would take around 5 seconds in my case. Running the code in my head would take roughly the same at first, but then if it's in an interview I'd take the time to double check. And then check a few more times because I'd expect some kind of trick here.
Considering there's no (explicit) instruction forbidding or discouraging it, I'd consider the REPL solution to be perfectly valid. In fact some interview tests specifically look for this kind of problem solving.
I get it still, I'd expect some valuable signal from this test. Candidates who execute this code are likely to do so because they really want to avoid running the code in their head, not just because it's more straightforward, and that's probably a bad sign. And pasting that into an LLM instead of a REPL would be a massive red flag.
I just don't think answering "-11" here is a signal strong enough to disqualify candidates on its own.
If you're looking for junior-ish python devs, I'd expect a good chunk of the better ones to have a python repl open and ready just as a matter of habit.
So for them, yes, it would clearly be faster to run the code than to work through it manually.
What you're doing here is selecting for candidates who are less comfortable with using the tools that they'd be expected to use every day in the role you're hiring for. It's likely to provide a negative signal.
Maxatar|2 months ago
Eckter2|2 months ago
Considering there's no (explicit) instruction forbidding or discouraging it, I'd consider the REPL solution to be perfectly valid. In fact some interview tests specifically look for this kind of problem solving.
I get it still, I'd expect some valuable signal from this test. Candidates who execute this code are likely to do so because they really want to avoid running the code in their head, not just because it's more straightforward, and that's probably a bad sign. And pasting that into an LLM instead of a REPL would be a massive red flag.
I just don't think answering "-11" here is a signal strong enough to disqualify candidates on its own.
roryirvine|2 months ago
So for them, yes, it would clearly be faster to run the code than to work through it manually.
What you're doing here is selecting for candidates who are less comfortable with using the tools that they'd be expected to use every day in the role you're hiring for. It's likely to provide a negative signal.
ctoth|2 months ago
OR:
I could run the code in the interpreter and be 100% certain.
I know what attitude I would prefer out of my developers.
lukeinator42|2 months ago
kevin061|2 months ago
mattbee|2 months ago