[mind-blight]

One issue we're running into at my job: we're struggling to find entry-level candidates whoaren't lying about what they know by using an LLM.

For the tech side, we've reduced behavioral questions and created an interview that allows people to use cursor, LLMs, etc. in the interview - that way, it's impossible to cheat.

We have folks build a feature on a fake code base. Unfortunately, more junior folks now seem to struggle a lot more with this problem

[OkayPhysicist]

The thing about entry-level candidates is that we expect them to know relatively little, anyway. When I've been delegated to participate in interviewing new candidates, a question I really like is "What's your favorite project you've worked on lately? What was interesting about it? Run into any tricky problems along the way? It can be anything: for work, school, a hobby project. Doesn't even need to software".

It slices through the bullshit fast. Either the person I'm interviewing is a passionate problem solver, and will be tripping over themselves to describe whatever oddball thing they've been working on, or they're either a charlatan or simply not cut out for the work. My sneaking suspicion is that we could achieve similar levels of success in hiring for entry level positions at my current company if we cut out literally the entirety of the rest of the interviews, asked that one question, and hired the first person to answer well.

[floren]

We came up with some simple coding exercises (about 20 minutes total to implement, max) and asked candidates to submit their responses when applying. Turns out one of the questions regularly causes hallucinated APIs in LLM responses, so we've been able to weed out a large percentage of cheaters who didn't even bother to test the code before submitting.

The other part is that you can absolutely tell during a live interview when someone is using an LLM to answer.