[sp0ck]

My current conslusion is quite often these LLM's are liars. I've asked for some Rust async code - it provided perfecly reasonable code with some crates(libs) I was not familiar. When asked about them I've spilled my drink on an answer: "This is imaginary crate providing async implementation for SNMP". Crazy.. The biggest problem with LLM'are they never tell you "I don't know/there is no answer I can find" - don't know the answer - make it up :)

[simonw]

The best LLMs do that a lot less these days. Claude 3.5 Sonnet is a lot more likely to say "I don't know" in my experience.