(no title)
aeneas_ory | 1 month ago
To find most runtime bugs (e.g. incorrect regex, broken concurrency, incorrect SQL statement, ...) you need to understand the mental model and logic behind the code - finding out if "is variable XYZ unused?" or "does variable X oveshadow Y" or other more "esoteric" lint rules will not catch it. Likelihood is high that the LLM just hallucinated some false positive lint rule anyways giving you a false sense of security.
normie3000|1 month ago
I'm not sure if there's some subtlety of language here, but from my experience of javascript linting, it can often prevent runtime problems caused by things like variable scoping, unhandled exceptions in promises, misuse of functions etc.
I've also caught security issues in Java with static analysis.
aeneas_ory|1 month ago
But the author claims that you can catch runtime bugs by letting the LLM create custom lint rules, which is hyperbole at least and wrong at most and giving developers a false sense of security at worst.
shepherdjerred|1 month ago
kaoD|1 month ago
Why is this such a common occurrence here? Does this fallacy have a name?
EDIT: seems to be https://en.wikipedia.org/wiki/Nirvana_fallacy
qarl|1 month ago
My LLM has theorized that its success at answering trivia questions has left some people feeling threatened.