[jerezzprime]

There is a fallacy here. If you know they don't perform well in places where you have the expertise to detect it, it is a mistake to assume they perform well in other areas. My conclusion has been that they are useful for kicking off a task, sometimes useful for ideation, but generally will fall down on anything reasonably complex. The dangerous bit is that it's hard to detect the line where they start to fail.

[saurik]

But like, the expert also knows that the middle-of-the-bell-curve engineer makes mistakes and knows not to blindly trust their code; meanwhile, the person who is "incompetent at a topic" could be seen literally hiring the middle-of the-bell-curve engineer either as a tutor or a consultant... so I don't think trusting them when you are in that position is fair to call a "fallacy".