[yunruse]

LLMs by their nature are good at word or concept association. Cohesiveness in length is where they begin to break down.

I tend to liken them to very drunken scholars. They know things, usually at about a Wikipedia level, and they’re cheery too. But they lack a capacity to doubt themselves; they often are confidently wrong.

Often the greatest help is understanding natural language, but given its hiccups… time using it is probably best spent using it as a drunken librarian – to teach how to phrase and fetch information

As one telling recent example, I tried using an LLM to help with some jq (with which I’m rusty); it got a few basics and then repetitively tripped over a syntax hiccup on loop, “correcting” itself to the same answer each time. A StackOverflow search or two, for comparison, answered my questions and taught some new syntax too. Probably took less time, but more critical thought.

That, coupled with the fact LLMs tend to give an answer and then also an unnecessary verbose step-by-step, means I tend to dislike them.

I also have a huge bugbear about “AI” as a term because it tells you very little. Plenty of applied statistics (markov chains, clustering algos, deep learning eg computer vidion; even SearchRank) are used heavily in research and other cases to do a lot of good. Even for the layman: the Seek app by iNaturalist is awesome for identifying common plant species; Stockfish is (now) a NN that dominates in chess.

But these are classifiers, not generators. By their very nature it is just statistics to evaluate a classifier on a test dataset. Generators, however, are far, far thornier to test, and seem a lot more prone to overfitting.

While I’m not familiar with a typical trained generator tensor, I imagine the optimal one will be surprisingly sparse, though not in a structured way - corresponding to a more clustered “small world” network, which IRL seem the most productive.

[paulcole]

> they often are confidently wrong.

Only when they can also dismissively sneer at a marketing person will they be truly ready to replace programmers.