[toasteros]

That's because you're beginning with the assertion that LLMs are tools. You cannot create things with an LLM. You can't be creative with them. They are not useful for building things. What you get out of an LLM is a regurgitation of existing information, peppered with assumptions based on your query.

They're not tools.

A hammer helps you build a house. An LLM helps you read StackOverflow in the most inefficient means possible.

[Legend2440]

This is just nonsense, and obviously untrue.

It is easy to show that LLMs can create new things (“write a poem about libc”) or answer questions that cannot be googled (“can a pair of scissors cut through a Ford F150?”)

[toasteros]

The poem won't be unique. It will follow a basic formula and make sure to pick up the words you asked it to use. You might not get the exact same poem every single time, but there's a reason that you can tell when something has been generated.

A good recent example are the lyrics of the songs on an album by a band called Saor. The album is called Amidst the Ruins.