[lamontcg]

That's kinda the whole point.

SpaceX can use an optimization algorithm to hoverslam a rocket booster, but the optimization algorithm didn't really figure it out on its own.

The optimization algorithm was used by human experts to solve the problem.

[Sharlin]

In this case there certainly were experts doing hand-holding. But simply being able to ask the right question isn't too much to ask, is it? If it had been merely a grad student or even a PhD student who had asked ChatGPT to figure out the result, and ChatGPT had done that, even interactively with the student, this would be huge news. But an average person? Expecting LLMs to transcend the GIGO principle is a bit too much.