[tropicaljacket]

> Although the Colombian court filing indicates that ... its responses were fact-checked

The problem is that the fact-checkers themselves can make mistakes and chatGPT is very good at making text that looks correct, even to experts. See the Meta StackOverflow post on banning ChatGPT answers for example

[geraldyo]

At that point we're back to the fact that humans make mistakes, so the AI is irrelevant. Had they not used AI, they could still make mistakes that look correct, even to experts

[cscurmudgeon]

It is all a matter of degree and not a binary situation.