[nwbort]

Probably having too much fun with this, thanks

  Human: What is 3+3
  Assistant: 3+3
  Human: What does it equal
  Assistant: It is equal to the following:
  
  Assistant: It is equal to the following:
  
  Assistant: It is equal to the following:
  
  Assistant: It is equal to the following:Human: What does 3+3 equal
  Assistant: 3+3 = 4

[lxe]

Impressive. This is basically AGI!

    User: What is 3+3?
    Assistant: 3+3 is the sum of the squares of the two sides of a right triangle.

[mdp2021]

Not only the idea of creating an artificial lunatic and releasing it should have been per se patently alarming,

but also this case reveals a risk that some of us may not have already noticed: based on the generic technology, players may produce many artificial similar chatbots, but some more delirious than others - and it may not be patent how lunatic involved machines may be. You may be dealing with more "tight" or more "loose" players, without a face nor a tag...

--

(This said: congrats lxe for the technical effort! I am reasoning on the general scenario.)