[brianshaler]

No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools.

I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.