This is exactly the kind of thing I hope LLM chatbots will be genuinely useful for. Though, how often do you find it completely hallucinating endpoints / parameters etc. ?
I use it for similar things as GP, and find its strengths to be similar too.
ChatGPT hallucinates SVG path attributes. Ask it to make an svg of a unicorn - it will give you markup that looks okay, but if you look at the values of the paths, it's clearly gibberish.
(SVG is a particularly interesting case because it's XML on the outside, but several attributes are highly structured, esp g.transform and path.d. Path.d is basically the string of a Logo-like programming language. I was specifically looking at these attributes for realism, and didn't find it.)
Great question. If you ask it for an API endpoint that is described online but isn't well documented publicly, it seems to default back to what it thinks you should do. For example, in one example, it hallucinates that you need a bearer token.
I don't know whether that is because that is a common way of doing things or whether a previous prompt responded with a bearer token... But it wasn't right.
For me, it's a leaping off point that often saves time if I ask the right question. To your point, you have to be quick to know enough about the API to deduce whether you and Chat GPT are in the same universe.
javajosh|2 years ago
ChatGPT hallucinates SVG path attributes. Ask it to make an svg of a unicorn - it will give you markup that looks okay, but if you look at the values of the paths, it's clearly gibberish.
(SVG is a particularly interesting case because it's XML on the outside, but several attributes are highly structured, esp g.transform and path.d. Path.d is basically the string of a Logo-like programming language. I was specifically looking at these attributes for realism, and didn't find it.)
dr_dshiv|2 years ago
jmann99999|2 years ago
I don't know whether that is because that is a common way of doing things or whether a previous prompt responded with a bearer token... But it wasn't right.
For me, it's a leaping off point that often saves time if I ask the right question. To your point, you have to be quick to know enough about the API to deduce whether you and Chat GPT are in the same universe.