My hunch is that this is exactly what he was expecting. There is a lot of hype around ChatGPT passing the medical exam and this exercise is a counter point to that.
The medical exam has very specific questions which define what you're expected to include in the answer. The question asked in this case was nowhere near that detailed, do I don't think they're comparable. To really evaluate something beyond the "random generic user" level, you need to be familiar with the tech as well.
The article really tells us more about the experience of someone with no chatgpt knowledge checking their own symptoms rather than its usability for emergency diagnosis.
any example that doesn't use the current sota isn't a very good counter point to be honest. 3 barely passed. 4 aced it. For all we know, GPT-4 erases most of his concerns (not saying it would).
thrill|2 years ago
nyokodo|2 years ago
With a subscription you can use GPT-4 with ChatGPT. ChatGPT is just the wrapper to the model.
viraptor|2 years ago
The article really tells us more about the experience of someone with no chatgpt knowledge checking their own symptoms rather than its usability for emergency diagnosis.
famouswaffles|2 years ago