[joshgel]

Ya, internist here.

For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.

So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.

Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.

[capableweb]

I agree that we shouldn't get ahead of ourselves with the current technology, but what you said earlier applies to practically every industry and science. What you learn at the actual job is always far more up to date than what you learn in school, no matter if it's being a engineer, doctor or just a lowly programmer.

[notch898a]

Yes but the difference is most engineers, and pretty much all lowly programmers are unlicensed. AI or some non-human accessible in the countryside so you don't have to order questionable "fish" antibiotics or "cat" anti-parasitic would be a nice step up from the current gatekeeping of medicine from people with limited access.

[sizzle]

What if the AI is trained on board exams and other high signal testing/examination materials? Surely it will become superhuman in its medical abilities?

[Madmallard]

Ive never experienced anything in my care with doctors that I couldn’t understand with a days worth of research into things like UpToDate. It isn’t complicated. It is largely memorization and application of an algorithm which is just borderline useless for complex conditions that are emerging more modernly.

[cguess]

Have you ever had anything besides a bad cold? Tell me you understand every acronym in this article, and the ability to explain it succinctly to a patient much less to be able to hand off a case to "another" physician https://www.nejm.org/doi/full/10.1056/NEJMoa2206714

Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).

[Veen]

How do you know that you actually understand a topic as well as a doctor? How do you verify that? It's not unusual for people to think they comprehend a topic at expert level, when in fact they do not. The correlation between confidence and understanding is not a reliable measure. That's why doctors are trained by more expert colleagues who can judge their true understanding, have to take exams, etc.