[monocultured]

Just the other day there were news about how many mistakes the AI-assisted journaling caused – wrong names, wrong diagnoses, typos (buksmärta -> kuksmärta which is hilarious but serious).

Some of it is surely teething problems, but unless there is a robust check upon implementation it might just add another layer of inefficient new public management make-work to the system.

https://sverigesradio.se/artikel/ai-journaler-i-sjukvarden-k...

[Angostura]

It feels to me like an autopilot problem in the making. "This thing means that you don't have to keep your eyes on the road - but please ensure you keep your eyes on the road, in case of errors"

[pixl97]

The issue is, if you have any kind of rare condition, it already is this way. Much like the entire white side of a semi being presented to you across the road is a rare condition for autopilot, a huge number of 'rare' diseases already present problems for humans leading doctors and their staff to make errors by assuming the most likely condition. There is some saying like "It's probably a horse and not a zebra", but when it comes to hospitals zebras and even unicorns do show up, especially in the cases with recurring problems.

[kohbo]

I found it interesting that your mind went to Tesla's autopilot. My mind went to operating airplanes. Most newer small planes have some form of GPS but you're technically not supposed to use instrument navigation until your certified to do so. I haven't met a single pilot that didn't do so, though.

Anyway, it creates the very problem you mentioned but just replace "road" with "outside the cockpit".

[williamcotton]

Reading and doing minor edits is much less of a cognitive load than writing.

The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question.

[ceejayoz]

> The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question.

But that's what will inevitably happen at some point, when they get to the point of only rarely making big dangerous mistakes.

[dv_dt]

thinking reading and doing edits as less work than entering it yourself is exactly what will cause critical errors to be made. It may not suggest that but just like there are Tesla drivers who are supposed to watch the road there are users who will not check. And in a medical record that can be deadly.

[ThePowerOfFuet]

> typos (buksmärta -> kuksmärta which is hilarious but serious)

To save people looking it up, that one-char difference changes "abdominal pain" into "cock pain".

Wow.

[hulitu]

Well, they are close enough to each other, aren't they ? /s

That is the main problem with the AI: it is close enough, but never there.

[chollida1]

This seems very similar to self driving cars.

At the beginning they will be worse than humans and cause deaths that humans would have prevented while at the same time probably saving lives where a human would make a mistake.

But not far down the road they'll become much better than humans even if they do occasionally make a mistake and cause a death that a human wouldn't' have they'll save far more lives due to them not making the mistakes that humans do.

[b112]

But not far down the road they'll become much better than humans

While I think you're correct, there is no proof this will ever be achieved.

It very well may not be possible along our current path. It may take a 100 years, 1000 to get there.

And yes it could take only 20 more. But to state this as a certainty?

No.

[elitan]

I wouldn't be surprised if Region Blekinge were using something much worse and much more expensive than Whisper for their transcription.

I've been transcribing A LOT of SR (Swedish Radio) shows as part of https://nyheter.sh/, and Whisper (self-hosted) has been very accurate.

[popinman322]

Tangent here: really? I've found base Whisper has concerning error rates for non-US English accents; I imagine the same is true for other languages with a large regional mode to the source dataset.

Whisper + an LLM can recover some of the gaps by filling in contextually plausible bits, but then it's not a transcript and may contain hallucinations.

There are alternatives that share Whisper internal states with an LLM to improve ASR, as well as approaches that sample N-best hypotheses from Whisper and fine-tune an LLM to distill the hypotheses into a single output. Haven't looked too much into these yet given how expensive each component is to run independently.

[fencepost]

This is a situation where running within the context of the EMR and having access to the existing chart data is likely to make a lot of difference. My bigger concern is that there are a lot of different things being lumped together under "AI", and this is going to hit a bunch of different areas of machine learning.

[masto]

This would be a major concern for me.

The one time I used AI meeting notes, some important details were wrong. And beyond that, the notes were just terrible. A literal transcription can be useful. A summary of the substance of the meeting can be useful. This was neither. A human would know that a tangent talking about the weather is not important, but AI notes are just as likely to fill the document with "Chris mentioned it had rained yesterday but he was hoping to cook hamburgers when the sun comes out. Alice and Bob expressed opinions about side dishes, with the consensus being that fries are more appropriate than potato salad." as it was to miss a nuanced point that a human would have recorded because they understood the purpose of the meeting. And then it'd give me an action item to buy corn.

[_wire_]

> Some of it is surely teething problems...

Just gonna adjust the temperature of your baby here, ok, now he should grow up just fine.