[levocardia]

The original study: https://link.springer.com/article/10.1007/s11547-025-02161-1

It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer.

The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

[OJFord]

> It was retrospective-only, i.e. a case series on women who were known to have breast cancer, so there were zero false negatives and zero true negatives, because all patients in the study truly had cancer.

Well yes, that's the denominator for determining selectivity, which is what the headline claim is about.

Also, they need to set up their next paper:

> However, the retrospective, cancer-only design limits generalizability, highlighting the need for prospective multicenter screening trials for validation.

[operatingthetan]

>The AI system used was a ConvNet used commercially circa 2021, which is when the data for this case series were collected.

Does this mean that newer AI systems would perform significantly differently?

[energy123]

Strictly in terms of architecture, CNNs are still SOTA for small data visual tasks, especially when the target is a locally specific phenomenon where global context isn't as necessary. It has good inductive bias for this.

The main known way to improve performance on tasks like this is getting more data.

[dfee]

Well, certainly not. We shouldn’t draw conclusions about modern AI systems from multi-generation old systems: one way or the other.

[f1shy]

Not at all. There is no implication, implicit or explicit, that anything in the world is better or worse. It is just a statement of fact.

[d0liver]

> there were zero false negatives

Wouldn't this mean that AI identitied them all has having cancer?

[cbarnes99]

They did all have cancer

[aurareturn]

2021 is an eternity in AI industry.

Edit: I have a problem with the way the title uses "AI" as a singular unchanging entity. It should really be "An AI system misses nearly...". There is no single AI and models are constantly improving - sometimes exponentially better.

[boxed]

I believe there's a big issue in the US of over-diagnosing breast cancer too. "Known to have breast cancer" might not be so clear cut.

[ako]

That statement would benefit from a link to your source.