[breadislove]

For everyone wondering how good this and other benchmarks are:

- the OmniAI benchmark is bad

- Instead check OmniDocBench[1] out

- Mistral OCR is far far behind most Open Source OCR models and even further behind then Gemini

- End to End OCR is still extremely tricky

- composed pipelines work better (layout detection -> reading order -> OCR every element)

- complex table parsing is still extremely difficult

[1]: https://github.com/opendatalab/OmniDocBench

[hakunin]

Wish someone benchmarked Apple Vision Framework against these others. It's built into most Apple devices, but people don't know you can actually harness it to do fast, good quality OCR for you (and go a few extra steps to produce searchable pdfs, which is my typical use case). I'm very curious where it would fall in the benchmarks.

[wahnfrieden]

It is unusable trash for languages with any vertical writing such as Japanese. It simply doesn’t work.

[graeme]

Interesting. How do you harness it for that purpose? I've found apple ocr to be very good.

[CaptainOfCoit]

Yeah, if it was cross-platform maybe more people would be curious about it, but something that can only run on ~10% of the hardware people have doesn't make it very attractive to even begin to spend time on Apple-exclusive stuff.

[cheema33]

> the OmniAI benchmark is bad

According to Omni OCR benchmark, Omni OCR is the best OCR. I am sure you all will find no issues with these findings.