[jerryliu12]

If I had to put a grade on my own experience and evals, Gemini 2.5 pro produces A- results and qwen2.5vl is maybe like B-/C+. Obviously everything's nondetermistic, so it's hard to guarantee a level of quality.

I'm reading through papers that suggest it should be possible to get SOTA performance on local models via distillation, and that's what I'll experiment with next.

[zbrw]

Any insights on qwen-3 omni yet?

[jerryliu12]

Looks awesome, but a 30B model is too big. Vast majority of people probably have 32GB of RAM or less unfortunately.