[phildougherty]

Even if it is 10x cheaper and 2x worse it's going to eat up even more tokens spinning its wheels trying to implement things or squash bugs and you may end up spending more because of that. Or at least spending way more of your time.

[amarcheschi]

The benchmark of swe places it in a comparable score with respect to open models and just a few points below the top notch models though