[geekodour]

Hi Jeremy, I am trying to navigate the space and trying to understand what fits where.

Could you shed some lights on what parts of bge-m3 would modernbert overlap with or would this is comparing apples to oranges?

https://huggingface.co/BAAI/bge-m3

[bclavie]

Hey! It’s more like comparing apples to apple pie.

BGE-M3 is a fine-tuned embedding models. This means that they’ve taken a base language model, which was trained for just language modeling, then applied further fine-tuning to make it useful for a given application, in this case, retrieval.

ModernBERT is one step back earlier in the pipeline: it’s the language model that application-specific models such as M3 build on.