top | item 41649645

(no title)

danielcampos93 | 1 year ago

Not mentioned in their blog posts but on the model cards on huggingface: "Molmo 72B is based on Qwen2-72B and uses OpenAI CLIP as vision backbone. Molmo-72B achieves the highest academic benchmark score and ranks second on human evaluation, just slightly behind GPT-4o." Others are based on Qwen 7B. What happened to the Olmo chain?

discuss

order

jszymborski|1 year ago

I think the "Molmo-7B-O" and "MolmoE-1B" models are using Olmo, judging by the fact its LLM backbone is the only one listed as having open data.

EDIT: From the post "For the LLM, we have trained models on a variety of choices at different scales and degrees of openness including: the fully open-weight and data OLMo-7B-1024 (using the October, 2024 pre-released weights, which will be public at a later date), the efficient fully open-weight and data OLMoE-1B-7B-0924, open-weight Qwen2 7B, open-weight Qwen2 72B, open-weight Mistral 7B, open-weight Gemma2 9B, and Phi 3 Medium). Today we are releasing 4 samples from this family."

comp_raccoon|1 year ago

This is correct! we wanted to show that you can use PixMo dataset and our training code to improve any open model, not just ours!