[longdog]

I've been playing with Llama 3 8b instruct but I've found it to be surprisingly low quality compared to some of the better Mistral 7b finetunes (zephyr, dolphin, openorca). Rather surprising because there's no way Mistral or any of the organizations doing the finetuning did even a fraction of the training volume that Meta did.

[causal]

Complete opposite impression here

[d-z-m]

Depending on the kind of questions you're asking it, the mistral finetunes may be much better positioned to give a high quality answer. An apples-to-apples comparison IMO would be Mistral 7B instruct vs Llama3 8b instruct.

[d13]

I’ve also found that Mistral instruct base is just as good, and gives less chatty replies. It all also gave me more consistently correct responses.

[Sammi]

Shouldn't the finetunes be better than the vanilla llms? That's the point of a finetune? Maybe wait until there are llama finetunes to compare to the mistral finetunes?