> We are using mistral-common internally for tokenization and want the community to use it to unlock full capacities of our models. As mistral-common is a Python library, we have opened a PR to add a REST API via FastAPI to make it easier for users who are not in the Python ecosystem.
A cpp binary depending on a python server is a bit sad.
I appreciate Mistral (and others) releasing their weights for free. But given how llama.cpp underpins a lot of the programs which allow users to run open weight models, it is a little frustrating to have companies which brag about releasing models to the community, leave the community to their own devices to slowly try and actually implement their models.
I hear the reason for this is that llama.cpp keeps breaking basic things, so they have become an unreliable partner. Seems this is what Ollama is trying to address by diluting their connections to llama.cpp and directly contacting companies training these models to have simultaneous releases (e.g. GPT-OSS).
There's a lot of extremely subtle politics going on in the link.
Suffice it to say, as a commercial entity, there's a very clever way to put your thumb on the scale of what works and what doesn't without it being obvious to anyone involved, even the thumb.
flakiness|6 months ago
A cpp binary depending on a python server is a bit sad.
I hope this is a stopgap measure and someone port it to C++ eventually:https://github.com/mistralai/mistral-common/blob/main/src/mi...
the_mitsuhiko|6 months ago
hodgehog11|6 months ago
I hear the reason for this is that llama.cpp keeps breaking basic things, so they have become an unreliable partner. Seems this is what Ollama is trying to address by diluting their connections to llama.cpp and directly contacting companies training these models to have simultaneous releases (e.g. GPT-OSS).
mattnewton|6 months ago
They do release high quality inference code, ie https://github.com/mistralai/mistral-inference
refulgentis|6 months ago
llama.cpp also got GPT-OSS early, like Ollama.
There's a lot of extremely subtle politics going on in the link.
Suffice it to say, as a commercial entity, there's a very clever way to put your thumb on the scale of what works and what doesn't without it being obvious to anyone involved, even the thumb.
baggiponte|6 months ago