[henry_pulver]

On the tokenizers Mistral use for proprietary models, this isn't common knowledge.

This tokenizer is correct for the 7B open model and 8x7B MoE model. It'll probably be the closest to the ones their proprietary API-only models use