Rather than using symbols, have you considered using common, self-descriptive words that would likely be one token? If the model hasn't been trained on the language, and the language itself isn't self-descriptive, then the language spec would have to be part of the context window too.
No comments yet.