[2bitencryption]

TypeChat: let's try really hard to try to convince the model to make the highest-scoring tokens follow the grammar we want.

Guidance (and this project?): Let's not even bother with trying to convince the model; instead, we'll only sample from the set of tokens that are guaranteed to be correct for the grammar we want to emit.

[btwillard]

Yeah, and our addition to all that is to almost completely remove the cost of determining the next valid tokens on each step.