Constrained generation guarantees syntax. It does not guarantee semantic correctness tho. Imagine you want a json object with "hp" and "damage". If you use a grammar, the model will be forced to output a json object with those two values. But it's not guaranteed to get sensible values.
With a 2nd pass you basically "condition" it on the text right above, hoping to get better semantic understanding.
I'm pretty sure the grammar is generated from the Json schema, it doesn't just constrain json syntax, it constraints on the schema (including enums and such). The schema is also given to the model (at least in openai) you can put instructions in the json schema as well that will be taken into account.
NitpickLawyer|5 months ago
With a 2nd pass you basically "condition" it on the text right above, hoping to get better semantic understanding.
lyu07282|5 months ago