[xkgt]

I was recently researching structured output generation for my project and I enjoyed using Outlines library a lot. It felt quite fast as it uses FSM and indexing. There are few fine prints though:

1. Sometimes constraints can decrease the quality of the output since syntax of the response is prioritized more than quality of the response 2. For memory constrained inferences, certain sampling strategies like top-k can cause OOM errors if the max_token is too high. I haven't tested that it is entirely due to structured generation but I suppose it is possible for certain regexes. 3. Vision models and other multi-modal models are not supported yet.

Apart from this, closed models also have json output but I am not sure how consistent they are

1. https://platform.openai.com/docs/guides/text-generation/json... 2. https://docs.anthropic.com/en/docs/build-with-claude/tool-us... 3. https://ai.google.dev/gemini-api/docs/api-overview#json