[apalmer]

Trying to understand why this is necessary? LLMs cannot reliably generate valid Jason?

[dwallin]

Two ways in which it is useful over existing techniques:

- It is guaranteed to match your schema

- It is much lighter weight

[Der_Einzige]

Also, it costs tokens to ask a model to do something, and it may choose not to do it re: constraints

You can force it by banning the vocabulary which violates a constraint for free.

[SkyPuncher]

There are typically two issues:

* Occasionally, random junk will get tossed in alongside valid JSON. If you're a human, it's easy to identify and fix. Programmatically, it can be much harder to fix.

* Context lengths can create issues.

[tysam_and]

Yes, but Mike on the other hand....