[jawiggins]

So cool to see Anthropic support this feature. I’m a heavy user of the OpenAI version, however they seem to have a bug where frequently the model will return a string that is not syntactically valid json, leading the OpenAI client to raise a ValidationError when trying to construct the pydantic model. Curious if anyone else here has experienced this? I would have expected the implementation to prevent this, maybe using a state machine to only allow the model to pick syntactically valid tokens. Hopefully Anthropic took a different approach that doesn’t have this issue.

[brianyu8]

Brian on the OpenAI API team here. I would love to help you get to the bottom of the structured outputs issues you're seeing. Mind sending me some more details about your schema / prompt or any request IDs you might have to by[at]openai.com?

[jawiggins]

Thanks so much for reaching out, sent an email :).

[robot-wrangler]

https://github.com/pydantic/pydantic-ai/issues/582 https://github.com/pydantic/pydantic-ai/issues/2405

[matheist]

yeah I have, but I think only when it gets stuck in a loop and outputs a (for example) array that goes on forever. a truncated array is obviously not valid JSON. but it'd be hard to miss that if you're looking at the outputs.