[jawiggins]

How sure are you that OpenAI is using that?

I would have suspected it too, but I’ve been struggling with OpenAI returning syntactically invalid JSON when provided with a simple pydantic class (a list of strings), which shouldn’t be possible unless they have a glaring error in their grammar.

[gradys]

You might be using JSON mode, which doesn’t guarantee a schema will be followed, or structured outputs not in strict mode. It is possible to get the property that the response is either a valid instance of the schema or an error (eg for refusal)

[jawiggins]

How do you activate strict mode when using pydantic schemas? It doesn't look like that is a valid parameter to me.

No, I don't get refusals, I see literally invalid json, like: `{"field": ["value...}`

[koakuma-chan]

https://github.com/guidance-ai/llguidance

> 2025-05-20 LLGuidance shipped in OpenAI for JSON Schema

[mmoskal]

OpenAI is using [0] LLGuidance [1]. You need to set strict:true in your request for schema validation to kick in though.

[0] https://platform.openai.com/docs/guides/function-calling#lar... [1] https://github.com/guidance-ai/llguidance

[jawiggins]

I don't think that parameter is an option when using pydantic schemas.

class FooBar(BaseModel): foo: list[str] bar: list[int]

prompt = """#Task Your job is to reply with Foo Bar, a json object with foo, a list of strings, and bar, a list of ints """

response = openai_client.chat.completions.parse( model="gpt-5-nano-2025-08-07", messages=[{"role": "system", "content": FooBar}], max_completion_tokens=4096, seed=123, response_format=CommentAnalysis, strict=True )

TypeError: Completions.parse() got an unexpected keyword argument 'strict'

[simonw]

You have to explicitly opt into it by passing strict=True https://platform.openai.com/docs/guides/structured-outputs/s...

[jawiggins]

Are you able to use `strict=True` when using pydantic models? It doesn't seem to be valid for me. I think that only works for json schemas.

class FooBar(BaseModel): foo: list[str] bar: list[int]

prompt = """#Task Your job is to reply with Foo Bar, a json object with foo, a list of strings, and bar, a list of ints """

response = openai_client.chat.completions.parse( model="gpt-5-nano-2025-08-07", messages=[{"role": "system", "content": FooBar}], max_completion_tokens=4096, seed=123, response_format=CommentAnalysis, strict=True )

> TypeError: Completions.parse() got an unexpected keyword argument 'strict'