I am trying to understand structured output from LLMs, and as training I am building a text game.
OpenAI is using pydantic to create objects directly in its beta branch, it's quite nice. Anthropic is a bit more involved, as you need to involve tool calling.
LM Studio 0.3.5 introduced a bug for structured output when using pydantic and enums or literals, I created a workaround.
Also Gemini is not using Pydantic at all, instead TypedDict to create the json scheme.
RHab|1 year ago
OpenAI is using pydantic to create objects directly in its beta branch, it's quite nice. Anthropic is a bit more involved, as you need to involve tool calling.
LM Studio 0.3.5 introduced a bug for structured output when using pydantic and enums or literals, I created a workaround.
Also Gemini is not using Pydantic at all, instead TypedDict to create the json scheme.
And Gemini does not have a system message.
I am quite proud to have everything working, if you want to check out my code please take a look: https://github.com/HabermannR/Fantasy-Tribe-Game
Here is the backend: https://github.com/HabermannR/Fantasy-Tribe-Game/blob/main/L...
For example, this is how I call Gemini:
completion = model.generate_content(
messages[0]['content'] + ": " + messages[1]['content'],
generation_config=genai.GenerationConfig(
response_mime_type="application/json", response_schema=response_types.typed_dict
),
)
result = response_types.pydantic_model.model_validate_json(completion.text)
Happy for any feedback!