[Treesrule14]

Has anyone else found a good way to swap out models between companies, Langchain has made it very easy for us to swap between openai/anthropic etc

[riwsky]

The point is that you don’t need a framework for that; the APIs are already similar enough that it should be obvious how to abstract over them using whatever approach is natural in your programming language of choice.

[refulgentis]

I have a consumer app that swaps between the 5 bigs and wholeheartedly agree, except, God help you if you're doing Gemini. I somewhat regret hacking it into the same concepts as everyone else.

I should have built stronger separation boundaries with more general abstractions. It works fine, I haven't had any critical bugs / mistakes, but it's really nasty once you get to the actual JSON you'll send.

Google's was 100% designed by a committee of people who had never seen anyone else's API, and if they had, they would have dismissed it via NIH. (disclaimer: ex-Googler, no direct knowledge)

[pveierland]

Using Llama Index for this via the `llama_index.core.base.llms.base.BaseLLM` interface. Using config files to describe the args to different models makes swapping models literally as easy as:

  chat_model:
    cls: llama_index.llms.openai.OpenAI
    kwargs:
      model: gpt-4

  chat_model:
    cls: llama_index.llms.gemini.Gemini
    kwargs:
      model_name: models/gemini-pro

[me_vinayakakv]

Vercel AI SDK[1] shines in this aspect in JS ecosystem.

They have the concept of providers [2] and switching between them is easy as changing parameters of a function[3]

[1]:https://sdk.vercel.ai/docs/introduction

[2]: https://sdk.vercel.ai/docs/foundations/providers-and-models

[3]: https://sdk.vercel.ai/docs/ai-sdk-core/overview#ai-sdk-core

[spdustin]

LiteLLM.

https://www.litellm.ai/

[ponywombat]

LiteLLM seemed to be the best approach for what I needed - simple integration with different models (mainly OpenAI and the various Bedrock models) and the ability to track costs / limit spending. It's working really well so far.

[emporas]

Didn't know about LiteLLM. That seems to be the right kind of middleware most people would need, instead of Langchain.

[ilaksh]

Use a consistent argument structure and make a simple class or function for each provider that translates that to the specific API calls. They are very similar APIs. Maybe select the function call based on the model name.

[Havoc]

Use openrouter. One OpenAI like api but lots of models

[nosefurhairdo]

The strategy design pattern would be suitable for this.

[skeledrew]

Openrouter maybe?