WingNews logo WingNews
top | item 44877353

(no title)

CptanPanic | 6 months ago

Sounds like litellm which I use, I wonder how it compares?

discuss

order

vid|6 months ago

There is also https://github.com/maximhq/bifrost which apparently overcomes some performance issues of litellm and is easy to get going.

tomhoule|6 months ago

Yeah they definitely belong in the same space. Nexus is an LLM Gateway, but early on, the focus has been on MCP: aggregation, authentication, and a smart approach to tool selection. There is that paper, and a lot of anecdotal evidence, pointing to LLMs not coping well with a selection of tools that is too large: https://arxiv.org/html/2411.09613v1

So Nexus takes a tool search based approach to solving that, among other cool things.

Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.

fbjork|6 months ago

Founder of Grafbase here.

Here are a few key differentiators vs LiteLLM today:

- Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing

- The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc.

- Nexus is written in Rust - LiteLLM is written in Python

That said, LiteLLM is an impressive project, but we're just getting started with Nexus so stay tuned for a steady barrage of feature launches the coming months:)