top | item 39146945 (no title) j-b | 2 years ago The trick is to access the "bard-jan-24-gemini-pro" model, available in direct chat mode here: https://chat.lmsys.org/. Significantly better than the prior model. discuss order hn newest jxy|2 years ago how odd! What exactly is lmsys using? Some hidden API that google give them so they can have a better ranking there? j-b|2 years ago Most likely through this platform: https://console.cloud.google.com/vertex-ai load replies (1) declaredapple|2 years ago > Some hidden API that google give them so they can have a better ranking there?I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models.After all, Human A/B tests are far better then the current automated benchmarks.I would like more info from lmsys as to how they're accessing these though. bazmattaz|2 years ago Thanks for sharing. Is this a free way to access GPT4-turbo then or are there some limitations?
jxy|2 years ago how odd! What exactly is lmsys using? Some hidden API that google give them so they can have a better ranking there? j-b|2 years ago Most likely through this platform: https://console.cloud.google.com/vertex-ai load replies (1) declaredapple|2 years ago > Some hidden API that google give them so they can have a better ranking there?I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models.After all, Human A/B tests are far better then the current automated benchmarks.I would like more info from lmsys as to how they're accessing these though.
j-b|2 years ago Most likely through this platform: https://console.cloud.google.com/vertex-ai load replies (1)
declaredapple|2 years ago > Some hidden API that google give them so they can have a better ranking there?I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models.After all, Human A/B tests are far better then the current automated benchmarks.I would like more info from lmsys as to how they're accessing these though.
bazmattaz|2 years ago Thanks for sharing. Is this a free way to access GPT4-turbo then or are there some limitations?
jxy|2 years ago
j-b|2 years ago
declaredapple|2 years ago
I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models.
After all, Human A/B tests are far better then the current automated benchmarks.
I would like more info from lmsys as to how they're accessing these though.
bazmattaz|2 years ago