[jessep]

Two things interest me about Claude being better than GPT-4:

1) We are all breathless that it is better. But a year has passed since GPT4. It’s like we’re excited that someone beat Usain Bolt’s 100 meter time from when he was 7. Impressive, but … he’s twenty now, has been training like a maniac, and we’ll see what happens when he runs his next race.

2) It’s shown AI chat products have no switching costs right now. I now use mostly Claude and pay them money. Each chat is a universe that starts from scratch, so … very easy to switch. Curious if deeper integrations with my data, or real chat memory, will change that.

[tempusalaria]

The current version of GPT-4 is 3 months old not 1 year old. Anthropic are legitimately ahead on performance for cost right now. But their API latency I don’t think matches OpenAI

We’ll see what GPT4.5 looks like in the next 6 months.

[lynx23]

Did you mean two or four months? Because 3 months is somewhere in december, and there were no updates around that time.

[HarHarVeryFunny]

I don't think it's just that Claude-3 seems on par with GPT-4, but rather the development timescales involved.

Anthropic as a company was only created, with some of the core LLM team members from OpenAI, around the same time GPT-3 came out (Anthropic CEO Dario Amodei's name is even on the GPT-3 "few-shot learners" paper). So, roughly speaking, in same time it took OpenAI (big established company, with lots of development momentum) to go from GPT-3 to GPT-4, Anthropic have gone from start-up with nothing to Claude-3 (via 1 & 2) which BEATS GPT-4. Clearly the pace of development at Anthropic is faster than that at OpenAI, and there is no OpenAI magic moat in play here.

Sure GPT-4 is a year old at this point, and OpenAI's next release (GPT-4.5 or 5) is going to be better than GPT-4 class models, but given Anthropic's momentum, the more interesting question is how long it will take Anthropic to match it or take the lead?

Inference cost is also an interesting issue... OpenAI have bet the farm on Microsoft, and Anthropic have gone with Amazon (AWS), who have built their own ML chips. I'd guess Athropic's inference cost is cheaper, maybe a lot cheaper. Can OpenAI compete with the cost of Claude-3 Haiku, which is getting rave reviews? It's input tokens are crazy cheap - $300 to input every word you'll ever speak in your entire life!

[theturtletalks]

Claude may be beat GPT-4 right now, but I remember ChatGPT in March 2023 being leagues better. Over the past year, it’s gotten regressive, but faster.

Claude is also lacking web browsing and code interpreter. I’m sure those will come, but where will GPT be by then? ChatGPT also offers an extensive free tier with voice. Claude’s free plan caps you as a few messages every few hours.

[jessep]

Yeah, it's a good point, but I think that our intuitions are different on this one. I don't have a horse in this race, but my assumption is that the next OpenAI release will be a massive leap, that makes GPT 4/Claude 3 Opus look like toys. Perhaps you're right though, and Anthropic's curves with bend upward even more quickly, so that they get to that they'll start catching up more quickly, until eventually be they're ahead.

[londons_explore]

They're comparing against gpt-4-0125-preview, which was released at the end of January 2024. So they really are beating the market leader for this test.

[humansareok1]

Model Updates != New Models.

GPT5 will be substantially better than even the latest GPT4 update.

[worldsayshi]

ChatGPT does have at least a year head start so this doesn't seem surprising. This proves that OpenAI doesn't really have any secret sauce that others can't reproduce.

I suppose size will become the moat eventually but atm it looks like it could become anyone's game.

[CuriouslyC]

Size is absolutely not going to become the moat unless there's some hardware revolution that makes running big models very very cheap, but that requires a very large up-front capital cost to deploy. Big models are inefficient, and as smaller models improve there will be very few use cases where the big models are worth the compute.

[mikkom]

Anthropic is ex-openai so even if there is a secret sauce that openai uses, they might know it.

[Alifatisk]

> We are all breathless that it is better. But a year has passed since GPT4. It’s like we’re excited that someone beat Usain Bolt’s 100 meter time from when he was 7.

Sounds like some sort of siding with closedAI (openAi), when I need to use an llm, I use whatever performs the best at the moment. It doesn’t matter who’s behind it to me, at the moment it is Claude.

I am not going to stick to ChatGPT because closedAi have been pioneers or because their product was one of the best.

I hope I didn’t sound too harsh, excuse me in that case.

[danielbln]

> closedAI (openAi)

Is this supposed to be clever? It's like saying M$ back in the 90s. Yeah, OpenAI doesn't deserve its own name, but maybe we can let that dead horse rest.

[artdigital]

Claude has way too many safeguards for what it believes is correct to talk about and what isn’t. Not saying ChatGPT is better, it also got dumbed down a lot, but Claude is very heavy on being politically correct on everything.

Ironically the one I find the best for responses currently is Gemini Advanced.

I agree with you that there is no switching cost currently, I bounce between them a lot

[bredren]

Does this matter in pure software dev?

[j45]

Hard things aren’t easy to do.

Openai is not only faster at updating, the updates deliver. Then things like sora out of nowhere.

It’s great to see other models keeping up or getting ahead because a year ago the gap was bigger

[lenerdenator]

What's a good way to have access to many chatbots in one place?

[longnguyen]

If you’re on macOS, give BoltAI[0] a try. Other than supporting multiple AI services and models, BoltAI also allows you yo create your own AI tools. Highlight the text, press a shortcut key then run a prompt against that text.

Disclaimer: I build it :D

[0]: https://boltai.com

[gorbypark]

I use an app called MindMac for macOS that works with nearly "all" of the APIs. I currently am using OpenAI, Anthropic and Mistral API keys with it, but it seems to support a ton of others as well.

[budududuroiu]

Not affiliated in any way, but I use openrouter.ai to pay per token rather than have monthly subscriptions

[mupuff1234]

MSFT trying to hedge their bets makes it seems like there's a decent chance OpenAI might have hit a few roadblocks (either technical or organizational)

[JKCalhoun]

Billion-dollar corporation hedging its bets is standard practice and I personally wouldn't read anything into it.

[foobar_______]

I agree with your analogy. Also, there is a quite a bit of "standing on the shoulders of giants" kind of thing going on. Every company's latest release will/should be a bit better than the models released before it. AI enthusiasts are getting a bit annoying - "we got a new leader boys!!!!*!" for each new model released.