[whytai]

Every day this video ages more and more poorly [1].

categories of startups that will be affected by these launches:

- vectorDB startups -> don't need embeddings anymore

- file processing startups -> don't need to process files anymore

- fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming

- cost reduction startups -> they literally lowered prices and increased rate limits

- structuring startups -> json mode and GPT4 turbo with better output matching

- vertical ai agent startups -> GPT marketplace

- anthropic/claude -> now GPT-turbo has 128k context window!

That being said, Sam Altman is an incredible founder for being able to have this close a watch on the market. Pretty much any "ai tooling" startup that was created in the past year was affected by this announcement.

For those asking: vectorDB, chunking, retrieval, and RAG are all implemented in a new stateful AI for you! No need to do it yourself anymore. [2] Exciting times to be a developer!

[1] https://youtu.be/smHw9kEwcgM

[2] https://openai.com/blog/new-models-and-developer-products-an...

[morkalork]

If you want to be a start-up using AI, you have to be in another industry with access to data and a market that OpenAI/MS/Google can't or won't touch. Otherwise you end up eaten like above.

[dragonwriter]

> a market that OpenAI/MS/Google can't or won't touch.

But also one that their terms of service, which are designed to exclude the markets that they can't or won't touch, don't make it impractical for you to service with their tools.

[Cali_cramoisie]

Or you can treat what OpenAI is doing like a commodity like AWS and leverage it to solve a meaningful problem.

[ushakov]

We just launched our AI-based API-Testing tool (https://ai.stepci.com), despite having competitors like GitHub Co-Pilot.

Why? Because they lack specificity. We're domain experts, we know how to prompt it correctly to get the best results for a given domain. The moat is having model do one task extremely well rather than do 100 things "alright"

[Rastonbury]

Even if you aren't eaten, the use case will just be copied and run on the same OpenAI models by competitors, having good prompts is not good enough a moat. They win either way

[renewiltord]

Writer.ai is quite successful, and is totally in another industry that Google+MS participate in.

[ren_engineer]

depends on how much developers are willing to embrace the risk of building everything on OpenAI and getting locked onto their platform.

What's stopping OpenAI from cranking up the inference pricing once they choke out the competition? That combined with the expanded context length makes it seem like they are trying to lead developers towards just throwing everything into context without much thought, which could be painful down the road

[klabb3]

> depends on how much developers are willing to […] getting locked onto their platform.

I mean.. the lock in risks have been known with every new technology since forever now, and not just the risk but the actual costs are very real. People still buy HP printers with InkDRM and companies willingly write petabytes of data into AWS that they can’t even afford to egress at current prices.

To be clear, I despise this business practice more than most, but those of us who care are screaming into the void. People are surprisingly eager to walk into a leaking boat, as long as thousands of others are as well.

[keithwhor]

I suspect it is in OpenAI's interest to have their API as a loss leader for the foreseeable future, and keep margins slim once they've cornered the market. The playbook here isn't to lock in developers and jack up the API price, it's the marketplace play: attract developers, identify the highest-margin highest-volume vertical segments built atop the platform, then gobble them up with new software.

They can then either act as a distributor and take a marketplace fee or go full Amazon and start competing in their own marketplace.

[stuckkeys]

Reminds me of that sales entrapment approach from cloud providers. “Here is your free $400, go do your thing” next thing you know you have build so much on there already that it is not worth the time and effort to try and allocate it regardless of the 2k bill increase -haha. Good times.

[vikramkr]

i mean sure it's lock in, but it's lock in via technical superiority/providing features. Either someone else replicates a model of this level of capability or anyone who needs it doesn't really have a choice. I don't mind as much when it's because of technical innovation/feature set (as opposed to through your usual gamut of non-productive anti-competitive actions). If I want to use that much context, that's not openAIs fault that other folks aren't matching it - they didn't even invent transformers and it's not like their competitors are short on cash.

[larodi]

Well, if said startups were visionaries, the could've known better the business they're entering. On the other hand - there are plenty of VC-inflated balloons, making lots of noise, that everyone would be happy to see go. If you mean these startups - well, farewell.

There's plenty more to innovate, really, saying OpenAI killed startups it's like saying that PHP/Wordpress/NameIt killed small shops doing static HTML. or IBM killing the... typewriter companies. Well, as I said - they could've known better. Competition is not always to blame.

[karmasimida]

TBH those are low-hanging fruits for OpenAI. Much of the value still being captured by OpenAI's own model.

The sad thing is, GPT-4 is its own league in the whole LLM game, whatever those other startups are selling, it isn't competing with OpenAI.

[teaearlgraycold]

I’ve been keeping my eye on a YC startup for the last few months that I interviewed with this summer. They’ve been set back so many times. It looks like they’re just “ball chasing”. They started as a chatbot app before chatgpt launched. Then they were a RAG file processing app, then enterprise-hosted chat. I lost track of where they are now but they were certainly affected by this announcement.

You know you’re doing the wrong thing if you dread the OpenAI keynotes. Pick a niche, stop riding on OpenAI’s coat tails.

[riku_iki]

> - vectorDB startups -> don't need embeddings anymore

they don't provide embedings, but storage and query engines for embeddings, so still very relevant

> - file processing startups -> don't need to process files anymore

curious what is that exactly?..

> - vertical ai agent startups -> GPT marketplace

sure, those startups will be selling their agents on marketplace

[dragonwriter]

> they don't provide embedings, but storage and query engines for embeddings, so still very relevant

But you don't need any of the chain of: extract data, calculate embeddings, store data indexed by embeddings, detect need to retrieve data by embeddings and stuff it into LLM context along with your prompt if you use OpenAI's Assistants API, which, in addition to letting you store your own prompts and manage associated threads, also lets you upload data for it to extract, store, and use for RAG on the level of either a defined Assistant or a particular conversation (Thread.)

[visarga]

It's easy to host your query engine somewhere else and integrate it as a search function in chatGPT. Quite easy to switch providers of search.

[make3]

they definitely do provide embeddings, https://openai.com/blog/new-models-and-developer-products-an... ctrl+f retrieval, "... won't need to ... compute or store embeddings"

[blibble]

HN is quite notorious for that Dropbox comment

I suspect that video is going to end up more notorious, it's even funnier given it's the VCs themselves

[thisgoesnowhere]

I'm firmly in the camp that in a vacuum that comment looks dumb but the thread was actually great.

Those were valid concerns at the time and the market for non technical file storage like they were building was non existant.

Perfectly rational to be skeptical and Drew answered all his questions with well thought out responses.

[arcanemachiner]

More context, please.

EDIT: I guess it's this:

https://news.ycombinator.com/item?id=8863#9224

[lazzlazzlazz]

Embeddings are still important (context windows can't contain all data + memorization and continuous retraining is not yet viable), and vertical AI agent startups can still lead on UX.

[dragonwriter]

Separate embedding DBs are less important if you are working with OpenAI, since their Assistants API exists to (among other things) let you bring in additional data and let them worry about parsing it, storing it, and doing RAG with it. Its like "serverless", but for Vector DBs and RAG implementations instead of servers.

[Finbarr]

Context windows can't contain all data... yet.

[Yadayadaaaa]

Just because something is great doesn't mean that others can't compete. Even a secondary good product can easily be successful due to a company having invested too much, not being aware of openai (ai progress in general), due to some magic integration, etc.

If it would be only me, no one would buy azure or aws but just gcp.

[bluecrab]

Vector DBs should never have existed in the first place. I feel sorry for the agent startups though.

[m3kw9]

How does this absolve vectordbs

[Der_Einzige]

Startups built around actual AI tools, like if one formed around automatic1111 or oogabooga, would be unaffected, but because so much VC money went to the wrong places in this space, a whole lot of people are about to be burned hard.

[throwaway-jim]

damn hahaha it's oobabooga not oogabooga

[yawnxyz]

i'm excited for the open source, local inferencing tech to catchup. The bar's been raised.

[andrewjl]

None of those categories really fall under the second order category mentioned in the video. Using their analogy they all sound more like a mapping provider versus something like Uber.

[felixding]

Offtopic. I find it's amusing that we not only have "chatGPT" but now also "vectorDB". Apple's influence is really strong.

[mvkel]

Probably best not to make your company about features that a frontier AI company would have a high probability of adding in the next 6-12 months.

[bilsbie]

Why don’t you need embedding?

[monkeydust]

You might. Depends what your trying to do. For RAG seems like they can 'take care of it' but embeddings also offer powerful semantic search and retrieval ignoring LLMs.

[colordrops]

I haven't been paying attention, why are embeddings not needed anymore?

[sharemywin]

Retrieval: augments the assistant with knowledge from outside our models, such as proprietary domain data, product information or documents provided by your users. This means you don’t need to compute and store embeddings for your documents, or implement chunking and search algorithms. The Assistants API optimizes what retrieval technique to use based on our experience building knowledge retrieval in ChatGPT.

The model then decides when to retrieve content based on the user Messages. The Assistants API automatically chooses between two retrieval techniques:

it either passes the file content in the prompt for short documents, or performs a vector search for longer documents Retrieval currently optimizes for quality by adding all relevant content to the context of model calls. We plan to introduce other retrieval strategies to enable developers to choose a different tradeoff between retrieval quality and model usage cost.

[karmasimida]

Embedding is poor man's context length increase. It essentially increases your context length but with loss.

There is a cost argument to make still, embedding-based approach will be cheaper and faster, but worse result than full text.

That being said, I don't see how those embedding startups compete with OpenAI, no one will be able to offer better embedding than OpenAI itself. It is hardly a convincing business.

The elephant in the room is the open source models aren't able to match up to OpenAI models, and it is qualitative, not quantitive.

[lazzlazzlazz]

OP is incorrect. Embeddings are still needed since (1) context windows can't contain all data and (2) data memorization and continuous retraining is not yet viable.

[emadabdulrahim]

I believe their API can be stateful now: https://openai.com/blog/new-models-and-developer-products-an...

[treprinum]

There is not much info about retrieval/RAG in their docs at the moment - did you find any example on how is the retrieval supposed to work and how to give it access to a DB?

[baq]

Checking hn and product hunt a few times a week gives you most of that awareness and I don’t need to remind you about the person behind hn ‘sama’ handle.

[seydor]

more startups should focus on foundation models, it's where the meat is. Ideally there won't be a need for any startup as the platform should be able to self-build whatever the customer wants.

[echelon]

We don't want Open AI to win everything.

[cityzen]

Where is the part about embeddings?

[atleastoptimal]

There will be a lot of startups who rely on marketing aggressively to boomer-led companies who don't know what email is and hoping their assistant never types OpenAI into Google for them.