[JimsonYang]

short answer it depends and idk. When I was doing some testing with prompts like "what should I have for dinner" adding variations, "hey ai, plz, etc" doesn't deviate intention much. As ai is really good at pulling intent. But obv if you say "i'm on keto what should i have for dinner" it's going to ignore things like "garlic, pesto, and pasta noodles". Although it pulls a similar response to "what's a good keto dinner". From there we really assume the user can know their customers what type of prompts led them to chatgpt. You might've noticed sites asking if you came from chatgpt, i would take that a step further and asked them to type the prompt they used.

But you do bring a good perspective because not all prompts are equal especially with personaliztion. So how do we solve that problem-I'm not sure. I have yet to see anything in the industry. The only thing that came close was when a security focused browser extension started selling data to aeo companies- that's how some companies get "prompt volume data".

[n_u]

I see what you are saying, perhaps no matter the conversation before as long as it doesn't filter out some products via personalized filters (e.g. dietary restrictions) it will always give the same answers. But I do feel the value prop of these AI chatbots is that they allow personalization. And then it's tough to know if 50% of the users who would previously have googled "best running shoes" instead now ask detailed questions about running shoes given their injury history etc and that changes what answers the chatbot gives.

I feel like without knowing the full distribution, it's really tough to know how many/what variations of the query/conversation you need to sample. This seems like something where OpenAI etc. could offer their own version of this to advertisers and have much better data because they know it all.

Interesting problem though! I always love probability in the real world. Best of luck, I played around with your product and it seems cool.