[talos_]

This analogy doesn't map to the actual problem here.

Perplexity is not visiting a website everytime a user asks about it. It's frequently crawling and indexing the web, thus redirecting traffic away from websites.

This crawling reduces costs and improves latency for Perplexity and its users. But it's a major threat to crawled websites

[shadowgovt]

I have never created a website that I would not mind being fully crawled and indexed into another dataset that was divorced from the source (other than such divorcement makes it much harder to check pedigree, which is an academic concern, not a data-content concern: if people want to trust information from sources they can't know and they can't verify I can't fix that for them).

In fact, the "old web" people sometimes pine for was mostly a place where people were putting things online so they were online, not because it would translate directly to money.

Perhaps AI crawlers are a harbinger for the death of the web 2.0 pay-for-info model... And perhaps that's okay.

[short_sells_poo]

There's an important distinction that we are glossing over I think. In the times of the "old web", people were putting things online to interact with a (large) online audience. If people found your content interesting, they'd keep coming back and some of them would email you, there'd be discussions on forums, IRC chatrooms, mailing lists, etc. Communities were built around interesting topics, and websites that started out as just some personal blog that someone used to write down their thoughts would grow into fonts of information for a large number of people.

Then came the social networks and walled gardens, SEO, and all the other cancer of the last 20 years and all of these disappeared for un-searchable videos, content farms and discord communities which are basically informational black holes.

And now AI is eating that cancer, but IMO it's just one cancer being replaced by an even more insidious cancer. If all the information is accessed via AI, then the last semblance of interaction between content creators and content consumers disappears. There are no more communities, just disconnected consumers interacting with a massive aggregating AI.

Instead of discussing an interesting topic with a human, we will discuss with AI...