top | item 42497297

(no title)

whoistraitor | 1 year ago

Perplexity is listed, but do they actually abide by llms.txt? And how can we prove they do? Is it all good faith? I wish there were a better way.

discuss

order

jsheard|1 year ago

llms.txt isn't an opt-out signal like robots.txt, it's a way to provide a simplified version of pages that are easier for LLMs to ingest. It's more of an opt-in for being scraped more accurately.

weare138|1 year ago

Or scraped inaccurately. It seems like you could have some fun with this if you were so inclined...