top | item 44788232 (no title) hundchenkatze | 6 months ago You left out the part that says Perplexity-User generally ignores robots.txt because it's used for user requested actions.> Since a user requested the fetch, this fetcher generally ignores robots.txt rules. discuss order hn newest No comments yet.
No comments yet.