top | item 22181946 (no title) vectorEQ | 6 years ago i'm wondering if that robots.txt might then get you sued due to blocking scrapers / bots? discuss order hn newest herval|6 years ago A robots.txt file doesn’t block scrapers, it’s the equivalent of a no trespassing sign. I don’t think putting up signs will suddenly become illegal? setr|6 years ago Robots.txt doesn't interfere with anything. It's a suggestion.
herval|6 years ago A robots.txt file doesn’t block scrapers, it’s the equivalent of a no trespassing sign. I don’t think putting up signs will suddenly become illegal?
herval|6 years ago
setr|6 years ago