top | item 36416617

(no title)

ez_mmk | 2 years ago

Can somebody put this on the Internet Archive?

discuss

pabs3|2 years ago

On its way via ArchiveBot:

http://archivebot.com/ https://wiki.archiveteam.org/index.php/ArchiveBot

Lammy|2 years ago

I've grown wary of relying only on IA

  [lammy@popola#Webrips] du -hs www.rr-fallenflags.org 
   43G    www.rr-fallenflags.org
  [lammy@popola#Webrips] find www.rr-fallenflags.org | wc -l
    479221

usr1106|2 years ago

Will that work? The site mentions that it's not indexed by search machines. So I guess it has a robots.txt (on my phone and did not check...) Would the archive respect that?

Jefenry|2 years ago

Annoyingly, Internet Archive doesn't respect robots.txt. I specifically excluded ia_archiver from my site which worked for a number of years until they decided to ignore it because robots.txt "do not necessarily serve our archival purposes." They do remove your site if you email them though.

https://www.digitaltrends.com/computing/internet-archive-rob...