WingNews logo WingNews
top | item 46918289

(no title)

neoromantique | 23 days ago

Ask HN: How does one archive websites like this without being a d-ck?

I want to save this for offline use, but I think recursive wget is a bit poor manners, is there established way one should approach it, get it from archive somehow?

discuss

order

prmoustache|23 days ago

As long as you don't mirror daily and use rate limit there is no reason you would be a dick doing it.

FWIW I have a local copy of sheldown brown's website I mirrored a few years back when they announced the shop would close as I expected they would eventually shutdown the website too. I don't know if his wife is still alive, she had her own space nor if someone has taken over the maintenance.

OJFord|23 days ago

A single user's one-off recursive wget seems fine? Browsers also support it iirc, individual pages at very least (and saved to the same place, the links will work).

No doubt it's already in many archive sites though, you could just fetch from them instead of the original?

neoromantique|23 days ago

I ask in more general sense, if there is a way to fetch this stuff directly from webarchive or something along those lines.

Gotta hit the search I feel :)

ssl-3|23 days ago

In the old-web days, I just used wget with slow pacing (and by "pacing" I mean: I don't need it to be done today or even this week, so if it takes a rather long time then that's fine. Slow helped keep me from mucking up the latency on my dial-up connection, too.)

I don't think that's being a dick for old-web sites that still exist today. Most of the information is text, the photos tend to be small, it's all generally static (ie, light-weight to serve), and the implicit intent is for people to use it.

But it's pretty slow-moving, so getting it from archive.org would probably suffice if being zero-impact is the goal.

(Or, you know: Just email the dude that runs it like it's 1998 again, say hi, and ask. In this particular instance, it's still being maintained.)