akriukas's comments

Same here.

I don't get what these "done()" mean. But otherwise, the logic looks good.

Very useful for some big eshops if they list their sitemaps. However not all stores have sitemaps and even robots.txt site...