[fpgaminer]

Maybe this is an insane idea, but ... how about a spider P2P network?

At least for local AIs it might not be a terrible idea. Basically a distributed cache of the most common sources our bots might pull from. That would mean only a few fetches from each website per day, and then the rest of the bandwidth load can be shared amongst the bots.

Probably lots of privacy issues to work around with such an implementation though.