I think Archive.org wayback machine and archive.today are going to be extremely valuable resources for "historical research" in the future. I hope they can continue their work via donor funding.
Are there any developments towards helping these efforts out by decentralizing / backing up parts of the archive and way back machine data?
I know torrents are hugely important in helping to decentralize and maintain many important files in academia and such (e.g. NN weights and CERN data) but I think cephfs is also trying to allow decentralized data storage with redundancy.
It seems like there is some solution which can provide a huge data source to be decentralized over arbitrary number of nodes, where each node can hold or back up just some part of the data, and allow for a dashboard view that shows the level of redundancy over all of the data for each of its parts.
I worry that archive.org or wayback will not have sufficient funding and will need to close, and the sheer impact of that is greater than most realize. I hope there is a decentralized archive project
chaxor|2 years ago
It seems like there is some solution which can provide a huge data source to be decentralized over arbitrary number of nodes, where each node can hold or back up just some part of the data, and allow for a dashboard view that shows the level of redundancy over all of the data for each of its parts.
throwoutway|2 years ago