Finding this sort of thing always makes me stop short and realize, "Oh yeah, no matter how smart I think I am, there are people a million times more accomplished than me on HN." Not to make me despair so much as take care to avoid blathering the first thing on my mind before reading and considering what others have said.
Ups to you! I think (and hope) that we haven't seen the end of magnet yet. I wish the scientific community would start using magnet links for paper references, instead of archaic bibtex layouts combined with online publishers who charge 30 dollars without adding value.
When multiple downloads of the same file are available it's usually beneficial to see and select the one with the highest number of seeds. If DHTs become the norm, will it still be possible to obtain and use such data?
> Perhaps even better, without the torrent files everyone can soon host a full copy of The Pirate Bay on a USB thumb drive, which may come in handy in the future.
I've been saying this for years: governments playing an arms race with hackers is like playing Whack-a-mole or cutting of heads of hydras. Every time you block one means of communication (starting in this case with Napster), a new, more decentralized, harder-to-combat protocol is going to emerge. Even if you're the MPAA and don't like copyright violations, you have to face that fact. That doesn't mean you (necessarily) have to throw in the towel and abandon the idea of copyright infringement altogether, but it does mean you need to start being creative instead of engaging in a direct legal-technological battle - an arms race.
I wonder how long it will take before the big players realize this and try to figure out a way to use this new playing field (the Internet) to their advantage, instead of trying to squelch any technological development so that they can cling to old models of payment and distribution. It was nice while it happened, but let's face it - we're past the point of no return. Even if they everyone hosted their own copies of TPB on their thumb drives and then they found some way to shut that down, I'm certain there'd be some hacker smart enough (like gojomo above) to come up with something that just makes things even less centralized, more difficult to track, and more difficult to shut down.
It is still fascinating how resilient not only Piratebay but also others such as Demonoid have become. A few years ago it looked like they were close to getting shut down. It is impressive how they are now able to withstand all the "forces" and Governments had to shift their focus towards the ISPs.
One drawback to magnet links is that you can not in advance see what files are there.
So if you only want to download some of them, you first have to wait for the magnet to download the torrent, then go back to it and pick the files you want.
A magnet link also makes it hard to check if the link you are looking at is a duplicate of what you have already.
With a torrent you can check the file size and compare to others.
Can someone fill me in on the first step a client (BT) would take to find its first peers? That part is a bit magic to me right now. What would be the first IP it would query and how would it know?
There are three main methods (in order of preference):
1. Use a cached list from a previous run. This is ideal in terms of load balancing, and also helps with bootstrap time if your IP hasn't changed, so all major torrent clients have such a cache.
2. Ask peers you're connected to via a traditional tracker and .torrent file. This is the next best thing, but can only be used if the first torrent you download is via a traditional tracker.
3. Use some bootstrap peers hardcoded into the torrent client. Has a single point of failure, but it works.
Each torrent client can use a different set of bootstrap peers, note - they're nothing special, just a DHT peer that has the bandwidth and CPU power to deal with bootstrap requests (which isn't a very high hurdle - you only get hit once, on the initial install, or if the client's been shut down for so long it no longer has any valid peers). Or the user might even supply their own, if need be.
If the TPB were to actively oppose SOPA, it would just give the SOPA proponents more to work with - "Look, TPB is against it, so it must be a good thing!".
So, what if you don't run the client on the same machine you're browsing TPB with, like people who have torrent enabled routers or VPS/seedboxes? I suppose you can copy-paste the link to your client, but clients which had automatic pick-up of .torrent files (like rtorrent) were nice because you could just drop them on a remote directory and have them be downloaded.
I wonder if I could write a small application just to download the torrent file from the magnet link/DHT to copy it to the remote server afterwards.
I suspect it'll be easier to patch the torrent client to accept magnet links fed through a named pipe, or in a text file in the drop-off directory, if it already supports magnet links.
Why not set up an email fetch service on the remote box, and email magnet links? Then just throw together a wrapper that looks for magnet links in new email. Viola!
They announced that they were planning on doing this right when magnet links came out. I haven't used .torrent files since I discovered magnet links and I actually find they more convenient than downloading a torrent file.
I've been using mostly magnet links for the past year and haven't experienced any issues. If people understand that they work just like a normal link/torrent file, this won't make the process any more complicated. Hopefully this is another win in the long run, as links are harder to stop than files.
Only slightly as I understand it, since from a legal perspective the intent of someone providing providing links instead of files doesn't change, and that's what counts.
From a technical perspective, and as someone who doesn't understand how magnet works under the hood, I'm slightly concerned that DHTs might be easier to attack in an underhanded manner than an HTTP server would have been.
Talking about magnet links I remembered KAD with ed2k and eMule. Its almost same. Bittorrent with magnet links just have different chunk size possibility to make faster download with smaller piece size.
From what I understand, DHT has a big trade-off vs BitTorrent: DHTs are crawlable[1] and copyright holders can more easily track who holds copies of what yet it's easier to duplicate search engines like TPB within hours for the same reason.
So the effect seems to be that the RIAA, MPAA, etc. will likely not be able to take down trackers; they'll have to revert back to suing their "customers" (or lobbying to pass absurd legislation for that matter).
It seems that TBP just taken another step to push judge and jury's confusion during next trials even further.
"Y'know, we're trying to shut down pirates' secret base that... doesn't serve a single file!"
What I’m wondering is whether it’s not yet possible to have a distributed, decentralised torrent database.
You could already put up a copy of the database as a torrent and distribute the magnet link, but you’d need some method for efficiently keeping it up to date.
DHTs/Torrents are great for static data (like a movie) but bad for dynamic data, like a website with a list of movies, their ratings, user comments, etc.
There are, however, other systems that are designed to combat this, like freenet. They tend to be overwhelmingly slow, because you need to pass lots of data around to make it consistent.
Then again, magnet links, titles, and a little html are probably not a lot of information, so it probably could be done. I just haven't heard of any attempts yet. It's tempting to go and write one. Could you make a decentralized, P2P version of reddit with distributed trust? I think it's possible but hasn't been tried.
Since they are doing this big change, is there a way to make it more secure on the user side, too? Like encrypt the traffic and make it impossible for RIAA to track IP's?
Also since they say that it's like every user would have the TPB site on their computer, does it mean blocking the site would be completely useless? And since they are just links, and links are pretty much speech, I figure it would be impossible to turn it into law as well, to specifically target magnet based sites like TPB.
The entire design of Bittorrent is predicated on clients advertising to each other which torrents they're seeding and which pieces are available. You can encrypt your traffic to get it past your ISP's deep packet inspection, but the peer at the other end has to be able to decrypt it. And you have no way of knowing what nefarious organization controls that peer.
Don't the torrent files contain all the hashes for each piece? Doesn't that mean if a single piece is bad, the entire torrent can't be verified? Torrent files contain a lot of useful data that isn't found in a magnet URI. Is this just being ignored?
The hash in a magnet URI is of the info dictionary in the .torrent file. It's used to query the BitTorrent DHT for peers running the same torrent. Once another peer is found, your client will download the full metadata file from it, and from there it works just like any other BitTorrent download.
This is definitely a step in the right direction. Not saying this is a good step to increase piracy, but there really is no need for trackers when we have magnets, and this will lead to a healthier bittorent community.
[+] [-] gojomo|14 years ago|reply
Great to see it still evolving and spreading, based simply on its loosey-goosey merits.
[+] [-] invalidOrTaken|14 years ago|reply
[+] [-] nestlequ1k|14 years ago|reply
But when I click on the magnets things seem to download immediately. So looks like you built something pretty amazing. Congrats!
[+] [-] skrebbel|14 years ago|reply
[+] [-] noch|14 years ago|reply
[+] [-] nodata|14 years ago|reply
[+] [-] chimeracoder|14 years ago|reply
I've been saying this for years: governments playing an arms race with hackers is like playing Whack-a-mole or cutting of heads of hydras. Every time you block one means of communication (starting in this case with Napster), a new, more decentralized, harder-to-combat protocol is going to emerge. Even if you're the MPAA and don't like copyright violations, you have to face that fact. That doesn't mean you (necessarily) have to throw in the towel and abandon the idea of copyright infringement altogether, but it does mean you need to start being creative instead of engaging in a direct legal-technological battle - an arms race.
I wonder how long it will take before the big players realize this and try to figure out a way to use this new playing field (the Internet) to their advantage, instead of trying to squelch any technological development so that they can cling to old models of payment and distribution. It was nice while it happened, but let's face it - we're past the point of no return. Even if they everyone hosted their own copies of TPB on their thumb drives and then they found some way to shut that down, I'm certain there'd be some hacker smart enough (like gojomo above) to come up with something that just makes things even less centralized, more difficult to track, and more difficult to shut down.
[+] [-] Helianthus|14 years ago|reply
It's away from exploitation of artists and the control of artistic message.
[+] [-] Corrado|14 years ago|reply
[+] [-] ajtaylor|14 years ago|reply
[+] [-] stfu|14 years ago|reply
[+] [-] ars|14 years ago|reply
So if you only want to download some of them, you first have to wait for the magnet to download the torrent, then go back to it and pick the files you want.
A magnet link also makes it hard to check if the link you are looking at is a duplicate of what you have already.
With a torrent you can check the file size and compare to others.
[+] [-] ben0x539|14 years ago|reply
[+] [-] atlbeer|14 years ago|reply
[+] [-] bdonlan|14 years ago|reply
1. Use a cached list from a previous run. This is ideal in terms of load balancing, and also helps with bootstrap time if your IP hasn't changed, so all major torrent clients have such a cache.
2. Ask peers you're connected to via a traditional tracker and .torrent file. This is the next best thing, but can only be used if the first torrent you download is via a traditional tracker.
3. Use some bootstrap peers hardcoded into the torrent client. Has a single point of failure, but it works.
Each torrent client can use a different set of bootstrap peers, note - they're nothing special, just a DHT peer that has the bandwidth and CPU power to deal with bootstrap requests (which isn't a very high hurdle - you only get hit once, on the initial install, or if the client's been shut down for so long it no longer has any valid peers). Or the user might even supply their own, if need be.
[+] [-] tomkin|14 years ago|reply
[+] [-] bdonlan|14 years ago|reply
[+] [-] icebraining|14 years ago|reply
I wonder if I could write a small application just to download the torrent file from the magnet link/DHT to copy it to the remote server afterwards.
[+] [-] botker|14 years ago|reply
I tested it, and it works as advertised.
For firefox integration, in about:config create a new boolean as:
For chromium integration, you need to use Xdg-open.[+] [-] ben0x539|14 years ago|reply
[+] [-] sliverstorm|14 years ago|reply
[+] [-] iamandrus|14 years ago|reply
[+] [-] Zirro|14 years ago|reply
[+] [-] forgotusername|14 years ago|reply
From a technical perspective, and as someone who doesn't understand how magnet works under the hood, I'm slightly concerned that DHTs might be easier to attack in an underhanded manner than an HTTP server would have been.
[+] [-] GBiT|14 years ago|reply
[+] [-] Hemospectrum|14 years ago|reply
[+] [-] jmtame|14 years ago|reply
So the effect seems to be that the RIAA, MPAA, etc. will likely not be able to take down trackers; they'll have to revert back to suing their "customers" (or lobbying to pass absurd legislation for that matter).
[1] http://z.cs.utexas.edu/users/osa/unvanish/papers/vanish-brok...
[+] [-] Fester|14 years ago|reply
[+] [-] sjmulder|14 years ago|reply
You could already put up a copy of the database as a torrent and distribute the magnet link, but you’d need some method for efficiently keeping it up to date.
[+] [-] jxcole|14 years ago|reply
There are, however, other systems that are designed to combat this, like freenet. They tend to be overwhelmingly slow, because you need to pass lots of data around to make it consistent.
Then again, magnet links, titles, and a little html are probably not a lot of information, so it probably could be done. I just haven't heard of any attempts yet. It's tempting to go and write one. Could you make a decentralized, P2P version of reddit with distributed trust? I think it's possible but hasn't been tried.
[+] [-] pornel|14 years ago|reply
Somebody did that last time PB was in trouble, e.g. one of the pieces: magnet:?xt=urn:btih:4232363a47fe29acdf2c77874365a5e3368854b4&
That's a pretty interesting dataset to mine.
[+] [-] chimeracoder|14 years ago|reply
[+] [-] ward|14 years ago|reply
Is there a list of countries where this has been ordered by courts? I know it's already the case in my country (Belgium) as well[1].
[1]: http://torrentfreak.com/belgium-starts-blocking-the-pirate-b...
[+] [-] nextparadigms|14 years ago|reply
Also since they say that it's like every user would have the TPB site on their computer, does it mean blocking the site would be completely useless? And since they are just links, and links are pretty much speech, I figure it would be impossible to turn it into law as well, to specifically target magnet based sites like TPB.
[+] [-] teraflop|14 years ago|reply
[+] [-] afhof|14 years ago|reply
[+] [-] skymt|14 years ago|reply
http://bittorrent.org/beps/bep_0005.html http://bittorrent.org/beps/bep_0009.html
[+] [-] sp0rus|14 years ago|reply
[+] [-] joejohnson|14 years ago|reply
[+] [-] Refringe|14 years ago|reply