I am thinking of releasing new versions once a week and putting the hash of the torrent of the newest version on some public site. (Say, some twitter account.)
But it would still be more proof of concept than really anything useful - the comments and descriptions ARE important.
edit: More I am thinking about it, the less useful it sounds.
First, the information about seeders vary constantly, especially with the new torrents.
Also, it STILL depends on single point of failure - the Pirate Bay itself. If TPB will be down for any reason, I will have no place to scrape this from and it will all fall apart anyway.
Plus, I think Pirate Bay itself should make dumps like this. It would probably be much better for their database anyway :)
The thing is, new torrents are uploaded to Pirate Bay all the time, so one can only archive TPB in any given moment - which has to be, of course, before the creation of this torrent.
The archive is static, the TPB database is dynamic :)
Hello, I am an author of the scrape. I did it more to try it, but who knows, maybe it will be useful to someone.
I went trough the description pages like http://thepiratebay.se/torrent/$i by increasing the $i and saving the magnet if pirate bay didn't return 404 error. I went trough the pages as unlogged user, though, so that might be the reason why I got only 1.5m torrents.
I didn't know pirate bay has hidden porn torrents; there is TONS of porn in the scrape already.
The script is in perl, I will post it to pastebin in a moment.
Just one thing: I see you are splitting by |, and some torrents (very few, but some) have | in their name (I didn't bother with escaping that).