Google’s Abandoned Library of 700 Million Titles: Usenet(wired.com)
wired.com
Google’s Abandoned Library of 700 Million Titles: Usenet
http://www.wired.com/epicenter/2009/10/usenet/
15 comments
Well, Wired is no NGO either... and yet its article prompted Google to do something about it, as reported by nebula [http://www.wired.com/epicenter/2009/10/usenet_fix/]. A nice example of two "evil" companies benefiting the public good...
I must confess to being a brainwashed Googlevangelist, but anyway... if it wasn't for Google's (lucrative) actions, maybe non-profit projects such as the Internet Archive might not have saved those magtapes in time.
I must confess to being a brainwashed Googlevangelist, but anyway... if it wasn't for Google's (lucrative) actions, maybe non-profit projects such as the Internet Archive might not have saved those magtapes in time.
I used to participate in Usenet a lot, back in the mid 90s. Rather stupidly, I set the no archive header (http://en.wikipedia.org/wiki/X-No-Archive) on all my posts. Now I can only find myself quoted in replies.
Shame, really.
Shame, really.
Well, I didn't set the X-No-Archive header. And now there's no chance that I'll ever get elected to be President of the United States.
The privacy apocalypse is coming. People will stop caring, even more than they already have, and if someone can be elected* after repeatedly breaking campaign promises, then surely past behavior only matters as an excuse, rather than a reason.
* I don't have anyone specific in mind, here; politicians, all of 'em.
* I don't have anyone specific in mind, here; politicians, all of 'em.
Hmm, look at the glass half full. Someday when you're popular, people will find your Usenet remains and quote you on it. That's cool.
How about an X-No-Profits header?
Same here. Sheesh, remember when we all used to use our full names when posting online?
Actually that's a nice way of filtering mailing lists nowadays. Posts worth reading and posts signed with the full name are almost the same ones... If you missed something really interesting from other posts, it will most likely be quoted later on anyways.
There was this griefer mythomaniac in the Swedish groups, who sent cancels on others' messages all the time when he couldn't answer (I assume with constructed headers so it wouldn't be removed on your local server).
He also did things like send cancels on people quoting his insulting stuff long afterwards, so they wouldn't show up with his name in Google.
I had read the rfc:s, but couldn't even imagine anyone would be so unserious at the time. Ah, innocence.
I don't know if Google still remove messages when cancels show up on Usenet, but you could try...
Edit: OK, I can't complain about the downvotes; it was a joke but it is arguably immoral to remove information.
He also did things like send cancels on people quoting his insulting stuff long afterwards, so they wouldn't show up with his name in Google.
I had read the rfc:s, but couldn't even imagine anyone would be so unserious at the time. Ah, innocence.
I don't know if Google still remove messages when cancels show up on Usenet, but you could try...
Edit: OK, I can't complain about the downvotes; it was a joke but it is arguably immoral to remove information.
Sorry! - Downvote was iPhone fatfingered mistake.
(For a group so enthusiastic about version control with a leader who pushed for etherpad's timeline document history, this one vote then lockin feels a bit out of character)
(For a group so enthusiastic about version control with a leader who pushed for etherpad's timeline document history, this one vote then lockin feels a bit out of character)
I upvoted to compensate. Good on you for explaining what happened.
An extra bit of room between those arrows would go a long way to solving that I think.
It would if you think I was going for the up arrow - I was actually going for the reply link of the parent post (yes, I had already zoomed in).
Shame? You think I want my offspring to know what my 20 year old-self posted on Usenet? Does anyone? :-)
The sad thing is that if Google had had the foresight to build in decent browsing, filtering etc., and polish up NNTP a bit, they could own the vast internet message board space, and we would be reading this on alt.hacking.ycombinator or suchlike.
It's not that people aren't interested in chatting to each other on the internet, I waste depressingly large quantities of time reading and posting in 5-6 different forums, this being one of them. There's still one usenet group I read regularly, but it's degraded to 50% spam and embittered ranting from an unhappy kibo clone.
It's not that people aren't interested in chatting to each other on the internet, I waste depressingly large quantities of time reading and posting in 5-6 different forums, this being one of them. There's still one usenet group I read regularly, but it's degraded to 50% spam and embittered ranting from an unhappy kibo clone.
There've been a bunch of companies that have tried to do just that - EZBoard comes to mind.
The problem is that individual communities almost always want to control the look & feel of their forums. So as they start growing, they eventually outgrow the hosted, consistent UI and get something like vBulletin that they can customize themselves. The biggest ones end up ditching third-party forums entirely and writing custom software that's integrated with the rest of their site.
Forums are not a natural monopoly - there's no economic reason for one company to "own" them. We have the option of creating a Google Group that's basically alt.hacking.ycombinator with a prettier name. Several hacking groups do just that. But this community is big enough that it has special needs, so news.YC arises naturally anyway.
The problem is that individual communities almost always want to control the look & feel of their forums. So as they start growing, they eventually outgrow the hosted, consistent UI and get something like vBulletin that they can customize themselves. The biggest ones end up ditching third-party forums entirely and writing custom software that's integrated with the rest of their site.
Forums are not a natural monopoly - there's no economic reason for one company to "own" them. We have the option of creating a Google Group that's basically alt.hacking.ycombinator with a prettier name. Several hacking groups do just that. But this community is big enough that it has special needs, so news.YC arises naturally anyway.
I would much prefer to have all my web message boards available through a single interface, in a single program.
I haven't seen any web message board UI that truly brings something new to the threaded-discussion style that pretty much all the old Usenet clients used.
Which is a shame, really, because I'd much prefer to read blogs via an NNTP reader (I suspect, anyway).
I haven't seen any web message board UI that truly brings something new to the threaded-discussion style that pretty much all the old Usenet clients used.
Which is a shame, really, because I'd much prefer to read blogs via an NNTP reader (I suspect, anyway).
I'd much prefer to read blogs via an NNTP reader
If you use Thunderbird RSS reader to read blogs, you pretty much get the Same look and feel. I was used to Thunderbird NNTP client, when I started adding blog subscriptions to Thunderbird, the transition was seamless
If you use Thunderbird RSS reader to read blogs, you pretty much get the Same look and feel. I was used to Thunderbird NNTP client, when I started adding blog subscriptions to Thunderbird, the transition was seamless
"... they could own the vast internet message board space, and we would be reading this on alt.hacking.ycombinator or suchlike. ..."
alt.games.programming used to be one such place.
alt.games.programming used to be one such place.
In the States, Verizon and Comcast have stopped including Usenet because of kiddie porn concerns (or it might have been an excuse), that's probably 70% of why it's moribund.
Google Groups used to be a great service when they first took over Deja News, but things have been gradually deteriorating ever since.
They used to be read-only; now, they allow anyone with a fake account to post. They are a huge source of Usenet spam. I have been kill-filing posts from Google Groups for about a year. It's not a pleasant thing to do, since there still are quite a few people who I like to read but only see quoted in replies now because they use Google Groups.
Whenever someone posts the same article 10 times in a row on a Usenet newsgroup, I always look at the mail header, and every single time it is from Google Groups. I don't know what is wrong, but there must be some user interface bug that causes people to resubmit their post. Maybe they don't realize that Usenet has a lag between posting and seeing your post. Maybe there's something wrong with the submit button. I don't know, but I've been seeing this phenomenon for a few years now.
Initially, Google Groups made a clear distinction between Usenet newsgroups and Google's own groups, but they've been blurring the two more and more over the years. The result is that a lot of people don't even know they're posting to Usenet, and are unaware of the different etiquette and social customs there (real names, top posting, etc.). Lately they've started including third-party forums as well, so now it's basically a free-for-all, and there is no way to just search Usenet anymore. You have a choice between "Google Groups" and "all groups", where the latter includes Google Groups, Usenet, and random bulletin boards on the web.
If Google isn't interested in supporting Usenet with a decent archive anymore (and I can understand their reluctance, since I'm sure there's little to no money in it) I wish they would pass the responsibility to someone else and get out of the way, because at this point I wish Deja News was still around - I'd have no need for Google Groups anymore.
And I know everyone loves to bash on Usenet, about how it's obsolete and dead, but I have to say that at least when it comes to programming language newsgroups, it's still hopping. I follow about a dozen programming newsgroups, and I can't keep up. About once a week I have to "mark all as read". As long as it still causes information overload, I don't think it can be fairly declared extinct.
They used to be read-only; now, they allow anyone with a fake account to post. They are a huge source of Usenet spam. I have been kill-filing posts from Google Groups for about a year. It's not a pleasant thing to do, since there still are quite a few people who I like to read but only see quoted in replies now because they use Google Groups.
Whenever someone posts the same article 10 times in a row on a Usenet newsgroup, I always look at the mail header, and every single time it is from Google Groups. I don't know what is wrong, but there must be some user interface bug that causes people to resubmit their post. Maybe they don't realize that Usenet has a lag between posting and seeing your post. Maybe there's something wrong with the submit button. I don't know, but I've been seeing this phenomenon for a few years now.
Initially, Google Groups made a clear distinction between Usenet newsgroups and Google's own groups, but they've been blurring the two more and more over the years. The result is that a lot of people don't even know they're posting to Usenet, and are unaware of the different etiquette and social customs there (real names, top posting, etc.). Lately they've started including third-party forums as well, so now it's basically a free-for-all, and there is no way to just search Usenet anymore. You have a choice between "Google Groups" and "all groups", where the latter includes Google Groups, Usenet, and random bulletin boards on the web.
If Google isn't interested in supporting Usenet with a decent archive anymore (and I can understand their reluctance, since I'm sure there's little to no money in it) I wish they would pass the responsibility to someone else and get out of the way, because at this point I wish Deja News was still around - I'd have no need for Google Groups anymore.
And I know everyone loves to bash on Usenet, about how it's obsolete and dead, but I have to say that at least when it comes to programming language newsgroups, it's still hopping. I follow about a dozen programming newsgroups, and I can't keep up. About once a week I have to "mark all as read". As long as it still causes information overload, I don't think it can be fairly declared extinct.
It might be worth it to point out that a lot of projects' mailing lists are mirrored on Usenet. I'm not 100% sure if any or all of them accept posts from Usenet, but I'm fairly certain that at least a few of them do.
i.e. I believe that comp.lang.python is the same as the python-list mailing list.
i.e. I believe that comp.lang.python is the same as the python-list mailing list.
When google first put the archive online, I could find 100's of posts I made in the 80's. Now, there are a few 10's. At this point, I don't even use the search for old stuff, assuming that the bits have just rotted so much that it's not worth it.
I had noticed problems a few weeks ago, but it seems to be working today.
http://groups.google.com/group/alt.hypertext/search?group=al...
And the advanced search always worked and still works better. http://groups.google.com/advanced_search?q=&sitesearch=g...
And the advanced search always worked and still works better. http://groups.google.com/advanced_search?q=&sitesearch=g...
Thanks for the tip about advanced search. That brought back memories of a decade ago when I was very active on Usenet. All the messages I was expecting to find appear to be there.
(Update: Google has begun fixing the Usenet archive in response to this article)Good info.
Any pointers to details?
Any pointers to details?
Google Begins Fixing Usenet Archive
http://www.wired.com/epicenter/2009/10/usenet_fix/
http://www.wired.com/epicenter/2009/10/usenet_fix/
Time to take back Usenet and build another archive independently of Google, somehow.
Machiavelli warned Princes about the danger of being an 'unarmed prophet.'
He might similarly warn you about being an 'unfunded revolutionary.'
He might similarly warn you about being an 'unfunded revolutionary.'
I'm not name dropping, but I have exchanged emails with Brewster Kahle of the Internet Archive, and he's taken a couple of my ideas under advisement.
Simon Bolivar was an 'unfunded revolutionary.' Under-funded for many of his campaigns, anyways. His exploits show that one can also derive significant energy from a general environment of horrendous suck.
[deleted]
The biggest problem would seem to be either liberating the raw data from Google or finding someone who still has readable archives.
Google has a 'captcha' on Google Groups which prevents mass copying. There is a guy with those tapes, but they're the early 80s - I don't know where Dejanews got their original archives. But that guy would be a start.
When I had a Giganews account I looked at how far back their text groups went, and I think they went back to 2003, but that was a couple of years ago. Might be worth looking at EasyNews or Giganews, at least for more recent history.
Or just wait for Google to fix it, I guess.
Uh,
The Google books settlement gives Google something like a monopoly on "orphaned" titles, titles that would otherwise be entirely unavailable. Whether that's justified or not is debatable.
But unless there's something I don't know, Google Groups is just one archive of usenet and anyone with enough space could create another - in fact, I seem encounter other such archives regularly in my ... Google searches.
The Google books settlement gives Google something like a monopoly on "orphaned" titles, titles that would otherwise be entirely unavailable. Whether that's justified or not is debatable.
But unless there's something I don't know, Google Groups is just one archive of usenet and anyone with enough space could create another - in fact, I seem encounter other such archives regularly in my ... Google searches.
It isn't just the usenet archive search that sucks -- it includes pretty much anything under the google groups banner. I'm often searching groups dedicated to projects under google code, and the search results are dismal more often than not.
Upvoted to show support for the article and its point. I've once wondered into Google's Usenet jungle, it's a terrible mess.
A bit off-topic, but I just read the features list for mosaic (http://groups.google.com/group/alt.hypertext/msg/7fde2f6d4d5...) and was really surprised how close it is to browsers nowadays. The only serious improvements that current firefox has over that mosaic are: tabs, JS and blocking stuff. (I'm not counting stuff that's just a simple improvement - like rendering new html+css, ability to use icons for favourite pages, etc.)
It's a bit sad :(
It's a bit sad :(
What exactly were you hoping for?
It looks like they already fixed the search. I can find my own embarrassing posts from when I was a teenager. I had forgotten how excited I was about computer games. :P
They may claim that they've fixed it, but they haven't.
Try to search for old posts by USENET group and with a fixed e-mail address - it will not work properly. What doesn't work:
- search by e-mail address,
- search by USENET group, e.g., comp.lang.perl.misc,
- you must log in to get any reasonable functionality.
Why don't they give it back to the Deja people along with sufficient funding? They were doing it just fine, thank you. Wouldn't worry me if _all_ Google Groups posts were trashed and USENET restored to it's primary role.
USENET was once the most important Internet resource to me and the only one I would have paid for. It's been seriously damaged under Google's reign. Heads really, really should roll over this.
Try to search for old posts by USENET group and with a fixed e-mail address - it will not work properly. What doesn't work:
- search by e-mail address,
- search by USENET group, e.g., comp.lang.perl.misc,
- you must log in to get any reasonable functionality.
Why don't they give it back to the Deja people along with sufficient funding? They were doing it just fine, thank you. Wouldn't worry me if _all_ Google Groups posts were trashed and USENET restored to it's primary role.
USENET was once the most important Internet resource to me and the only one I would have paid for. It's been seriously damaged under Google's reign. Heads really, really should roll over this.
http://news.ycombinator.com/item?id=870126
In reference.
In reference.