I wish google would let me crawl blogger pages for my index. I don't think that is fair that google can index the web, but has created a walled garden for the content they host.
Basically, if you start indexing the web, and some sites are hosted in blogger, google will block your crawler if I remember correctly
wow. that's really insightful. thanks for the post. I guess you're right, there is room for improvement, but . . .
I wonder if the failure rate is higher, because those ideas are common, and everyone tries it. Yeah maybe the same number will succeed, but since everyone and their mom is doing it, it probably won't be you . . . dunno.
wow. that's an entire module in the system. we do several things that analyze feeds and decide if it fits a category. Basically, lots of text analysis and analysis of the web.
Yeah, we don't have content ourselves, our value add is the ranking and filter service. Also, allowing you to see the conversation across the web.
UGC would be tough, because I'm at a chicken and egg state right now as you pointed out. And crap comments would lead to more crap comments. Imagine if Quora or Hacker News had a comment bot. Sets a bad tone for the future.
i guess there may be a flaw in my initial strategy.
when training my algorithm (and as i make iterations), I compare the results to google news to make sure we're as good if not better. That's why a lot of the initial top-level categories match up pretty closely.
yeah. the algorithm I built takes a little training per topic/vertical, but after the initial training, it just starts learning.
I have a long list of potential topics that the algorithm would be good at. pretty much any sports topic it would kick butt on.
My brother keeps telling me I need to split up sports too. Basically, the initial verticals were to stress out the algorithm to make sure it would work on various concepts and still keep things interesting.
Basically, you could imagine every concept http://alltop.com/ tracks running through my algorithms.
Thanks for the feedback though. That's the kind of information that really helps a hacker getting started.