Blocked Sites is discontinued(support.google.com)
support.google.com
Blocked Sites is discontinued
http://support.google.com/websearch/bin/answer.py?hl=en&answer=1210386
140 comments
Personal Blacklist is an official Google extension, it just doesn't sync with anything or save to a database. At least for my purposes, I don't need it anywhere but my desktop machine to help out in result pruning (experts exchange, fixya).
Their offered download link doesn't even work for me, it just notifies me of its shutdown.
Their offered download link doesn't even work for me, it just notifies me of its shutdown.
>Their offered download link doesn't even work for me, it just notifies me of its shutdown.
After you go to that link and read the notification, there is another link to download the blocklist as a plain-text file. Mine contained a number of domains I've added during the years.
It's quite interesting to see all the domains I've blocked so far :)
After you go to that link and read the notification, there is another link to download the blocklist as a plain-text file. Mine contained a number of domains I've added during the years.
It's quite interesting to see all the domains I've blocked so far :)
Mine was surprisingly short..
www.w3schools.com
www.w3schools.com
Well I had all those sites that scrape StackOverflow added to my list.
I love that SO uses a CC license but it does have some unintended consequences.
I love that SO uses a CC license but it does have some unintended consequences.
Is there a public list of such site? I think I'd find it quite useful (for my own blocking).
I don't think there is. I did it on a case-by-case basis.
They also use these domains:
w3schools.com
ww.w3schools.com
wwww.w3schools.com
Probably others too.
Probably others too.
Are you logged in? There is a link at the bottom of the page to download it. It isn't very prominent; I almost missed it.
im not getting it either: http://d.pr/i/CMx0
Neither am I. Could it be because I don't have any sites blocked?
It's either that, or a bug... And I can't really know which.
It's either that, or a bug... And I can't really know which.
I can't even count the number of times I've attempted to block every variant of "w3schools".
[deleted]
This really sucks for me because it means I'll have to see experts-exchange results again.
Why is Google shutting every secondary project/feature down right now? Reader, labs, blocked sites, code search... Any engineers able to weigh in on how this is affecting innovation or the 80/20 policy?
Why is Google shutting every secondary project/feature down right now? Reader, labs, blocked sites, code search... Any engineers able to weigh in on how this is affecting innovation or the 80/20 policy?
I'm an engineer at Google.
20% time is alive and well, but you have to be careful when thinking that 20% time is going to magically result in fully-polished new products. How much code can you write in one day a week? How much legacy code can you maintain in one day a week? The fact that programming is hard puts an upper bound on what you can effectively accomplish in your 20% time.
Effective 20% projects, in my opinion, are projects that add something new to an existing product and whose maintenance costs are low enough to be absorbed by the project's primary contributors. I've been working on Android lint checks recently: it's something that's self-contained, can be done without interfering with my 80% project, and won't make extra work for the project's main maintainers. (It's also open source, so it can't be canceled. Hah!)
It's also worth pointing out that not all 20% projects at Google are going to be visible outside of Google. I spent my first couple quarters at Google working on maintaining the internal Emacs extensions and user-base; most of my time was spent answering questions on the mailing list. This isn't going to result in Google Maps or Gmail, but it is still needs to be done.
If you want to do something with more public impact, I think you should work towards making that idea your 80% project. Get buy-in from your peers, write a design doc or description of the product, and talk to people who work on similar projects. Once you have a team ready to work on the project, a good design, and clear criteria for success, it shouldn't be hard to become a "real" project.
(I've succeeded in doing this for an internal utility I wanted to write. I wrote a design doc, got some feedback from my peers, and asked my manager for a quarter to work on it, and did. Now it exists and has users!)
20% time is alive and well, but you have to be careful when thinking that 20% time is going to magically result in fully-polished new products. How much code can you write in one day a week? How much legacy code can you maintain in one day a week? The fact that programming is hard puts an upper bound on what you can effectively accomplish in your 20% time.
Effective 20% projects, in my opinion, are projects that add something new to an existing product and whose maintenance costs are low enough to be absorbed by the project's primary contributors. I've been working on Android lint checks recently: it's something that's self-contained, can be done without interfering with my 80% project, and won't make extra work for the project's main maintainers. (It's also open source, so it can't be canceled. Hah!)
It's also worth pointing out that not all 20% projects at Google are going to be visible outside of Google. I spent my first couple quarters at Google working on maintaining the internal Emacs extensions and user-base; most of my time was spent answering questions on the mailing list. This isn't going to result in Google Maps or Gmail, but it is still needs to be done.
If you want to do something with more public impact, I think you should work towards making that idea your 80% project. Get buy-in from your peers, write a design doc or description of the product, and talk to people who work on similar projects. Once you have a team ready to work on the project, a good design, and clear criteria for success, it shouldn't be hard to become a "real" project.
(I've succeeded in doing this for an internal utility I wanted to write. I wrote a design doc, got some feedback from my peers, and asked my manager for a quarter to work on it, and did. Now it exists and has users!)
Programming, maintenance, scaling, etc, is not hard.
If you think it is hard you are either doing it wrong, or it's the wrong job for your personality.
It can sometimes be a challenge, but that is the fun part, not the hard part.
However this explains why Google seems so abrasive... they think it's hard!
If you think it is hard you are either doing it wrong, or it's the wrong job for your personality.
It can sometimes be a challenge, but that is the fun part, not the hard part.
However this explains why Google seems so abrasive... they think it's hard!
> Programming, maintenance, scaling, etc, is not hard.
> If you think it is hard you are either doing it wrong, or it's the wrong job for your personality.
You're replying to jrockway. Perhaps you could curb your arrogance. Programming is indeed an extremely hard topic filled with compromises and tradeoffs that are often unattractive.
Picking the correct solution that fulfills time, effort, complexity constraints is why some people are paid so well.
You're replying to jrockway. Perhaps you could curb your arrogance. Programming is indeed an extremely hard topic filled with compromises and tradeoffs that are often unattractive.
Picking the correct solution that fulfills time, effort, complexity constraints is why some people are paid so well.
Experts exchange stopped showing up in the results back when Google released the new indexing engine over a year ago. You should be fine.
I still see it sometimes. Meanwhile, Quora is in the works to become next EE...
Already is. I noticed the Google Blocklist was broken explicitly because I was trying to remove Quora from my results. Quora is another example of a company/site imploding.
I don't understand this comment. Quora doesn't show up in results bc they're not in the index, it's a private site. Unless that's changed?
They seem to be showing up for me. I don't have any recent organic examples, but here's query for one of the most popular threads, and the quora thread is the first result:
https://www.google.com/search?q=How+do+I+get+over+my+bad+hab...
What do you mean by "private site"?
The other objectionable/EE-like thing about it is how it hides the answers unless you log in. That really sucks.
https://www.google.com/search?q=How+do+I+get+over+my+bad+hab...
What do you mean by "private site"?
The other objectionable/EE-like thing about it is how it hides the answers unless you log in. That really sucks.
EE never hid their answers. They were only obfuscated. First[1] they made it look like the page ended, but you could scroll further and get the answers. Later, I saw that they had a section that looks like the answers, but all of the text was blurred. Again, if you scrolled past the 'end of the site' you could still get to the answers.
Looks like Quora gives you the question, and the first answer, but all other answers are Private-Only.
[1] I think that in the beginning they may have really hidden their answers, but were forced to actually show them by Google.
Looks like Quora gives you the question, and the first answer, but all other answers are Private-Only.
[1] I think that in the beginning they may have really hidden their answers, but were forced to actually show them by Google.
If you add ?share=1 to the end of a Quora URL you can get the full content.
They do show the first result even if you are not logged in (there is a dismissable modal popup that forces you to register though). Google indexes first result and that makes quora pages show up in relevant searches.
But somehow Quora still has first page results.
Thanks for the update! I'm less concerned now.
According to various sources, Larry Page had occasion to speak with Steve Jobs and ask his advice--in general terms--a few months before Jobs' death. Jobs originally wasn't inclined to give Page his time (he was still bitter about Android) but felt that, as he had reached out to an earlier generation of Silicon Valley entrepreneurs himself, he should pay it forward.
Jobs' advice was to focus on the most important products and prune the rest. It appears Google is now following this advice.
Jobs' advice was to focus on the most important products and prune the rest. It appears Google is now following this advice.
Works great for a company that risks generating a lot of noise to consumers (i.e. producing and marketing physical products). But for Google it's pervasiveness and breadth of services were part of the appeal, it's not like it's minor products interfered with the major parts of the company nor cost them much to maintain.
The niche services that have been killed were (by definition) used by small groups, groups which tend to be over-represented here and on other tech-oriented outlets. Even a service like Reader was something that only a small number of people used (compared to e.g. gMail).
One way to look at it is that these services are now validated (if small) market needs that smaller entrepreneurs can pursue. In the past, entrepreneurs lived in fear that Google would muscle in on their specialty; now it looks like they not only won't do that, but are in fact disengaging from things that do not have mass market appeal.
One way to look at it is that these services are now validated (if small) market needs that smaller entrepreneurs can pursue. In the past, entrepreneurs lived in fear that Google would muscle in on their specialty; now it looks like they not only won't do that, but are in fact disengaging from things that do not have mass market appeal.
I agree that Jobs' advice was a mismatch for Google. Trying a million things and seeing what sticks, like an incubator, is a better idea for Google. The problem is when that turns into Microsoft-esque trend chasing. But that only happens when upper management focuses on trend chasing. Otherwise they just end up as passing experiments.
I call it the Spaghetti Cannon Strategy.
Load up a cannon full of ideas, fire and see what sticks.
Microsoft were doing it before Google, but the financial-strategic dynamics of those businesses is very similar.
There's a fountain of cash and a series of spaghetti cannons around it blasting away furiously. Almost nothing sticks. But that's OK, because there's just so much money.
Load up a cannon full of ideas, fire and see what sticks.
Microsoft were doing it before Google, but the financial-strategic dynamics of those businesses is very similar.
There's a fountain of cash and a series of spaghetti cannons around it blasting away furiously. Almost nothing sticks. But that's OK, because there's just so much money.
Microsoft also fell into the trend-chasing trap before Google. It's a very fine line, though. Excel was probably trend-chasing at its time, but they were doing so many things at once back then that going head-to-head with Lotus 1-2-3 wasn't a bad idea to throw in there. Whereas stuff like Windows Phone or Zune seemed like major strategic shifts dictated from the top, not just "oh let's make one of those too".
The corollary of the spaghetti strategy is that if something doesn't stick, you kill it. Taking Jobs' advice is really just setting a higher bar for sticking.
Google+ didn't stick. It's more about choosing your own priorities to focus on, and while Apple was great at that, Google made a shit decision with G+ and are sticking with it obstinately.
Oh god, I've completely forgotten about expertsexchange! The horror, the horror.
This product shutdown is a little surprising, because it makes Google's core product worse and less useful.
Maybe they're confident their current ranking algorithms get rid of all the crap content farm sites now, and as someone else mentioned, this appears to be the case. But I suspect they'll enjoy a resurgence someday. Ugh.
This product shutdown is a little surprising, because it makes Google's core product worse and less useful.
Maybe they're confident their current ranking algorithms get rid of all the crap content farm sites now, and as someone else mentioned, this appears to be the case. But I suspect they'll enjoy a resurgence someday. Ugh.
Just imagine if you had gotten one from a non-expert.
I think they were just crowdsourcing the pruning effort. I don't have experts exchange blocked and I haven't seen it appear in my results for months.
Interesting theory, and genius if that's what they were doing with it. However, pulling it back off the market makes it look as though they failed at this.
or that they succeeded well enough that they no longer need additional data. note from the description of the panda algorithm [http://en.wikipedia.org/wiki/Google_Panda] "Google Panda affects the ranking of an entire site or a specific section rather than just the individual pages on a site."
I just want to see w3schools stop showing up at the top of results before MDN, which is a much higher quality resource.
You're right. I have an Alfred (http://www.alfredapp.com if using mac) shortcut to search google with "mdn" suffix.
> This really sucks for me because it means I'll have to see experts-exchange results again
The blocked sites list hasn't worked for like a year now, so if you haven't been seeing experts exchange links, you aren't going to now, either.
I noticed when w3schools starting showing back up last spring or so, but I haven't seen experts exchange in results for a long time (or at least, I've seen them at a low enough volume that I can't recall any now). Now the only somewhat equivalent experience is quora results, as others have said.
The blocked sites list hasn't worked for like a year now, so if you haven't been seeing experts exchange links, you aren't going to now, either.
I noticed when w3schools starting showing back up last spring or so, but I haven't seen experts exchange in results for a long time (or at least, I've seen them at a low enough volume that I can't recall any now). Now the only somewhat equivalent experience is quora results, as others have said.
larry page is trying to make the company more focussed (as per the advice of steve jobs). eric schmidt had a very different and unfocused philosophy.
besides google reader, most of the cuts have been good or just not important. I always felt like this feature was a reaction to blekko search engine, and not very interesting. curiously i found myself not wanting to block crappy sites like ehow just cuz they might have a good page and would rather just trust google.
besides google reader, most of the cuts have been good or just not important. I always felt like this feature was a reaction to blekko search engine, and not very interesting. curiously i found myself not wanting to block crappy sites like ehow just cuz they might have a good page and would rather just trust google.
I can provide evidence that this is a horrible idea.
Try typing a phone number into google.
Go ahead, try it.
Google has improved in some regards. I recall having to look through 3-4 pages to find a decent result in some queries a few years ago. They're usually on the first or second page now, but there are still these worthless results that end up in the 1-3 spot. That's enough evidence for me that they need expert intervention, not algorithmic guessing.
Try typing a phone number into google.
Go ahead, try it.
Google has improved in some regards. I recall having to look through 3-4 pages to find a decent result in some queries a few years ago. They're usually on the first or second page now, but there are still these worthless results that end up in the 1-3 spot. That's enough evidence for me that they need expert intervention, not algorithmic guessing.
[deleted]
Makes sense to me.
I'd rather have my browser know what results I don't want, than Google.
However it is Google that provides me a browser (for free) bundled with a sync feature.
Anyway, goodbye W3Schools! :)
I'd rather have my browser know what results I don't want, than Google.
However it is Google that provides me a browser (for free) bundled with a sync feature.
Anyway, goodbye W3Schools! :)
This feature has been discontinued for a few months now. I'm surprised nobody posted this until now.
I actually have found the plugin to be mostly adequate. There aren't a ton of sites that I feel deserve to be blocked from my search results, so syncing isn't a gigantic problem. I do find it annoying that there is a link added to each and every search result to block the site, but it might be useful if I decided to be really picky about my results.
I actually have found the plugin to be mostly adequate. There aren't a ton of sites that I feel deserve to be blocked from my search results, so syncing isn't a gigantic problem. I do find it annoying that there is a link added to each and every search result to block the site, but it might be useful if I decided to be really picky about my results.
I thought it was discontinued a while ago as well, but it was still in limbo as of late January of this year[1].
[1]: http://searchengineland.com/google-block-sites-feature-14640...
[1]: http://searchengineland.com/google-block-sites-feature-14640...
This sucks because the Chrome extension has a noticeable lag before the results are removed. I often accidentally click on a W3Schools link before the extension kicks in, and then wonder why it even came up in results at all.
This feature didn't work at the best of times, and became so unreliable...
I also don't believe that the blocked sites was linked to my Google account, because I had blocked sooo many different pages on W3Schools that I just abandoned using the blocked sites feature altogether.
I also don't believe that the blocked sites was linked to my Google account, because I had blocked sooo many different pages on W3Schools that I just abandoned using the blocked sites feature altogether.
> To block particular sites from your search results, we recommend the Personal Blocklist Chrome extension from Google.
Because, you know, everyone using Google also uses Chrome and only uses it on one device.
Because, you know, everyone using Google also uses Chrome and only uses it on one device.
Does someone know of a Firefox extension achieving the same result?
EDIT: just found this Greasemonkey script https://userscripts.org/scripts/show/33156
EDIT: just found this Greasemonkey script https://userscripts.org/scripts/show/33156
I think they're implying that you should switch to Chrome. They've shown notices to people running the latest version of Firefox that they should consider upgrading to Chrome on search pages before.
No solutions for people not using Chrome?
I found this greasemonkey script and it seems to do the trick.
http://userscripts.org/scripts/show/95205
http://userscripts.org/scripts/show/95205
I understand this from a technical perspective, but I always thought this feature had the potential to allow Google to gather better qualitative metrics about websites. For instance, if a huge chunk of people who are searching for web-related questions (end up on SO or MDN or something) and they all have ExpertSexChange and w3schools blocked, then perhaps those should be considered qualitatively worse and would therefore be ranked lower than the alternatives?
I guess not. I'm curious how they're determining quality in that case. Hey Bing, maybe you could pick this up and deliver quality results over faster results?
I guess not. I'm curious how they're determining quality in that case. Hey Bing, maybe you could pick this up and deliver quality results over faster results?
i'm sure that was the original intention, and the reason for the discontinuation is that nobody used it and the cost of maintaining the feature exceeded the value of the data they gathered from it. (or else everybody blocked the same few things in the first couple weeks the feature was available, and then they stopped learning anything from the feature)
Perhaps, but when useful features see lack of use, it's usually because it's a prohibitively difficult feature to use or very poorly presented.
Maybe if it was as simple as clicking a "never see results from this website again" it would've seen orders of magnitude more use and been far more useful. Perhaps an upvote/downvote system? Seems like these are relatively tiny time investments by a couple of engineers and would serve to massively increase the quality of results. Not sure why throwing useful features away is a better decision.
Maybe if it was as simple as clicking a "never see results from this website again" it would've seen orders of magnitude more use and been far more useful. Perhaps an upvote/downvote system? Seems like these are relatively tiny time investments by a couple of engineers and would serve to massively increase the quality of results. Not sure why throwing useful features away is a better decision.
A very good thing is the minus sign in google search. It allows you to exclude keywords and sites. They should expand that. My searchbar gets very crowded at times.
I guess I need to go back to manually appending "mdn" to all my html/css/js-related queries. How much longer is w3schools going to stay at the top of those results?
EDIT: Actually, it turns out I've been using the linked Chrome extension (went to install, discovered it was already installed). I just conflated the two services and forgot about the extension. So to make this productive, I recommend this extension. I haven't seen w3schools in months.
EDIT: Actually, it turns out I've been using the linked Chrome extension (went to install, discovered it was already installed). I just conflated the two services and forgot about the extension. So to make this productive, I recommend this extension. I haven't seen w3schools in months.
/me straightens DuckDuckGo independent evangelist hat and nervously rings doorbell
"Judging from your comment, you sound like the perfect user for Duck Duck Go!
If you set DDG as the default browser for chrome/ff, you can search mdn with just '!mdn foo'. Also at your fingertips is searching !jquery (and hundreds of other search shortcuts)
DuckDuckGo is great and makes searching super productive!"
/me smiles eagerly
"Judging from your comment, you sound like the perfect user for Duck Duck Go!
If you set DDG as the default browser for chrome/ff, you can search mdn with just '!mdn foo'. Also at your fingertips is searching !jquery (and hundreds of other search shortcuts)
DuckDuckGo is great and makes searching super productive!"
/me smiles eagerly
Or you can just go to MDN once, right click on the search box, choose "Add a Keyword (...)" and then use "mdn foo" in the URL bar.
Bang syntax seems such a small feature to change search engines.
Bang syntax seems such a small feature to change search engines.
How is that different from searching "mdn foo" right now?
This lets you use MDN's own search functionality, like DuckDuckGo's bang syntax.
Can I ask you why you see DDG as the perfect alternative to google? One would think that if you are concerned with the sort of things google is doing, you would bank on an open source search engine, so this sort of thing doesn't just repeat itself.
I have both YaCy and Seeks (distributed open source search engines) in my FF search bar, but the results are really... disappointing. I just get good results with DDG.
DDG's commitment to privacy and not bubbling, etc. I do not believe are going away, it's their biggest selling point.
Besides, as long as RMS is demanding that DDG be the default search engine in the GNU web browser, it's good enough for me.
DDG's commitment to privacy and not bubbling, etc. I do not believe are going away, it's their biggest selling point.
Besides, as long as RMS is demanding that DDG be the default search engine in the GNU web browser, it's good enough for me.
Because there is less typing to do searches. As a developer, I only have a limited number of keystrokes in me before I run out and die.
I'm using DDG, well, sometimes, and I really, really want to like it. But its reliability and performance -- just in producing search results let alone relevant ones, has been sorely lacking for the past 3-4 months.
Exactly.
I am all for a competitor to google that offers some sort of useful features google doesnt, but the straight truth is that DDG does not provide as relevant of results, and you can feel how "dumb" it is compared to the results google is giving you.
It could be some sort of personal bias, but I search the same term in both engines and get (in some cases) wildly different results, rarely in favor of DDG.
It could be some sort of personal bias, but I search the same term in both engines and get (in some cases) wildly different results, rarely in favor of DDG.
Doesn't DuckDuckGo's index come from BOSS[1]? What happens if Yahoo cuts that?
[1] http://developer.yahoo.com/boss/search/
[1] http://developer.yahoo.com/boss/search/
I switched to DDG permanently a while back, and using it as a pass-through for other searches has been great. Saves me the effort of keeping up with a bunch of custom search bookmarks myself, and they're intuitive enough to guess (for example, I started working with Node a couple of weekends ago and naturally assumed that !npm would be available; same experience with !bgg for BoardGameGeek).
The only thing I hate about using DDG is the lack of a synchronized history. It's very helpful for me to find past Google Searches through Web History, because it works cross device.
You might want to check YubNub [0], which is an enhanced version of this prefixes idea. If anything, you don't have to prepend your queries with ! there :)
[0] http://www.yubnub.org
[0] http://www.yubnub.org
In Chrome you can just add something like
https://www.google.com/search?q=mdn+%s
https://developer.mozilla.org/en-US/search?q=%s
to the bottom of you search engine list with shortcut letters of your choice (e.g. "m")
chrome://settings/searchEngines
Then typing "m query" into the omnibox is even faster than resorting to google first.
https://www.google.com/search?q=mdn+%s
https://developer.mozilla.org/en-US/search?q=%s
to the bottom of you search engine list with shortcut letters of your choice (e.g. "m")
chrome://settings/searchEngines
Then typing "m query" into the omnibox is even faster than resorting to google first.
Same can be done in Firefox on any bookmark using the "Keyword" attribute in the bookmark editor user interface.
w3schools was the first thing that came to my mind when I read the headline. Long time ago, I blocked this site and It's reappeared some time ago out of the blue with no option to block again.
[deleted]
[deleted]
For god's sake. I literally just started using it for W3Schools about two months ago. It costs google practically nothing to do that, and now they suggest adding another extension to chew out yet more of my memory.
Plus it looks like they aren't going to adhere to the implications of having a number of users blocking a site. cough w3schools.
Plus it looks like they aren't going to adhere to the implications of having a number of users blocking a site. cough w3schools.
I just noticed w3schools appearing again in my search results.. well this is the cause.
I wonder when will Google discontinue Gmail and offer to download all mail in a file.
You can remove up to the top ones million sites with MillionShort: http://millionshort.com/about.html and you can specify individual sites to remove via settings: http://millionshort.com/settings.php
Removing a feature that fixes a major shortcoming of your product (spam sites) and directing your customers to use an extension to fix it is awful.
What faith I once had in Google being a company that puts users first is long gone.
What faith I once had in Google being a company that puts users first is long gone.
One could argue that making users manually select "spam sites" is unfriendly, too. If I were the PM for Google Search, I'd want to make a product that "just worked" without requiring the user to screw around with bells and whistles. A work of art is done when there's nothing left to remove, not when there's nothing more to add.
Providing an extension for power users to tweak the bells and whistles sounds like a fine gap-filler while perfection is achieved, however.
Providing an extension for power users to tweak the bells and whistles sounds like a fine gap-filler while perfection is achieved, however.
Google has been shutting down features left, right and centre. I'm left wondering: what useful thing has google done since gmail and ads?
From the top of my head: V8, Golang, AngularJS, Closure Tools, Search by Image¹, Maps + Street View, Android, Now, Goggles, Fiber², Public DNS, GWT.
¹ not original, but works better than Tineye, in my opinion
² granted, only for a limited number of people
¹ not original, but works better than Tineye, in my opinion
² granted, only for a limited number of people
Self driving cars? Android?
I do love my self driving car
What's your point? It's not useful because you don't have one?
It's not useful because nobody has one. Google have yet to "do" it. Their self driving car is as useful to anyone reading this as human teleportation right now. They're great idea's but until they are accessible in some form to the public, it is still just an idea that needs work not an innovation.
I look forward to the day I can say I do have one though.
I look forward to the day I can say I do have one though.
"I don't use Golang, so it doesn't count as a useful innovation!" ;-)
What have the Romans ever done for us?
well, android
Was that feature really that indispensable? I can usually get away by refining my search query, and more often that not, using the minus sign solves the problem.
Back when the Chrome plug-in shipped (it preceded Blocked Sites by a bit) I compiled an extensive list of content farms as a starting guide for people wanting less spam in their SERPs. If you're migrating over to the extension it's worth a look (and as always, any new suggestions for the list):
http://www.jongales.com/blog/2011/02/14/list-of-content-farm...
http://www.jongales.com/blog/2011/02/14/list-of-content-farm...
I'm curious how all this "wood putting" is affecting staff attrition/layoffs.
If had invested my work into one of the major cut projects, I'd be fuming as an employee right now.
Also, it's interesting how many of the cut services seem to be around working, but slowly decaying and abandoned projects. It says something about google's ability to keep working on and developing products...especially since most of their products are essentially public betas.
If had invested my work into one of the major cut projects, I'd be fuming as an employee right now.
Also, it's interesting how many of the cut services seem to be around working, but slowly decaying and abandoned projects. It says something about google's ability to keep working on and developing products...especially since most of their products are essentially public betas.
As many others, I used this mainly to block w3schools from my searches. Disappointing to see it go.
I don't understand why this was discontinued. Surely this data would be a gold mine in improving the quality of search results, which is core to what Google does.
A blacklist that can be added to anonymously and unaccountably is not something you want affecting search results.
Sure it is, obviously you don't take any small amount of blocks as a signal. By getting a significant amount though and looking at each persons block list to normalize there blocks I think it would be useful.
> Sure it is, obviously you don't take any small amount of blocks as a signal.
You do realize spammers are extremely good generating large amounts of things, right?
> By getting a significant amount though and looking at each persons block list to normalize there blocks I think it would be useful.
Spammers are also very good (though not great) at adding innocuous data alongside their spam to look like real users.
You don't realize it, but your solution has just exploded in complexity.
You do realize spammers are extremely good generating large amounts of things, right?
> By getting a significant amount though and looking at each persons block list to normalize there blocks I think it would be useful.
Spammers are also very good (though not great) at adding innocuous data alongside their spam to look like real users.
You don't realize it, but your solution has just exploded in complexity.
The same thing is true of backlinks. Sure there is a point in which it may not be worth the trouble, not questioning that.
I find both its birth and death interesting. The birth because the fact that you could mark sites as spam was one of the early talking points at Blekko, the back end architecture of our engine includes a 'selector' mechanism (slashtags) and maintaining a personal 'spam' slash tag came along for free. Its one of the features I continue to use.
And then it showed up as a feature in Google's results which I found interesting because having worked at Google and had the 'deeper than non-Googlers, not as deep as someone in the search group' classes on how the two services that made up Google search at the time, I had a feel for how much lubricant it would take to squeeze that feature into the existing pipeline. It made me wonder if Google was following us :-)
The death of it is also interesting, because having it in the browser as a plug-in vs the results means two things; You can't offer it as a service to your partners, and you can't know apriori if you're sending junk. If Blekko's partners say "We'd like to use your index but we don't want any results that include x,y or z" we can do that but that is at the API level with results coming right out of the index filtered by a 'negative' slashtag, but its unclear if anyone can (or does) use Google's index in that way (unlike BOSS for example). On the browser side, since you don't know what the plug-in is going to kill, how do you select the 10 documents to send? It makes me wonder that if you're doing a search in a highly contested search (like 'weight loss') and Google can't know that the 10 blue links it is about to send you are all spam (and on really contested keywords this is not uncommon) are you left with just sponsored links and no organic results?
'Panda' update all you want, even Google now admits they are adding staff to curate results (Microsoft/Bing had already gone public with their 'editor's choice' announcement). This is a good thing, but it only covers half the situation. At Blekko we got flamed by a user for not having any 'alternative' medicine sites in our 'health' slashtag, which started a conversation with that user about creating their own slashtag with all of those web sites that were unfairly penalized by the medical establishment just because their methods and claims weren't the product of some 'big pharma' company. But they could (and I believe they did) create a slashtag of all those 'hidden' sites and they got great search results for them. Reinforcing to me and others that 'spam' can be relative, and really it has two sides, user and index.
So pondering this move on Google's part makes for interesting reading.
And then it showed up as a feature in Google's results which I found interesting because having worked at Google and had the 'deeper than non-Googlers, not as deep as someone in the search group' classes on how the two services that made up Google search at the time, I had a feel for how much lubricant it would take to squeeze that feature into the existing pipeline. It made me wonder if Google was following us :-)
The death of it is also interesting, because having it in the browser as a plug-in vs the results means two things; You can't offer it as a service to your partners, and you can't know apriori if you're sending junk. If Blekko's partners say "We'd like to use your index but we don't want any results that include x,y or z" we can do that but that is at the API level with results coming right out of the index filtered by a 'negative' slashtag, but its unclear if anyone can (or does) use Google's index in that way (unlike BOSS for example). On the browser side, since you don't know what the plug-in is going to kill, how do you select the 10 documents to send? It makes me wonder that if you're doing a search in a highly contested search (like 'weight loss') and Google can't know that the 10 blue links it is about to send you are all spam (and on really contested keywords this is not uncommon) are you left with just sponsored links and no organic results?
'Panda' update all you want, even Google now admits they are adding staff to curate results (Microsoft/Bing had already gone public with their 'editor's choice' announcement). This is a good thing, but it only covers half the situation. At Blekko we got flamed by a user for not having any 'alternative' medicine sites in our 'health' slashtag, which started a conversation with that user about creating their own slashtag with all of those web sites that were unfairly penalized by the medical establishment just because their methods and claims weren't the product of some 'big pharma' company. But they could (and I believe they did) create a slashtag of all those 'hidden' sites and they got great search results for them. Reinforcing to me and others that 'spam' can be relative, and really it has two sides, user and index.
So pondering this move on Google's part makes for interesting reading.
I think it'd be illogical for Google not to in some ways compete with alternatives, like Blekko in this case. Google's core business is search, and in the early days, it was this very naive behavior that allowed Google to sneak right under places like Inktomi. Just because a search engine starts small doesn't mean it can't gain traction or market, and if it has compelling enough features, it'll happen in a heartbeat.
I suggest reading "I'm Feeling Lucky" by Douglass Edwards - the early chapters describe a lot of Google's anxiety of being killed by other search engines for one reason or another. It helps to explain why they'd be quick to adopt new strategies to keep results fresh.
I suggest reading "I'm Feeling Lucky" by Douglass Edwards - the early chapters describe a lot of Google's anxiety of being killed by other search engines for one reason or another. It helps to explain why they'd be quick to adopt new strategies to keep results fresh.
I'm not sure google is even clear about what its core business is, these days. It really seems quite clear that they've abandoned much of the approach and focus on being the information finder to now being the social-network wannabe.
It's as if something flipped 180 degrees - they went from being the company that wanted to help you find out about everything to being the company that wanted to find everything about you.
It's as if something flipped 180 degrees - they went from being the company that wanted to help you find out about everything to being the company that wanted to find everything about you.
Google's core business is advertising.
Anyone who disagrees with this other Jacques is invited to take a squiz at Google's annual reports.
Any of them since the introduction of adwords and adsense. It doesn't matter which. They all basically read the same.
Any of them since the introduction of adwords and adsense. It doesn't matter which. They all basically read the same.
Indeed – perhaps I was wrong to say that they don't currently know what business they're in – rather, they haven't known and recently, they've re-aligned around a singular strategy focused on that.
The problem is that for so long, they built their public facing brand on a completely different premise and image. Therefore, as they've re-aligned, they're facing a bit of cognitive dissonance in the minds of their products (err, the public).
The problem is that for so long, they built their public facing brand on a completely different premise and image. Therefore, as they've re-aligned, they're facing a bit of cognitive dissonance in the minds of their products (err, the public).
Might the leadership change have anything to do with that?
If google is still researching AI then it is likely because the war with the spammers can't be won in any other way. Sooner or later you end up in a situation where the amount of 'spam' versus the amount of 'ham' is such that no matter how good your algorithms and how good your computing infrastructure you'll end up spitting out a lot of spam.
Human curation is a stop-gap solution, computers can generate spam faster than humans can filter it. You could argue that the humans will only have to look at the good stuff but unfortunately they'll have to look at all of it to make a decision in any practical setup.
Google has stepped up the arms race against spam and for a while their algorithms gave them an edge, now we have reached the level where the spammers have the edge again and it will take another quantum leap before the good guys can regain the edge.
It would be funny if we end up with AI mostly because of the spammers :)
Human curation is a stop-gap solution, computers can generate spam faster than humans can filter it. You could argue that the humans will only have to look at the good stuff but unfortunately they'll have to look at all of it to make a decision in any practical setup.
Google has stepped up the arms race against spam and for a while their algorithms gave them an edge, now we have reached the level where the spammers have the edge again and it will take another quantum leap before the good guys can regain the edge.
It would be funny if we end up with AI mostly because of the spammers :)
It would indeed be an interesting situation if the spammers forced the creation of AI (either they doing it to make better spam or someone like Google to deny it).
One of the interesting things I have experienced in my time at Blekko has been that "growth" on the Web isn't really growing all that much. Sure there are trillions of pages being created but there are only so many things the few billion people in and around the Internet care about. There are 'hard information' places, which are things like libraries where reference searches are common, there are 'entity' places, be thay shops or service providers or SOMA startups, and there are "transient" places where information is current and then stale, to be stored and later reconstructed like coral into a 'dead' (in terms of change) but 'useful' base. Seeing the web from the point of view of a web scale crawler and indexer it starts to be clear that the mantra "Organize all the world's information" is getting tantalizingly close to a dynamically stable froth.
I have to believe that Google has figured this out, some of the smartest engineers I've worked with are at Blekko but Google has its share as well. When you trawl through the fishery and all you get are trash fish you start to wonder, "hmm did we actually catch all the fish there are?" So to it goes with "the Web".
I started doing some speculation [1] on how you could value information that was discoverable on the Internet. And one of the schemes I came up with is how many people would find that information "useful", where useful really means they would have some reason of seeking it out. And then scaling that value by the value to them of having it. So for example if my genome was online, there is maybe a dozen people who would find it "useful", and of that dozen probably on the insurance actuaries and perhaps the occasional researcher who would find it "valuable".
Now one takes that unit of applicability/value and scales it again by the "cost" to acquire it (find it, index it, etc). And from that you can compute the total size of the Web. Well estimate it at least. So far my upper bound (based primarily on the fact that world population is stabilizing) is about 72 trillion documents at any given time.
When you look at it that way you can see that ultimately the spammers lose. They lose because over time the actual information that rises to the useful vs cost threshold is identified and classified, or the legitimate channels that provide dynamic information, or the legitimate archival sources that provide distilled information are all, for the most part known and 99.99% of all your users can find everything they want. And as a spammer you are no longer given the free reign of "appear and be indexed" you have to ask for admittance though some form or another. And the level of new credible sources that are created is inherently a function of the number of people that exist, and the number of people that exist is stabilizing.
When Yahoo started with its human curation it was vastly better than anything anyone else could do, and then it was overwhelmed by a combination of growth and algorithms that could much more rapidly infer curation from the social signals of bookmark pages and article reference. Curation has come back into favor, and it combined with machine learning algorithms will create what is essentially a stable corpus of documents[2] known as "the web".
Wikipedia is a great analog for what is happening world wide. All the reference articles they want to put in are nearly done, the number of editors required has gone down not up, and the future of web search is, in my opinion of course, similar. The only new ground on the web is social networking and Google understands that it seems. Its fortunate for them that only deep pocketed entity that is possibly a near time threat there is Microsoft, and since to date Microsoft is trying to do exactly what they did, they benefit from having already been through that part and know exactly what Microsoft will have to do next.
[1] My side hobby is attempting to discern the economics of information.
[2] Documents are just that, pieces of information, I hardly count every page rendered by Angry Birds in a browser as a separate document, in fact applications are them selves a single "document" in the since that some number of people will seek them out, and connect/consume them over what we think of as the "web" interface.
One of the interesting things I have experienced in my time at Blekko has been that "growth" on the Web isn't really growing all that much. Sure there are trillions of pages being created but there are only so many things the few billion people in and around the Internet care about. There are 'hard information' places, which are things like libraries where reference searches are common, there are 'entity' places, be thay shops or service providers or SOMA startups, and there are "transient" places where information is current and then stale, to be stored and later reconstructed like coral into a 'dead' (in terms of change) but 'useful' base. Seeing the web from the point of view of a web scale crawler and indexer it starts to be clear that the mantra "Organize all the world's information" is getting tantalizingly close to a dynamically stable froth.
I have to believe that Google has figured this out, some of the smartest engineers I've worked with are at Blekko but Google has its share as well. When you trawl through the fishery and all you get are trash fish you start to wonder, "hmm did we actually catch all the fish there are?" So to it goes with "the Web".
I started doing some speculation [1] on how you could value information that was discoverable on the Internet. And one of the schemes I came up with is how many people would find that information "useful", where useful really means they would have some reason of seeking it out. And then scaling that value by the value to them of having it. So for example if my genome was online, there is maybe a dozen people who would find it "useful", and of that dozen probably on the insurance actuaries and perhaps the occasional researcher who would find it "valuable".
Now one takes that unit of applicability/value and scales it again by the "cost" to acquire it (find it, index it, etc). And from that you can compute the total size of the Web. Well estimate it at least. So far my upper bound (based primarily on the fact that world population is stabilizing) is about 72 trillion documents at any given time.
When you look at it that way you can see that ultimately the spammers lose. They lose because over time the actual information that rises to the useful vs cost threshold is identified and classified, or the legitimate channels that provide dynamic information, or the legitimate archival sources that provide distilled information are all, for the most part known and 99.99% of all your users can find everything they want. And as a spammer you are no longer given the free reign of "appear and be indexed" you have to ask for admittance though some form or another. And the level of new credible sources that are created is inherently a function of the number of people that exist, and the number of people that exist is stabilizing.
When Yahoo started with its human curation it was vastly better than anything anyone else could do, and then it was overwhelmed by a combination of growth and algorithms that could much more rapidly infer curation from the social signals of bookmark pages and article reference. Curation has come back into favor, and it combined with machine learning algorithms will create what is essentially a stable corpus of documents[2] known as "the web".
Wikipedia is a great analog for what is happening world wide. All the reference articles they want to put in are nearly done, the number of editors required has gone down not up, and the future of web search is, in my opinion of course, similar. The only new ground on the web is social networking and Google understands that it seems. Its fortunate for them that only deep pocketed entity that is possibly a near time threat there is Microsoft, and since to date Microsoft is trying to do exactly what they did, they benefit from having already been through that part and know exactly what Microsoft will have to do next.
[1] My side hobby is attempting to discern the economics of information.
[2] Documents are just that, pieces of information, I hardly count every page rendered by Angry Birds in a browser as a separate document, in fact applications are them selves a single "document" in the since that some number of people will seek them out, and connect/consume them over what we think of as the "web" interface.
I wonder how long it takes till someone comes up with "Blockedsitesly" ^^
I don't remember using this feature much, but when I downloaded my block list this is what I found:
www.teachexcel.com
craftkeys.com
softwaretopic.informer.com
www.telerik.com
eggheadcafe.com
www.kaboodle.com
bigresource.com
I know it fucking is! Which is why I am seeing Experts Exchange (a tall claim), ahead of Stackoverflow!
Note that you can continue to use the Chrome extension to block sites, which I believe we rolled out before this feature. Get it at https://chrome.google.com/webstore/detail/personal-blocklist...
And here's the blog post we did about the Chrome extension: http://googleblog.blogspot.com/2011/02/new-chrome-extension-...
And here's the blog post we did about the Chrome extension: http://googleblog.blogspot.com/2011/02/new-chrome-extension-...
Interestingly enough this means you have to install software on your own computer to get better search results from a company that tries to deliver a hosted ("cloud") service. Seems backwards to me.
Well, I'm with Matt on this one. When blocklist rolled out people were stressing about having their blocked selections stored in the cloud, blocking rings to kill competing sites and so on. I think it makes sense to build that functionality in at the client end.
Does the extension remove results before they are sent to the client? In other words, if a result page has ten results and three of them are blocked, will I see seven or ten result boxes on my screen?
Client-side spam blocking has been available for years. The primary advantage of having spam-blocking integrated into the search engine is that the user sees a full page of results for their query, even if some results were filtered out.
If the Chrome extension supports server-side filtering, then it must be sending some sort of "exclude results from these sites" message to google.com. It would be very nice if the message format was documented and official, even if it's something as simple as a JSON list. That would allow extension developers to add support to other major browsers, especially Firefox.
Client-side spam blocking has been available for years. The primary advantage of having spam-blocking integrated into the search engine is that the user sees a full page of results for their query, even if some results were filtered out.
If the Chrome extension supports server-side filtering, then it must be sending some sort of "exclude results from these sites" message to google.com. It would be very nice if the message format was documented and official, even if it's something as simple as a JSON list. That would allow extension developers to add support to other major browsers, especially Firefox.
I was curious too, so I checked; it's pretty obviously client-side blocking; there's even a flash of the blocked result before it's hidden, which is awful.
This really sucks. As said above, it wasn't working anyway. Too bad it happened. I wonder if Google could've gained more knowledge about the common "undesired" results and use it to improve the search.
My blocked sites list only had one entry: experts-exchange.com
Personal Blocklist(by me)
http://crossrider.com/download/29789
> here an extension that works with firefox and google chrome (and maybe with some version of ie)
> currently only block w3schools and expert-exchange :-)
> will be extendable soon
it is not on the browser marktplaces right now, so you kind of got to install it manually.
it is not on the browser marktplaces right now, so you kind of got to install it manually.
What disturbs me is how it's okay for Google to allow this "for Chrome users only" but it's a $700m fine for Microsoft for bundling Internet Explorer into the OS. Is Microsoft really more of a monopoly than Google? Of course not. It's ridiculous that MSFT gets fined such high amounts but no one questions Google's ethics or says, "Hey, wait a minute..."
Comparisons to Microsoft aside, it's not true that no one questions Google. The EU antitrust authorities have been examining them for two years now, they just haven't reached a conclusion yet: http://www.reuters.com/article/2013/02/22/eu-google-idUSL6N0...
Sorry - I misspoke. I meant "no one questions Google in this thread".
I came for the horror comments about experts-exchange.com and was not disappointed. At least two of you have mentioned it!
So they'll tailor search results for you personally based on what they think you'll want (or, eventually on what they want you to want), but they won't let you declare what you don't want?
Why is this even news? This was announced long time ago and I thought it was fairly evident. Use personal blocklist, instead.
Being able to personally fine tune those results myself, outside of what a robot thought to "personally" customize my experience to, was a great feature. Suggesting I use a third party plugin for a browser I don't use is hardly a compromise or a solution, and does nothing for my Google search results on mobile platforms where that plugin is not now nor will likely ever be available.