On Google, a Political Mystery That's All Numbers(online.wsj.com)
online.wsj.com
On Google, a Political Mystery That's All Numbers
http://online.wsj.com/article_email/SB10001424052970203347104578099122530080836-lMyQjAxMTAyMDAwNDEwNDQyWj.html?mod=wsj_valetleft_email
10 comments
I'm planning on blogging a bit on this particular topic tomorrow. If you have any questions, please let me know and I'll try to address them.
I don't think the gTrends argument works though (see http://news.ycombinator.com/item?id=4741590). Most of the results inserted are super-recent (and so I expect the recency of the trends data -- like the last 90 days -- to dominate). Of course it is all a black box so who knows.
Some of the more interesting things to me are:
--you can't reliably de-personalize (as you cited).
--the variation in results across our study was great.
--variation for signed out (even incognito) users was not much different from signed in users.
I don't think the gTrends argument works though (see http://news.ycombinator.com/item?id=4741590). Most of the results inserted are super-recent (and so I expect the recency of the trends data -- like the last 90 days -- to dominate). Of course it is all a black box so who knows.
Some of the more interesting things to me are:
--you can't reliably de-personalize (as you cited).
--the variation in results across our study was great.
--variation for signed out (even incognito) users was not much different from signed in users.
It would be vaguely interesting to know from study participants how unique their browser is.[1] Mine appeared to yield the same result whether in incognito mode or not, but I can't really see Google using this approach.
Your point about Google trends is counterintuitive but evidence-supported, which makes it particularly interesting. It may well be that they are looking at outcome measures after search personalization, and abandoning personalizations that yield no results. Or perhaps a certain volume is required over time for a particular query before triggering an automated personalization trial. As you say, it's a black box, so who knows.
[1] = https://panopticlick.eff.org/
Your point about Google trends is counterintuitive but evidence-supported, which makes it particularly interesting. It may well be that they are looking at outcome measures after search personalization, and abandoning personalizations that yield no results. Or perhaps a certain volume is required over time for a particular query before triggering an automated personalization trial. As you say, it's a black box, so who knows.
[1] = https://panopticlick.eff.org/
I don't know why they bother putting 'Mystery' in the headline.
http://www.google.ca/trends/explore#q=obama,%20romney
Surprise, the incumbent president is far more popular than the challenger, in terms of historic searches. There's no mystery, this is somewhere between a fluff political piece and an advertisement for Google's new search tech.
Come on WSJ, you can do better.
edit: A bigger mystery; does anyone know why these searches are more popular in Africa than the United States? The top five countries for Obama searches are:
1) Burundi 2) Guinea 3) Rwanda 4) Sierra Leone 5) United States
Meanwhile, Romney is pretty much only relevant in the US.
http://www.google.ca/trends/explore#q=obama,%20romney
Surprise, the incumbent president is far more popular than the challenger, in terms of historic searches. There's no mystery, this is somewhere between a fluff political piece and an advertisement for Google's new search tech.
Come on WSJ, you can do better.
edit: A bigger mystery; does anyone know why these searches are more popular in Africa than the United States? The top five countries for Obama searches are:
1) Burundi 2) Guinea 3) Rwanda 4) Sierra Leone 5) United States
Meanwhile, Romney is pretty much only relevant in the US.
That's actually not true. Here's the past 90 days in the US for Obama (blue), Romney (red) and taxes (yellow): http://www.google.com/trends/explore#q=obama,+romney,+taxes&...
If you restrict to just US news searches (as many of the inserted results are newsy), it is similar: http://www.google.com/trends/explore#q=obama%2C%20romney%2C%...
Actually a lot of election-related queries, most of which searched less than Romney and Obama transform results in this manner, e.g. social security, health care, abortion, taxes, ohio, election and many others. Just not Romney: http://www.google.com/trends/explore#q=romney%2C%20social%20...
If you restrict to just US news searches (as many of the inserted results are newsy), it is similar: http://www.google.com/trends/explore#q=obama%2C%20romney%2C%...
Actually a lot of election-related queries, most of which searched less than Romney and Obama transform results in this manner, e.g. social security, health care, abortion, taxes, ohio, election and many others. Just not Romney: http://www.google.com/trends/explore#q=romney%2C%20social%20...
I'm not sure if I follow the second part. Adding more comma-separated words doesn't do anything to the other lines, this is just an overlay of multiple graphs. Adding or removing 'taxes' doesn't do anything.
Sorry for not being clear (was also trying to not be verbose!). All it shows is that all those terms were searched a lot less than Romney for the past 90 days, and yet they all were found to alter subsequent search results after searching them (unlike Romney).
In a similar vein. Here is the trend with the addition of the candidates in the previous 2 elections:
http://www.google.ca/trends/explore#q=obama%2C%20romney%2C%2...
It will be interesting to see if Obama wins. If he does then winning in Google Trends will predict winning in the election.
http://www.google.ca/trends/explore#q=obama%2C%20romney%2C%2...
It will be interesting to see if Obama wins. If he does then winning in Google Trends will predict winning in the election.
Regarding popularity by country: I don't know, but I would speculate that decreased access in Burundi, Guinea, etc. means they don't use the internet for mundane stuff... shopping or looking at cats.
Thus Obama may constitute a larger proportion of searches in those places than in the US.
Thus Obama may constitute a larger proportion of searches in those places than in the US.
Alternate Headline: DuckDuckGo chief, trying to rustle up more users from Google personalization-fear-baiting, spoon feeds article to WSJ.
What'd I miss?
What'd I miss?
Is there evidence of this?
Sure, read the last few paragraphs of this Journal article. The reporter is up front about her source: "Mr. Weinberg brought the discrepancy to the Journal's attention.""
""In September, Gabriel Weinberg, founder and chief executive of tiny Duck Duck Go Inc., which markets itself as a privacy-protecting search engine, stumbled across the "you recently searched for" phenomenon in a study he conducted of Google's personalization efforts. Mr. Weinberg, whose site is based in Paoli, Pa., asked 131 of its users to search Google for several keywords at 2 p.m. Eastern time on Sept. 2: "Obama," "abortion" and "gun control." His testers received a wide variety of different results that appeared to be personalized by location and other factors.
Mr. Weinberg also noticed that some testers received results labeled "you recently searched for Obama," and discovered that he couldn't replicate the same label when searching for Romney. Mr. Weinberg brought the discrepancy to the Journal's attention.""
There's no shame in running an interesting story that's brought to your attention by someone with vested interests.
For example, notorious fraudster Barry Minkow dug up a list of all the executives in publicly traded corporations who lied about their college degrees and gave that information to the WSJ after short-selling the companies, http://online.wsj.com/article/SB122652836844922165.html.
Win-win -- the Journal got an important news tip, and he made some money on the bounce.
Unfortunately for Minkow, he couldn't resist going after just bad companies and ended up in prison for defrauding a home-building company, http://online.wsj.com/article/SB1000142405311190346110457646....
""In September, Gabriel Weinberg, founder and chief executive of tiny Duck Duck Go Inc., which markets itself as a privacy-protecting search engine, stumbled across the "you recently searched for" phenomenon in a study he conducted of Google's personalization efforts. Mr. Weinberg, whose site is based in Paoli, Pa., asked 131 of its users to search Google for several keywords at 2 p.m. Eastern time on Sept. 2: "Obama," "abortion" and "gun control." His testers received a wide variety of different results that appeared to be personalized by location and other factors.
Mr. Weinberg also noticed that some testers received results labeled "you recently searched for Obama," and discovered that he couldn't replicate the same label when searching for Romney. Mr. Weinberg brought the discrepancy to the Journal's attention.""
There's no shame in running an interesting story that's brought to your attention by someone with vested interests.
For example, notorious fraudster Barry Minkow dug up a list of all the executives in publicly traded corporations who lied about their college degrees and gave that information to the WSJ after short-selling the companies, http://online.wsj.com/article/SB122652836844922165.html.
Win-win -- the Journal got an important news tip, and he made some money on the bounce.
Unfortunately for Minkow, he couldn't resist going after just bad companies and ended up in prison for defrauding a home-building company, http://online.wsj.com/article/SB1000142405311190346110457646....
Strong circumstantial evidence. See also http://paulgraham.com/submarine.html
WSJ is really turning into Murdoch's mouthpiece. The breaking news is that at standing president of 4 years has different search results than his opponent?
Here's how to easily de-personalize your google experience
1. Disable cookies for encrypted.google.com
2. Set the default search engine to: https://encrypted.google.com/search?&q=%s
Edit: Sorry, de-personalize, not anonymize.
1. Disable cookies for encrypted.google.com
2. Set the default search engine to: https://encrypted.google.com/search?&q=%s
Edit: Sorry, de-personalize, not anonymize.
Thank you.
> a person who searches for "Harry Potter," and then for "Amazon," actually wants "Harry Potter" results from Amazon.com Inc.
For my case, this would never be true. I would simply append "amazon" to the original search. Now, I could probably train myself to not do that, but I don't want to. I don't want my search engine to have any kind of state, but unfortunately Google thinks I want the opposite.
I guess I want Google to be purely functional. The same query will give the same answer, no matter who asks it and no matter where in the world it's asked. I certainly don't want Google to create a little filter bubble just for me.
> a person who searches for "Harry Potter," and then for "Amazon," actually wants "Harry Potter" results from Amazon.com Inc.
For my case, this would never be true. I would simply append "amazon" to the original search. Now, I could probably train myself to not do that, but I don't want to. I don't want my search engine to have any kind of state, but unfortunately Google thinks I want the opposite.
I guess I want Google to be purely functional. The same query will give the same answer, no matter who asks it and no matter where in the world it's asked. I certainly don't want Google to create a little filter bubble just for me.
Anonymize != de-personalize, and this doesn't do either, unfortunately -- though is a good start at protecting yourself.
Sorry, meant to say depersonalize. Care to expand why this doesn't work? Just ran the same test, and I see the exact same results.
Yes, what you said sounds plausible and is indeed intuitive, but we've found it empirically to be untrue.
In fact, on fresh private mode browsing (cookies cleared) when people searched for the same result at the same time, we still saw significant variation in results, even in the same country and on non-location results.
In fact, on fresh private mode browsing (cookies cleared) when people searched for the same result at the same time, we still saw significant variation in results, even in the same country and on non-location results.
This may have nothing to do with personalization. http://googleblog.blogspot.com/2008/09/search-evaluation-at-...
"We also make use of experiments, in which small fractions of queries are shown results from alternative search approaches."
Many small experiments can lead to differences for a reasonably large fraction of queries.
Many small experiments can lead to differences for a reasonably large fraction of queries.
This TED talk does a good job explaining the concepts of what is happening in this article.
http://www.ted.com/talks/eli_pariser_beware_online_filter_bu...
http://www.ted.com/talks/eli_pariser_beware_online_filter_bu...
"...In the hands of a human, decisions like these might be viewed as biased. For a Google algorithm, they are simply a matter of numbers..."
I hate to continue the political thread, but "biased" doesn't quite cut it. Algorithms don't automatically make everything magic. There is a structural issue that just saying "it's the algorithm" doesn't explain or address.
No doubt the data leans that way because Obama has been a searchable term for longer. This just leads to the natural question: should incumbents be given extra chances at a target audience simply because they've generated a lot more content? I don't think so, but I find myself arguing with a mathematical formula. The nature of the social value of making democratic decisions is different from the nature of the personal value of targeting results.
Weird.
I hate to continue the political thread, but "biased" doesn't quite cut it. Algorithms don't automatically make everything magic. There is a structural issue that just saying "it's the algorithm" doesn't explain or address.
No doubt the data leans that way because Obama has been a searchable term for longer. This just leads to the natural question: should incumbents be given extra chances at a target audience simply because they've generated a lot more content? I don't think so, but I find myself arguing with a mathematical formula. The nature of the social value of making democratic decisions is different from the nature of the personal value of targeting results.
Weird.
"should incumbents be given extra chances at a target audience simply because they've generated a lot more content?"
Since the incumbent has already campaigned the previous election, fairness dictates that he shouldn't be allowed to campaign at all against the challenger.
Since the incumbent has already campaigned the previous election, fairness dictates that he shouldn't be allowed to campaign at all against the challenger.
[deleted]
[deleted]
> In the hands of a human, decisions like these might be viewed as biased. For a Google algorithm, they are simply a matter of numbers.
This is pure nonsense. Algorithms are written by humans and are just as biased as their creators.
This is pure nonsense. Algorithms are written by humans and are just as biased as their creators.
The fact that personalized search applies to political topics is unsurprising and probably reflects that Google is data-driven. The fact that Romney doesn't trigger the same personalization as Obama probably reflects that, until recently, his name was virtually never searched for (relatively).[1]
By introducing Mr. Weinberg, the author gives us hope that we will learn something about the nature of personalized search and its implications, but I think that HNers would be much more satisfied to re-read his blog entries on the topic.[2]
[1] = http://www.google.com/trends/explore#q=obama%2C%20romney&...
[2] = http://www.gabrielweinberg.com/blog/2012/08/how-do-you-compl...