Google Search Referrers Disappear In iOS 6(searchengineland.com)
searchengineland.com
Google Search Referrers Disappear In iOS 6
http://searchengineland.com/ios-6-removes-all-google-search-referer-data-134560
52 comments
Yep, that's what the RFC says. http://tools.ietf.org/html/rfc2616#section-15.1.3
"Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol."
"Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol."
Doesn't it also say not to send refer data to different domains? I'm trying to find docs for that.
EDIT: https://bugzilla.mozilla.org/show_bug.cgi?id=141641 Maybe I'm wrong but that seems to be what some browsers do/did.
EDIT: https://bugzilla.mozilla.org/show_bug.cgi?id=141641 Maybe I'm wrong but that seems to be what some browsers do/did.
Sending referer data to different domains is always allowed for non-secure URLs (otherwise, web analytics would be pretty boring). For secure URLs however, the browser should only send referer data if the destination page is also secure, regardless of domain.
That makes no sense, and defeats some of the protection of SSL. If you're on a secure site, there should be no referrer sent cross-domain, or cross-protocol.
I makes some sense. If you're using SSL and someone is trying to snoop on your connection. If you visit a non-SSL site from an SSL site and the referrer gets sent then the snooper knows the URL of the page you were on last. Whereas if the target site is SSL then no information is leaked to listeners.
The point of SSL is to protect from snooping, not as a general privacy-protection mechanism that should have everything privacy-related shoehorned into it.
The point of SSL is to protect from snooping, not as a general privacy-protection mechanism that should have everything privacy-related shoehorned into it.
[deleted]
There is no fix for this. In the article, see the related stories at the bottom, in particular "Google Puts A Price On Privacy."
Google deliberately prevents referrer data from passing regardless of the state of SSL. It only allows it to pass if someone's clicking on an ad.
Google deliberately prevents referrer data from passing regardless of the state of SSL. It only allows it to pass if someone's clicking on an ad.
Yes, this is the case. A quick search shows I'm being taken to Google via https, hence no referrer string. You'd hope the author would have worked that out :-/
On my desktop (with chrome) searching google via HTTPS I get search results that first redirect to a HTTP page on Google then does a client redirect to the search result. This produces referer information to the search result.
It seems this step is not being taken on iOS, it could be google who singling out iOS6 and not doing the redirect to the HTTP page.
It seems this step is not being taken on iOS, it could be google who singling out iOS6 and not doing the redirect to the HTTP page.
Tried this on my Nexus 7 and didn't get the redirect so it may be something they're doing for mobile user agents. Its definitely not specific to iOS6
Probably to reduce latency. Another hop is expensive on mobile networks.
That's interesting. But as a counter point, my blog gets a bunch of hits from Google marked as "[Secure Search]", so I guess they don't always do it.
Unfortunately no. Welcome to the future of web analytics, where all inbound keywords are (not provided) :(. Google claims this is a privacy thing, however AdWords keywords are still conveniently passed on!
If you want referrer data, offer your site over SSL...Browsers don't send a referrer header if the referring site is SSL and the target site is not.
I haven't tested any browsers or looked at any standards on this, but if that works it doesn't make any sense to me. I would think that the rule would be "Browsers don't send a referrer header [to a different domain] if the referring site is SSL" period (no "and the target site is not" part). If a URL on site X contains secret information, why would it be OK to transmit that secret information to some other website simply because the secret information is encrypted while in transit? At the very least, I would expect the user to be prompted to confirm such action before doing it (which would probably be more annoying to the user than just dropping the referrer info).
I haven't tested any browsers or looked at any standards on this, but if that works it doesn't make any sense to me. I would think that the rule would be "Browsers don't send a referrer header [to a different domain] if the referring site is SSL" period (no "and the target site is not" part). If a URL on site X contains secret information, why would it be OK to transmit that secret information to some other website simply because the secret information is encrypted while in transit? At the very least, I would expect the user to be prompted to confirm such action before doing it (which would probably be more annoying to the user than just dropping the referrer info).
This is exactly what's happening - though the author seems to think differently.
This isn't a huge deal as you can still get details of Google Queries using Google Webmaster Tools: http://www.google.com/webmasters/
This isn't a huge deal as you can still get details of Google Queries using Google Webmaster Tools: http://www.google.com/webmasters/
It is a huge deal and has been since Google stopped passing referrers for signed-in users last October. That's because...
1) If it's a privacy thing, then Google is happy having a leak by giving these terms to advertisers
2) The data is not provided in a way that can be used by third party ad retargeting terms, giving Google a competitive advantage
3) Publishers can't use the data to better target with landing pages
4) Google Webmaster Central data only goes back for a short period of time, and if you haven't constantly been downloading your data, all your historic traffic information related to a term has gone poof.
1) If it's a privacy thing, then Google is happy having a leak by giving these terms to advertisers
2) The data is not provided in a way that can be used by third party ad retargeting terms, giving Google a competitive advantage
3) Publishers can't use the data to better target with landing pages
4) Google Webmaster Central data only goes back for a short period of time, and if you haven't constantly been downloading your data, all your historic traffic information related to a term has gone poof.
Does it create a monopoly situation?
This is a massively huge deal for attribution of organic search.
The queries you can download from webmaster tools are just 1 column of data....nothing to tie queries to conversions or any meaningful metric.
If it was everyone who was losing this data...I could understand the change but if you buy adwords ads, you still get the referral data even in https Google.
The queries you can download from webmaster tools are just 1 column of data....nothing to tie queries to conversions or any meaningful metric.
If it was everyone who was losing this data...I could understand the change but if you buy adwords ads, you still get the referral data even in https Google.
I'm not familiar with the implications of not sending the referer data.
Can be interpreted that not sending the referer, and thus the decision to use the SSL version is in anyway malicious in intent (almost like what the author is implying in the headline)?
Can be interpreted that not sending the referer, and thus the decision to use the SSL version is in anyway malicious in intent (almost like what the author is implying in the headline)?
You can't have it all. The HTTP RFC is very clear that referrer data from HTTPS sites should not be sent to HTTP sites (in order to protect the content of the HTTPS site - remember: Even the URL is transmitted in an encrypted way, so it might contain sensitive data).
So by doing the Google searches over HTTPS, you make the life of pages and analytics vendors harder, but you are doing a huge favor to the end users because now their searches are private between them and Google. Nobody else on the way can sniff them. No government and no neighbor/coworker.
Of course, once the user clicked and found your site, the URL the user initially used isn't that confidential any more, so in theory, passing the referrer wouldn't be a problem in this case, but that's not what the standard says and it's not what browsers are doing.
If you want to see this changed, talk to the IETF or the W3C to have the standard changed such that, for example, google could add some rel= attribute to tell the browser that sending a referrer is OK.
Or just serve your page over SSL, further helping the end-user - and it's all about them in the first place - isn't it?
So by doing the Google searches over HTTPS, you make the life of pages and analytics vendors harder, but you are doing a huge favor to the end users because now their searches are private between them and Google. Nobody else on the way can sniff them. No government and no neighbor/coworker.
Of course, once the user clicked and found your site, the URL the user initially used isn't that confidential any more, so in theory, passing the referrer wouldn't be a problem in this case, but that's not what the standard says and it's not what browsers are doing.
If you want to see this changed, talk to the IETF or the W3C to have the standard changed such that, for example, google could add some rel= attribute to tell the browser that sending a referrer is OK.
Or just serve your page over SSL, further helping the end-user - and it's all about them in the first place - isn't it?
From my iOS6 4S p:
https://www.google.com?q=referer and then click to www.whatismyreferer.com- no referer as expected.
http://www.google.com?q=referer and then click to www.whatismyreferer.com - a valid referer as expected.
Nothing to see here folks.
https://www.google.com?q=referer and then click to www.whatismyreferer.com- no referer as expected.
http://www.google.com?q=referer and then click to www.whatismyreferer.com - a valid referer as expected.
Nothing to see here folks.
ironically if you follow this path in a desktop browser it bounces you through an HTTP url so the referrer is set.
Contrary to the article it looks like Google is doing something different to iOS 6 users, not Apple. (note I have no idea if this is something specific to iOS users, it could be happening on Android as well - I'm not trying to attribute this to malice)
EDIT: Just tried this on my nexus 7 and it also skips the http referer page, so I get the "no referer" message. it seems like something they do for mobile browsers?
Contrary to the article it looks like Google is doing something different to iOS 6 users, not Apple. (note I have no idea if this is something specific to iOS users, it could be happening on Android as well - I'm not trying to attribute this to malice)
EDIT: Just tried this on my nexus 7 and it also skips the http referer page, so I get the "no referer" message. it seems like something they do for mobile browsers?
I'm guessing that the deliberate skip the HTTP intermediate page to reduce latency on mobile clients. If you are on something like GPRS or Edge, that extra hop could add a lot of latency.
It's a trade-off between ensuring a good user experience (fast page loads) and a good website owner experience (full referer data).
FWIW, I find the HTTP intermediate step a tad annoying (it makes copy-and-pasting a result from Google a bit more awkward), but I can see why it's there.
It's a trade-off between ensuring a good user experience (fast page loads) and a good website owner experience (full referer data).
FWIW, I find the HTTP intermediate step a tad annoying (it makes copy-and-pasting a result from Google a bit more awkward), but I can see why it's there.
I think you missed the point of the above demonstration a bit. It was meant to demonstration that iOS6 is acting completely normal. No referer is sent for https, and that is expected behavior. Google even went so far as to send out emails, blog posts to the analytics and the webmaster tools blogs, and G+ posts about it a while back so GA users would not be surprised as there referrer data decreased. For those missing refers we now have to use the Webmaster Tools and try to piece the data together a bit.
I think you missed my point.
If I go to the https link above via a desktop browser, and see the results page (also https) with "What's my referrer" on it and click the link Google doesn't take me directly to the site, the link for "What's my referrer" is (not https)
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#...
Which then redirects me to whatismyreferer.com so that whatismyreferer gets a google referer. Apparently for whatever reason if you do that on iOS6 the search results takes you directly to whatismyreferer.com without the redirect.
If I go to the https link above via a desktop browser, and see the results page (also https) with "What's my referrer" on it and click the link Google doesn't take me directly to the site, the link for "What's my referrer" is (not https)
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#...
Which then redirects me to whatismyreferer.com so that whatismyreferer gets a google referer. Apparently for whatever reason if you do that on iOS6 the search results takes you directly to whatismyreferer.com without the redirect.
agree, this is something new. here's how google search on desktop acted the last time i checked:
from http://google.com --> redirects to a url that sends "google.com..." referrer with search terms
from https://google.com --> redirects to a url that sends "google.com..." referrer with search term stripped
from https://encrypted.google.com --> sends blank referrer
from http://google.com --> redirects to a url that sends "google.com..." referrer with search terms
from https://google.com --> redirects to a url that sends "google.com..." referrer with search term stripped
from https://encrypted.google.com --> sends blank referrer
You need to test it with a search. It's not that you don't get referrer data. It's that you don't get search referrer data. Do a search on Google, see if you get the terms you searched for embedded in the referrer. That's what we tested; that's what was removed.
Actually, scratch that. You shouldn't be getting a referrer at all. We'll check further, but you should check with Google searches.
[deleted]
Unrelated but I just learned that "referer" is a misspelling of "referrer" but we still use the former term as it is set in stone by the HTTP standard. http://en.wikipedia.org/wiki/HTTP_referer#Origin_of_the_term...
The last line of the article:
> However, it does seem like this issue is specific with Google. So I suspect it is either a bug on Google’s side (or a feature).
If this is what he suspects, why does he phrase the title as if it is something in iOS 6 that is causing the issue.
> However, it does seem like this issue is specific with Google. So I suspect it is either a bug on Google’s side (or a feature).
If this is what he suspects, why does he phrase the title as if it is something in iOS 6 that is causing the issue.
Well, the referer is set by the browser, so it is the browser that it not setting said HEADER.
I'm not sure it is mandatory to be set either [1], so it might be correct correct behaviour according to the spec, even though it's not what might be expected.
[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14...
[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14...
To get people to click!
If it's new to iOS 6 vs iOS 5, then it's clearly something that changed in iOS 6.
I agree with the logic for sure - My point is more about using the active verb "Removes", as if iOS 6 gets really close to passing the referer header, but removes it at the last second.
Maybe something like "omits" or "doesn't pass" would have been more true to the story - but I understand a writer needing to have a catchy title.
Maybe something like "omits" or "doesn't pass" would have been more true to the story - but I understand a writer needing to have a catchy title.
Default search site changed to be SSL. Are we really complaining that's a bad thing?
Lately if its anything Apple related it seems to be interpreted as a bad thing. It is getting tiresome.
Good point. I've edited the headline to make it clearer.
Referrer data still seems to be sent when I go to http://www.whatismyreferer.com/ from the SSL version of Google on my desktop.
In fact, I did the exact same thing but with the User-Agent set to iOS 6. No referrer data.
This is a problem on Google's side.
In fact, I did the exact same thing but with the User-Agent set to iOS 6. No referrer data.
This is a problem on Google's side.
That's right and while it's up to google - there is an unfortunate consequence that a very large analytics industry exists and funds thousands if not millions of jobs - not saying this won't have an impact on that industry would be short sighted - I am of the belief that if you don't want to be track there is one very simple solution turn off your computer.
Think of the children too.
Personally I couldn't care less if some people lose jobs.
SEO in general is all about gaming the algorithms of search engines to gain an unfair advantage. A search engine that needs SEO to get to the front page is a broken search engine. And because Google has the incentive and the resources to constantly improve it, an industry built around the deficiencies of a search engine is doomed for extinction from the start.
Of course, SEO is not the only reason for why some companies need analytics. But for all the other reasons, they should do what good businesses have always done and actually talk to their customers or potential customers, instead of looking in the records of their referrers, which often contains the deepest desires of people without them realizing that the data in question will be released.
As to your statement, you're saying it like it's an option to turn off the computer. Well, that's not an option anymore for many people, and it's increasingly clear that we're going to depend on computers connected to the Internet for everything, like for doing our shopping, for paying our bills, for communicating with others and for being able to do our jobs.
It's enough to say that privacy is a fundamental right that people should always keep fighting for and for which eternal vigilance is required. And if there ever was a time in history when people truly had privacy, that was only because some people fought for it.
Totalitarian states never needed high-tech, like interconnected computers or street video cameras with face recognition and GPS chips implants and all that crap, all they needed was for people to turn on each other, so the Internet is not even a prerequisite, but human stupidity and shortsightedness is.
Personally I couldn't care less if some people lose jobs.
SEO in general is all about gaming the algorithms of search engines to gain an unfair advantage. A search engine that needs SEO to get to the front page is a broken search engine. And because Google has the incentive and the resources to constantly improve it, an industry built around the deficiencies of a search engine is doomed for extinction from the start.
Of course, SEO is not the only reason for why some companies need analytics. But for all the other reasons, they should do what good businesses have always done and actually talk to their customers or potential customers, instead of looking in the records of their referrers, which often contains the deepest desires of people without them realizing that the data in question will be released.
if you don't want to be track there is one very
simple solution turn off your computer.
As to your statement, you're saying it like it's an option to turn off the computer. Well, that's not an option anymore for many people, and it's increasingly clear that we're going to depend on computers connected to the Internet for everything, like for doing our shopping, for paying our bills, for communicating with others and for being able to do our jobs.
It's enough to say that privacy is a fundamental right that people should always keep fighting for and for which eternal vigilance is required. And if there ever was a time in history when people truly had privacy, that was only because some people fought for it.
Totalitarian states never needed high-tech, like interconnected computers or street video cameras with face recognition and GPS chips implants and all that crap, all they needed was for people to turn on each other, so the Internet is not even a prerequisite, but human stupidity and shortsightedness is.
SEO is not about an unfair advantage. If that were the case, then Google itself wouldn't offer a free guide to SEO.
That's simply an unfortunately common and ignorant misconception. It's like saying that knowing HTML design requirements is an unfair advantage in making web pages.
That's simply an unfortunately common and ignorant misconception. It's like saying that knowing HTML design requirements is an unfair advantage in making web pages.
I don't disagree with your views on privacy, but what google does is simple hypocrisy. It still continues to provide visitors' search terms to paid advertisers who are probably much more likely to use them for questionable purposes than a random site from organic results.
If google is so concerned with privacy, why not block keyword passing from AdWords ads as well?
If google is so concerned with privacy, why not block keyword passing from AdWords ads as well?
SEO and analytics are not the same thing
They updated the bottom of the article to read Postscript: At about 12:15pm EDT I now see referer data being passed from mobile Safari on iOS 6 via Google either signed in or signed out. I am not sure if this was a bug or something else but it does now seem to pass referer data. It even passes query data if you are signed in.
This appears to be a mobile browser thing when searching google with https.
Search https://www.google.com in a desktop browser and you will get a results page where the links don't like you directly to the result page, they actually take you to an http redirect page which then forwards you to your final destination (which now gets a "google.com" referer.
On my iPhone and my Nexus 7 if I do a search on https://www.google.com the results page does not have the http redirect link. So the final destination does not get a referer.
This seems to be a behavior on Google's side that is different for desktop and mobile browsers, there's nothing specific to Apple here.
Search https://www.google.com in a desktop browser and you will get a results page where the links don't like you directly to the result page, they actually take you to an http redirect page which then forwards you to your final destination (which now gets a "google.com" referer.
On my iPhone and my Nexus 7 if I do a search on https://www.google.com the results page does not have the http redirect link. So the final destination does not get a referer.
This seems to be a behavior on Google's side that is different for desktop and mobile browsers, there's nothing specific to Apple here.
This will badly reflects on analytics softwares if you have a heavy mobile website.
As someone who spends a lot of time working on internet marketing, not getting the search terms due to this and all the other changes really stinks!
it will possibly affect registration wall publishers who are open to google referals (like WSJ, NYT, etc.).
The "fix" is "easy": If you want referrer data, offer your site over SSL or use the Google Webmaster tools (I don't know whether there is an API to get to that data).
Browsers don't send a referrer header if the referring site is SSL and the target site is not. This has always been the case even on the desktop.