Stop StopBadware.org and Google from UNFAIRLY blocking websites
21 comments
UPDATE: For anyone else running into this problem. Of course, Google never clearly stated this reason, I (and you) have to guess until you get it right. Each guess takes more and more time. My last (and successful) attempt, I had to wait 16 hours for Google to do its review and unblock our site.
Turns out sites are guilty by association. If Google finds malware on "old.chess.ca" it will bock every website ending in "chess.ca". Including "www.chess.ca" even though that website is on different servers (different IP) and contains zero links to the other bad website. Removing the server from DNS, plus disabling with .htaccess rules that redirect all requests, ... plus waiting 16 hours for Google to do another review ... is what got us back up & running.
Turns out sites are guilty by association. If Google finds malware on "old.chess.ca" it will bock every website ending in "chess.ca". Including "www.chess.ca" even though that website is on different servers (different IP) and contains zero links to the other bad website. Removing the server from DNS, plus disabling with .htaccess rules that redirect all requests, ... plus waiting 16 hours for Google to do another review ... is what got us back up & running.
This is a very scary situation. It means Google could block your website, not for something bad on your website, but for something bad on another website that you just happen to link to. That's beyond your control! Imagine this happening to a website like wikipedia.org, which as millions of links to external websites. If one of those millions of websites gets hacked, will Google block wikipedia.org? This is like being charged with a crime you didn't commit and being punished immediately without even an initial hearing to respond to the charges! Google and StopBadware.org should not have this power on the internet!
> Or should it be more like the criminal justice system, innocent until proven guilty including after an appeal.
In the criminal justice system, jail or restrictive bail conditions pending outcome of trial is common.
In the criminal justice system, jail or restrictive bail conditions pending outcome of trial is common.
Yes, when the accused is a threat to society or a flight risk. Our website is neither.
Yes, a site that you admit was infested with malware due to extended neglect is a threat to society.
And you recent admission that, contrary to your second description, it still deliberately links to the old site but with a warning page makes your whole claim of improper listing clearly baseless.
And you recent admission that, contrary to your second description, it still deliberately links to the old site but with a warning page makes your whole claim of improper listing clearly baseless.
(oh dear)
> We are the national governing body for a sport.
Link?
Link?
I hesitated to include it: didn't want HN to mess with our analytics and free hosting limits (or our review at the mercy of Google). What the hell, here it is: https://www.chess.ca/
We used to have Matt Cutts [1] here, and he used to take a look at somewhat similar problem. I just guess that if someone is going to give e hand, s/he needs the link to take an unofficial look.
There is still a link to the old site in https://www.chess.ca/en/ws/under-construction/
[1] https://en.wikipedia.org/wiki/Matt_Cutts
There is still a link to the old site in https://www.chess.ca/en/ws/under-construction/
[1] https://en.wikipedia.org/wiki/Matt_Cutts
The /en/ws/under-construction link is to http://www.chess.ca/en/ws/old-chess-ca/ which is a page on the new website that explains the dangers before you click a button to actually go to the old site. The link on the explanation page is what I removed.
> which is a page on the new website that explains the dangers before you click a button to actually go to the old site.
So the new site still intentionally (contrary to your prior claim that all links were removed and that you verified this in your source) links to the old site, and you don't understand why the appeal of the badware listing due to that link failed?
So the new site still intentionally (contrary to your prior claim that all links were removed and that you verified this in your source) links to the old site, and you don't understand why the appeal of the badware listing due to that link failed?
> The link on the explanation page is what I removed
I still see the link in the link in the English version https://imgur.com/a/GqrwNkc Perhaps you fixed the French version?
In your imgur, you see "LINK" will take you to "chess.ca/en/ws/old-chess-ca/". You've nicely pointed this out with red arrows. "/en/ws/old-chess-ca/" is NOT the old website. It is a page on the new website. It has an explanation of the situation. Previously, this explanation page had the link to the old website (at http://old.chess.ca/), but I removed that (English and French) this morning before requesting a review by Google. Even though these links are now gone, they failed my review.
I thought it was weird to put the old site as a subdirectory of the new one, but I think it is possible with some trick in Apache or whatever you are using. It make more sense to use another subdomain, and as yuo noted in the sibling comment it may be what is causing the problem. I really don't know. Good luck.
(How much info is in the old server? Is it possible to put the old server behind a proxy that rewrite all the pages and remove the malware or whatever is the problem? Perhaps remove all the links to external sites and to executable files or something like that. I'm 80% sure this is a bad idea, but perhaps it is possible with enough technical knowledge.)
(How much info is in the old server? Is it possible to put the old server behind a proxy that rewrite all the pages and remove the malware or whatever is the problem? Perhaps remove all the links to external sites and to executable files or something like that. I'm 80% sure this is a bad idea, but perhaps it is possible with enough technical knowledge.)
Many ways to do it, but another subdomain was easiest. The two websites are on different hosts.
There is still a lot of info on the old website. When we decided to go with an "emergency" rewrite of our website, instead of trying to cleanse Drupal, we rewrote only the core functions needed to run tournaments. To keep users from complaining, I kept old.chess.ca available. Now, they'll just have to wait. I do not want to risk anything that might provoke the Google monster into banishing the new website again.
There is still a lot of info on the old website. When we decided to go with an "emergency" rewrite of our website, instead of trying to cleanse Drupal, we rewrote only the core functions needed to run tournaments. To keep users from complaining, I kept old.chess.ca available. Now, they'll just have to wait. I do not want to risk anything that might provoke the Google monster into banishing the new website again.
Ya got me thinking. Maybe they failed my review because they expected both www.chess.ca and old.chess.ca to be entirely cleaned-up (both being *.chess.ca). I'm so desperate to get unblocked that I have deleted the DNS entry for old.chess.ca (TTL=15m so I hope it is gone before Google re-reviews me). FYI, since eventually you won't be able to go to old.chess.ca either.
DETAILS: On July 3, our website was blocked the 1st time. This was justified (although I disagree with the immediate punishment): it was an old Drupal 7 site that was installed in 2011 and, because of budget constraints, rarely received security patches. For years, it had the Drupageddon and Drupalgeddon exposures. Not surprising it was infested with malware. This is when I got involved (volunteering my time & skills). The website was rewritten as a static site (virtually unhackable) using Hugo on Netlify. On July 26, we were unblocked.
At 1am ET this morning (Aug 1), we were blocked by Google again! The new (Hugo) website had one link to the old (Drupal) website (now with a different subdomain) so users could access content we had not yet migrated to the new website. This link was the reason Google gave for blocking us. I read Google's notification at 6am and immediately removed the links and requested a review via Google's Search Console. At 9am, Google said the review had failed, our website was still blocked, without any additional information. Search Console still says the bad link is to the old website's subdomain. I verified my Hugo source files: there is zero instances of the old website's subdomain name. At 10:30 I submitted another request for review and am still waiting for a response. Their request pop-up says "Warning! Requesting a review of issues that weren't fixed will result in longer review cycles" so I may be waiting a longer time.
(BTW, anyone with inside Google connections, help would be appreciated)
At 1am ET this morning (Aug 1), we were blocked by Google again! The new (Hugo) website had one link to the old (Drupal) website (now with a different subdomain) so users could access content we had not yet migrated to the new website. This link was the reason Google gave for blocking us. I read Google's notification at 6am and immediately removed the links and requested a review via Google's Search Console. At 9am, Google said the review had failed, our website was still blocked, without any additional information. Search Console still says the bad link is to the old website's subdomain. I verified my Hugo source files: there is zero instances of the old website's subdomain name. At 10:30 I submitted another request for review and am still waiting for a response. Their request pop-up says "Warning! Requesting a review of issues that weren't fixed will result in longer review cycles" so I may be waiting a longer time.
(BTW, anyone with inside Google connections, help would be appreciated)
So, it's your complaint isn't, as you initially presented it, that you were unfairly listed as a badware site, but instead that you were rightly listed once, asked to be delisted after incompletely dealing with the problem, rightly failed the subsequent review, but this time you really did deal with it?
Huh? The key point here is I don't think Google should block any website without providing some warning an an opportunity to fix the issue before imposing an immediate punishment. What if google.com search engine linked to a badware website? Should it be immediately blocked until fixed pending a review? No, of course not. Do onto others as you would have others do onto you.
> The key point here is I don't think Google should block any website without providing some warning an an opportunity to fix the issue before imposing an immediate punishment.
This isn't punishment, it's end-user protection of the same type as other anti-malware tools. And the protection fundamentally cannot work if operated in the way you suggest.
To use your criminal justice analogue you've not only confessed to the crime, but confessed to continuing the crime while seeking pre-trial release, after an evolving series of denials.
This isn't punishment, it's end-user protection of the same type as other anti-malware tools. And the protection fundamentally cannot work if operated in the way you suggest.
To use your criminal justice analogue you've not only confessed to the crime, but confessed to continuing the crime while seeking pre-trial release, after an evolving series of denials.
Note that websites are blocked for Chrome and Firefox users, since both of these use StopBadware.org.
See "DETAILS" in a reply below (it wouldn't fit in Hackernews' 2000 char limit).
Our website is not some click-bait shady website. We are the national governing body for a sport. Our members need our website to find out about tournaments. Our tournament organizers need our website to lookup info about players to run their tournaments. So there is real economic and reputational damage to us as a result of Google deciding to block us without any due process. The punishment remains in effect while we wait for another review.
Should Google have this power? Should Google be allowed to inflict immediate punishment (blocking)? Should Google give at least 5 business days notice before they inflict punishment? Or should it be more like the criminal justice system, innocent until proven guilty including after an appeal.
This is important as there can be real economic and reputational damage resulting from these autocratic actions.