GoDaddy Responds To Namecheap Accusations, Removes “Normal” Rate Limiting Block(techcrunch.com)
techcrunch.com
GoDaddy Responds To Namecheap Accusations, Removes “Normal” Rate Limiting Block
http://techcrunch.com/2011/12/26/godaddy-responds-to-namecheap-accusations-removes-normal-rate-limiting-block/
8 comments
Your programmers might want to spend some time implementing some (more) error checking and logging.
The Whois protocol, RFC 812, does not specify the result format returned. The protocol is actually incredibly simple; you connect to port 43 via TCP, send a query delimited by a CRLF, the server sends back a plain text, human readable response.
Now, many registrars use a format that has lines that look like "Name: Value". But not all do. And those that do will frequently include other text, including terms of service for using their WHOIS service or providing instructions. And some don't use a "Name: Value" format, or they allow for multi-line indented values, or whatnot. Some seem to use "%" to delimit informational lines, as if it's a comment characters, while others don't bother to use anything.
So, parsing whois is a bit more like scraping the web than like parsing a real file format. It can break sometimes.
Now, many registrars use a format that has lines that look like "Name: Value". But not all do. And those that do will frequently include other text, including terms of service for using their WHOIS service or providing instructions. And some don't use a "Name: Value" format, or they allow for multi-line indented values, or whatnot. Some seem to use "%" to delimit informational lines, as if it's a comment characters, while others don't bother to use anything.
So, parsing whois is a bit more like scraping the web than like parsing a real file format. It can break sometimes.
I understand, and while I do have experience in text processing I would not want to take on a project like this. The number of people responding about this raises the question, though: why is this being reinvented so much? My impression is that everybody writes their own parser, which smells like NIH syndrome.
There is no standard format for whois data, it's just free form text. All whois parsers are fragile and error prone by definition.
When i was working in that field a few years ago best practice was to make some guesses, log everything and have a ton of manual checks.
When i was working in that field a few years ago best practice was to make some guesses, log everything and have a ton of manual checks.
There's even fewer standards for how to present the address in the WHOIS data. You leave something open to the end user and they will always get very, ah, creative.
Generally each registrar has their own format, but as there's thousands of registrars, some of which have various formats for different types of registrations, there's a confusion of possible layouts in what is generally assumed to be only for record-keeping purposes, not data-interchange.
If you've ever looked under the hood at a WHOIS parser you'll see it's filled with nice, neat code, a series of extensions, and then a whole pile of hacks to deal with hundreds of edge cases.
If you've ever looked under the hood at a WHOIS parser you'll see it's filled with nice, neat code, a series of extensions, and then a whole pile of hacks to deal with hundreds of edge cases.
A namecheap employee claims that they did contact goDaddy and that they resorted to the PR release only after they couldn't resolve the issue directly with them: http://community.namecheap.com/blog/2011/12/26/godaddy-trans...
There's contact and there's contact the right people. I have no trouble believing both sides here: Namecheap contacted someone at GoDaddy who didn't know what was going on and didn't relay the message to the relevant people.
While this sounds perfectly reasonable I'm wondering why these rate limits weren't already lifted for a very large, well known, established domain registrar like Namecheap.
Further more, it seems that the onus would still be on GoDaddy to investigate the large amount of traffic coming from the whois queries, notice they are legitimate and coming from Namecheap, and rectify the situation... though Namecheap should have contacted GoDaddy as well.
Further more, it seems that the onus would still be on GoDaddy to investigate the large amount of traffic coming from the whois queries, notice they are legitimate and coming from Namecheap, and rectify the situation... though Namecheap should have contacted GoDaddy as well.
I found the community manager response from NameCheap rather interesting. Looks like they did attempt to reach out which is what I would have expected prior to such a blog in the first place. http://community.namecheap.com/blog/2011/12/26/godaddy-trans...
Its not as clear when you are seeing traffic from a reseller like Namecheap. I can imagine that their query traffic comes through enom and their own systems depending on the TLD in question.
Volume. Namecheap's query level was probably quite low in the past as people were likely transferring from a variety of registrars, not a single one en-masse.
I think NameCheap might lose out on this PR exchange. GoDaddy's excuse checks out and NameCheap lowered themselves to their level with the accusation they posted.
This is very reminiscent of the slapfight that the Bing team had with Google over Microsoft possibly stealing Google's search results. In the end both companies came off as childish and petulant.
This is very reminiscent of the slapfight that the Bing team had with Google over Microsoft possibly stealing Google's search results. In the end both companies came off as childish and petulant.
Did Microsoft's excuse check out, though?
Microsoft used its toolbar to watch which results people actually landed on after doing a search. MS used this to refine their own search results, promoting pages that people tended to "end" their searches on.
Google "caught" MS stealing results by manually adding an unlinked page with a unique nonsense string on it into their index. Then they installed the Bing toolbar, searched for the string in Google, and clicked on the only result. Soon after, the same result showed up in Bing.
Google claimed that Bing must have "stolen" the search result because it's not linked from any other page, so there was no organic way for Bing to find it. But it seems pretty normal to me.
Google "caught" MS stealing results by manually adding an unlinked page with a unique nonsense string on it into their index. Then they installed the Bing toolbar, searched for the string in Google, and clicked on the only result. Soon after, the same result showed up in Bing.
Google claimed that Bing must have "stolen" the search result because it's not linked from any other page, so there was no organic way for Bing to find it. But it seems pretty normal to me.
But it seems pretty normal to me.
This assertion appears to be supported only by your use of scare-quotes around "caught." Is there any further detail you can supply as to why Google's charge should be treated with skepticism? How else would the page have gotten into Bing?
This assertion appears to be supported only by your use of scare-quotes around "caught." Is there any further detail you can supply as to why Google's charge should be treated with skepticism? How else would the page have gotten into Bing?
I think the question isn't how the page got into Bing, but whether that's a problem. According to sp332, Google ran their experiment with a Microsoft toolbar installed. That toolbar is reporting back to MS about what you visit, and that's how the page got into Bing. Nothing nefarious there, and not "stealing" results, but simply monitoring users.
Google never complained about the page getting into Bing. They complained that Bing indexed it with the exact (phony) search term that Google did--Google purposefully didn't include said term on the page itself.
Fair enough!
His post explains how. Did you read it? Installing the Bing toolbar opted users into gathering data on search traffic (including where a search ended up). Bing used that data to add pages to their index.
Microsoft's excuse ultimately did check out. That's still irrelevant with respect to my original point: it was a PR loss for them both because it revealed the ugly mudslinging sides of both companies and detracted from their images.
NameCheap has appeared to be "out for blood" since the opportunity arose with goDaddy's PR debacle with SOPA. And it's leaving both company's tarnishing their reputation.
goDaddy supports a draconian censorship bill. nameCheap jumps in the game and does their best to kick goDaddy while their down for personal gain. The capitalistic dream in action.
Want to transfer your domains to a registrar with merit and is actually against SOPA instead of trying to gain customers? Find one who was talking about how bad SOPA was before this debacle started last week.
goDaddy supports a draconian censorship bill. nameCheap jumps in the game and does their best to kick goDaddy while their down for personal gain. The capitalistic dream in action.
Want to transfer your domains to a registrar with merit and is actually against SOPA instead of trying to gain customers? Find one who was talking about how bad SOPA was before this debacle started last week.
I'm not a customer but I noticed Dreamhost who do domains were publicly opposing Sopa in November: http://blog.dreamhost.com/2011/11/22/dont-drop-the-soap-drop...
GoDaddy's response sounds like "well, if they would have told us what they wanted we might have given them access to secret privileges we have given others."
If that's indeed what they're trying to say, it's not particularly reassuring.
If that's indeed what they're trying to say, it's not particularly reassuring.
How so? A lot of public APIs work this way, there is a free limit and you can do x requests and then if you want to do more you get the limit raised, I've used a bunch of APIs with a similar limit policy. The only problem is if GoDaddy refused to do this for Namecheap, which doesn't appear to be the case, it seems Namecheap complained publicly first and didn't contact GoDaddy at all.
But they didn't, as they stated they didn't. At the end of the day, as Namecheap points out, it's against the ICANN rules to prevent whois's and frankly I wish ICANN had the teeth to trash GoDaddy's registrar status.
There seems to be two different conversations going on here. Namecheap says GoDaddy was "returning incomplete WHOIS information to Namecheap". GoDaddy says it was a rate limiter. In all the work I've done with external APIs, I don't think I would ever refer to being rate limited as getting incomplete results.
The "response" from Namecheap is pretty lame, they do not mention that this is standard and that if they'd contacted GoDaddy it would have been resolved and that they didn't before blaming them publicly:
> *Update 12:45 PT – GoDaddy has confirmed they have finally unblocked our queries. The transfer queue is being cleared and all transfers should go smoothly from here on. Many thanks to our customers and supporters for bringing this issue the attention it deserved!
"finally unblocked" implies that GoDaddy refused to fix it and is at fault, not Namecheap. GoDaddy have always had whois limits as every one who has tried to use their whois system will know (it's a big message on the page)
> *Update 12:45 PT – GoDaddy has confirmed they have finally unblocked our queries. The transfer queue is being cleared and all transfers should go smoothly from here on. Many thanks to our customers and supporters for bringing this issue the attention it deserved!
"finally unblocked" implies that GoDaddy refused to fix it and is at fault, not Namecheap. GoDaddy have always had whois limits as every one who has tried to use their whois system will know (it's a big message on the page)
It looks like they did reach out to them, according to this post at least: http://community.namecheap.com/blog/2011/12/26/godaddy-trans...
See, just like I said earlier, Namecheap should have contacted GoDaddy first instead of publicly slamming them without even trying to make it better for their clients.
NameCheap's folks have stated pretty unequivocally that they did contact GoDaddy ahead of time. I tend to believe them over GoDaddy.
http://community.namecheap.com/blog/2011/12/26/godaddy-trans...
http://community.namecheap.com/blog/2011/12/26/godaddy-trans...
They did. Reading helps.
Domain transfers was an afterthought to the multi-registrar model and was designed and implemented as policy (i.e. ICANN), not as a technical process (i.e. IETF). I wish we'd done a better job in 1999.