Learning from our Mistakes: The Failure of OpenID, AtomPub and XML on the Web(25hoursaday.com)
25hoursaday.com
Learning from our Mistakes: The Failure of OpenID, AtomPub and XML on the Web
http://www.25hoursaday.com/weblog/2011/01/30/LearningFromOurMistakesTheFailureOfOpenIDAtomPubAndXMLOnTheWeb.aspx
45 comments
I think AtomPub suffers from the same problems that SOAP does. It's too complex for what it does, or at least it feels that way. The fact that major player have adopted it doesn't change things, as many major players adopted SOAP.
I don't think it (or SOAP) will just vanish though, it just isn't having the impact its creators had hoped it would.
I don't think it (or SOAP) will just vanish though, it just isn't having the impact its creators had hoped it would.
I'm surprised that perception would exist. It's really a shame if that's (in addition to JSON hype[1]) what has killed Atom.
The main difference between SOAP and Atom is that I can explain the latter to anyone in 5 minutes. Actually that's what some Google engineers have exactly done and you can find the results on YouTube.
And there's plenty of XML APIs and JSON APIs and each one of that APIs reinvents the concepts of feeds and entries, because those concepts are very natural.
Anyway, I understand that AtomPub just isn't very popular[2], but I was interested in any technical difficulties that may arise for both services providers and consumers. That is, if I were to implement API for NextBigThing and were to implement AtomPub-based API, could that be a problem for anyone?
Well, I guess it's a shame I haven't had a chance to get this discussion going while this story was still on the front page…
[1] Which went from "XML is too complex, fuck it; Let's just dump our internal data structures and be done with it" to "Oh, now I understand what that schema thing is supposed to do; it's kinda neat. Let's just re-implement it in JSON."
[2] Actually it seems extremely anti-popular if you just go to ProgrammableWeb and filter several thousand APIs by protocol; there's only one page of results!
The main difference between SOAP and Atom is that I can explain the latter to anyone in 5 minutes. Actually that's what some Google engineers have exactly done and you can find the results on YouTube.
And there's plenty of XML APIs and JSON APIs and each one of that APIs reinvents the concepts of feeds and entries, because those concepts are very natural.
Anyway, I understand that AtomPub just isn't very popular[2], but I was interested in any technical difficulties that may arise for both services providers and consumers. That is, if I were to implement API for NextBigThing and were to implement AtomPub-based API, could that be a problem for anyone?
Well, I guess it's a shame I haven't had a chance to get this discussion going while this story was still on the front page…
[1] Which went from "XML is too complex, fuck it; Let's just dump our internal data structures and be done with it" to "Oh, now I understand what that schema thing is supposed to do; it's kinda neat. Let's just re-implement it in JSON."
[2] Actually it seems extremely anti-popular if you just go to ProgrammableWeb and filter several thousand APIs by protocol; there's only one page of results!
The other day I wanted to log in to my StackOverflow account. I was presented with this screen: http://i.imgur.com/SwuVJ.png
Shoot, did I previously log in to SO with my Google account? With my Facebook account? How can I remember? Or maybe, I created a "myOpenID" account? After trying various combinations, I became frustrated by the error "No OpenID endpoint found."
Finally, I ended up spending twenty minutes digging through my old e-mail to figure out that I did, in fact, create a dedicated OpenID account at myopenid.com.
It turned out, for some ungodly reason, that I have to enter my OpenID as "http://<my username>.myopenid.com/" -- I would not have ever guessed this.
As a developer, I can appreciate the motivation behind OpenID. But the execution is simply frustrating.
Shoot, did I previously log in to SO with my Google account? With my Facebook account? How can I remember? Or maybe, I created a "myOpenID" account? After trying various combinations, I became frustrated by the error "No OpenID endpoint found."
Finally, I ended up spending twenty minutes digging through my old e-mail to figure out that I did, in fact, create a dedicated OpenID account at myopenid.com.
It turned out, for some ungodly reason, that I have to enter my OpenID as "http://<my username>.myopenid.com/" -- I would not have ever guessed this.
As a developer, I can appreciate the motivation behind OpenID. But the execution is simply frustrating.
I don't get why people create ton of OpenID identities, then use them in a way they can't remember which one they use where. I have one "main" OpenID URI per identity (with several "backup" ones, which I use on sites who allow me to associate multiple OpenID URIs with the account) and have never experienced such kind of problem.
OpenID is fine in this regard. The root of the problem in lack of site with a short and concise explanation what OpenID is, and best-practice tips on how it should be used. With all contents in public domain or under very non-restrictive free license.
Also, I'd note that the problem you mention is the same for passwords/passphrases. I.e. you have to remember whenever you used one password generator or another (I, unfortunately, have two, because due to way-too-smart sites which decided they won't accept some ASCII non-alphanumeric characters, passwords made by first one, while being secure, weren't allowed to use), or the site was a special case where you typed man-made password. "Did I use my X password here? Or my Y password?" - it's exactly the same.
OpenID is fine in this regard. The root of the problem in lack of site with a short and concise explanation what OpenID is, and best-practice tips on how it should be used. With all contents in public domain or under very non-restrictive free license.
Also, I'd note that the problem you mention is the same for passwords/passphrases. I.e. you have to remember whenever you used one password generator or another (I, unfortunately, have two, because due to way-too-smart sites which decided they won't accept some ASCII non-alphanumeric characters, passwords made by first one, while being secure, weren't allowed to use), or the site was a special case where you typed man-made password. "Did I use my X password here? Or my Y password?" - it's exactly the same.
If you forget your password, you can have the site reset it for you, as long as you have your email, generally. You can't have the site tell you which OpenID endpoint you used though.
That you think OpenID fails because there is no concise explanations of best practices and tips is damning enough. Most users aren't going to read that stuff.
That you think OpenID fails because there is no concise explanations of best practices and tips is damning enough. Most users aren't going to read that stuff.
> as long as you have your email
Email is exactly the same as OpenID in this regard. Your forgot-which-OpenID-was-used could be compared to forgetting which email address was used. And this is, actually, popular. It's just a hype that everyone's talking about OpenID - totally forgetting that traditional systems have the same problems.
If you're doing it right - by having one primary OpenID URI (per identity) - you won't really forget what your OpenID is.
> Most users aren't going to read that stuff.
Nor passwords, nor OpenID were ever intended to work around this kind of problems. And I doubt there's any solution at all. Users will always forget all sort of things, use and reuse totally insecure passwords, keep their backdoor open wide with silly "password recovery" questions anyone could guess, leak all kinds of sensitive information and whatever else they could do wrong.
Sadly, "OpenID sucks" became a meme. And this is the main reason why OpenID suck now.
Email is exactly the same as OpenID in this regard. Your forgot-which-OpenID-was-used could be compared to forgetting which email address was used. And this is, actually, popular. It's just a hype that everyone's talking about OpenID - totally forgetting that traditional systems have the same problems.
If you're doing it right - by having one primary OpenID URI (per identity) - you won't really forget what your OpenID is.
> Most users aren't going to read that stuff.
Nor passwords, nor OpenID were ever intended to work around this kind of problems. And I doubt there's any solution at all. Users will always forget all sort of things, use and reuse totally insecure passwords, keep their backdoor open wide with silly "password recovery" questions anyone could guess, leak all kinds of sensitive information and whatever else they could do wrong.
Sadly, "OpenID sucks" became a meme. And this is the main reason why OpenID suck now.
What's up with all the OpenID hating?
I personally find OpenID totally awesome (not that it couldn't be better) when I want to try a new service and don't want to go through a registration form or hand out my password to an untrusted website or when I'm too lazy to type my username/password.
I personally find OpenID totally awesome (not that it couldn't be better) when I want to try a new service and don't want to go through a registration form or hand out my password to an untrusted website or when I'm too lazy to type my username/password.
I think OpenID ran into serious trouble when a bunch of big sites announced support for it, but really only supported it as providers, not consumers.
At one point, I had something like 3 or 4 different sites I was registered on that would serve as OpenID providers for me, but not a single place that I wanted to login to accepted OpenID. At that point, I pretty much forgot about OpenID. I bet a lot of would-be early adopters did the same.
By the time I actually started occasionally encountering sites that accepted OpenID and that I wanted to use, I had 1Password and didn't care about OpenID anymore.
At one point, I had something like 3 or 4 different sites I was registered on that would serve as OpenID providers for me, but not a single place that I wanted to login to accepted OpenID. At that point, I pretty much forgot about OpenID. I bet a lot of would-be early adopters did the same.
By the time I actually started occasionally encountering sites that accepted OpenID and that I wanted to use, I had 1Password and didn't care about OpenID anymore.
Is XML considered a failure? (Not being snarky, just that I find it in use everywhere.)
XML is a failure compared to the totally unjustified hype about how it would replace all other data formats and lead us into a new computing nirvana. Compared to the goals of its creators, XML is quite successful.
He's talking about XML on the web, not XML in general. XML is not a failure in general (although it should be...).
XML is incredibly powerful, somewhat complex, and therefore misunderstood.
It needs an "Modern XML" like the "Modern Perl" that's going around - a take from someone who has used it to solve problems and goes over the bright points and black marks.
For example, to the neophyte, do you pick DTD, XML Schema, RelaxNG, or Schematron to solve data validation task X? How about task Y?
Too many people tried to use XML as a CSV or other structured data replacement, where JSON is usually superior. I doubt that anyone thinks that JSON is a HTML replacement, but XML can be.
It comes down to different tool, different job, and using the wrong tools can leave a sour taste in one's mouth about a technology - how many people here have similar feelings about Java or Windows?
It needs an "Modern XML" like the "Modern Perl" that's going around - a take from someone who has used it to solve problems and goes over the bright points and black marks.
For example, to the neophyte, do you pick DTD, XML Schema, RelaxNG, or Schematron to solve data validation task X? How about task Y?
Too many people tried to use XML as a CSV or other structured data replacement, where JSON is usually superior. I doubt that anyone thinks that JSON is a HTML replacement, but XML can be.
It comes down to different tool, different job, and using the wrong tools can leave a sour taste in one's mouth about a technology - how many people here have similar feelings about Java or Windows?
I don't think it's fair to call XML a failure any more than you can call Perl a failure. XML does the job, and it's still used in a lot of places. It's just that it's being supplanted by a newer technology that is superior for the majority of use-cases.
XML is being abandoned as the medium for a variety of APIs at the moment (Google and Facebook especially). It looks like failure when my XSLT code deprecated before I can even release it.
umm do you mean "superior" in regards to capabilities or regards to simplicity. You could just as well argue that JSON replacing XML is akin to XML replacing SGML.
IMHO the only issue with OpenID in regard to its adoption is how it is presented to the user. He/she shouldn't have to deal with URLs or even the term 'OpenID'. Unfortunately the current UIs & the whole UX are solely engineer-driven...
Showing OpenID term makes sense as it's branded to be an alternative access choice for users who wish to use OpenID. You are right in that it's poor UI/UX but unfortunately it's the implementation of 3rd parties.
Facebook is implementing a similar proprietary alternative login: FacebookConnect.
Facebook is implementing a similar proprietary alternative login: FacebookConnect.
XML is not a failure and it's also not a failure on the web (XHTML? RDF? FOAF? &c... huge successes). OpenID also isn't a failure - it could stand to be supplanted by something better but it had to start somewhere.
>XHTML? RDF? FOAF? &c... huge successes
You're kidding, right?
You're kidding, right?
No I'm not. What in my tone makes you think I'm kidding? Could you please provide a counter argument instead of a flame bait comment?
XHTML hasn't revolutionised web page markup (it brought with it draconian changes {read: made web page markup more XML-like} and is now being replaced by HTML5, which makes some {all?} of its changes irrelevant).
RDF isn't as widespread as it could be (it could be used in Facebook and Wikipedia but isn't, for example).
(And I haven't even heard of FOAF.)
I wouldn't call these huge successes.
RDF isn't as widespread as it could be (it could be used in Facebook and Wikipedia but isn't, for example).
(And I haven't even heard of FOAF.)
I wouldn't call these huge successes.
XHTML has revolutionized page markup. Not because everyone is using it but because it blazed the path for better standards that the other document types follow. Also, many of the changes in XHTML were brought over to HTML5 - which does have an XML definition.
"Also, many of the changes in XHTML were brought over to HTML5 - which does have an XML definition."
Yea, the so-called XHTML5 that will be finally supported in IE9, BTW.
Yea, the so-called XHTML5 that will be finally supported in IE9, BTW.
Just out of curiosity, what is an example of a draconian change that was added by XHTML?
If you're properly serving XHTML 1.1, you're supposed to use the application/xhtml+xml Content-Type, which means browsers are supposed to FAIL to render the page if there are any well-formedness erors.
It turns out that's a really bad idea on the Web.
Further reading: http://diveintohtml5.org/past.html
It turns out that's a really bad idea on the Web.
Further reading: http://diveintohtml5.org/past.html
I guess reality is the best counter argument to your claims. RDF does not exist outside of academic toys and FOAF is so statistically irrelevant that it can easily be said that it never really existed in the first place. As the original article points out, XML may have a place in certain forms of document exchange, but as far as the web is concerned it is mostly just one failure after another.
FOAF is irrelevant in many respects, I'll give you that; but, RDF certainly does exist outside of academic toys.
Big names that I know of that use/publish RDF:
http://www.bbc.co.uk/blogs/bbcinternet/2010/07/the_world_cup...
http://wiki.freebase.com/wiki/RDF
http://richard.cyganiak.de/blog/2009/10/linked-data-at-the-n...
http://newsbreaks.infotoday.com/NewsBreaks/SciVerse-Hub-Appl...
http://en.wikipedia.org/wiki/Evri
[EDIT] Don't forget DBpedia and FreeBase (which uses RDF heavily) is used heavily by PowerSet which subsequently was bought by Microsoft which now (as far as I know) is used heavily for Bing.
Big names that I know of that use/publish RDF:
http://www.bbc.co.uk/blogs/bbcinternet/2010/07/the_world_cup...
http://wiki.freebase.com/wiki/RDF
http://richard.cyganiak.de/blog/2009/10/linked-data-at-the-n...
http://newsbreaks.infotoday.com/NewsBreaks/SciVerse-Hub-Appl...
http://en.wikipedia.org/wiki/Evri
[EDIT] Don't forget DBpedia and FreeBase (which uses RDF heavily) is used heavily by PowerSet which subsequently was bought by Microsoft which now (as far as I know) is used heavily for Bing.
Freebase wasn't built around RDF - they ended up emitting RDF because it made sense to do so, but the core APIs the service is built on are based around JSON, not RDF.
I don't see how anyone could regard those technologies as huge successes. All three would be in my list of technologies which were overhyped and hugely underdelivered.
XHTML was a failure. Everyone stuck with HTML4 and advantages of XHTML haven't materialised. I question the whole reasoning behind trying to retrofit HTML into XML.
The "semantic web" has never delivered on its promises, despite hanging around like a bad smell. Very few sites use RDF (or FOAF). The real-world problems that RDF was supposed to solve are, in practice, solved by simple JSON-based Web APIs.
The only XML-based web technology to become a success is RSS.
XHTML was a failure. Everyone stuck with HTML4 and advantages of XHTML haven't materialised. I question the whole reasoning behind trying to retrofit HTML into XML.
The "semantic web" has never delivered on its promises, despite hanging around like a bad smell. Very few sites use RDF (or FOAF). The real-world problems that RDF was supposed to solve are, in practice, solved by simple JSON-based Web APIs.
The only XML-based web technology to become a success is RSS.
I was with him until the last paragraph. The NoSQL trend is about bringing more options to the table when solving your specific problem, not trying to take a solution to a specific problem and make it general (as he claims OpenID, AtomPub, and XML do). He's right in saying developers should evaluate it and lord knows there's a lot of hype, but it's hardly related.
Now, if SQL were the new thing and had been developed by a group trying to make a standard API for CRUD operations based on some new generalized "Relational Algebra" concept that they extracted out of a couple of company payroll applications, I'd say he was onto something.
Now, if SQL were the new thing and had been developed by a group trying to make a standard API for CRUD operations based on some new generalized "Relational Algebra" concept that they extracted out of a couple of company payroll applications, I'd say he was onto something.
I think one of the mistakes this article makes is believing that people know what they want upfront. We all know that people are very poor judges of what could and couldn't be useful to them before you actually give it to them.
OpenID should probably have seen it was overly complex, but I'm not sure how anyone could have guessed that XML was going to be shunned in favor of JSON. It does have some pretty cool advantages, such as DTDs ensuring correctness. It's just that it got a lot of typing real fast (and probably was never supposed to be written by humans anyway) and the space sizes blew up too.
OpenID should probably have seen it was overly complex, but I'm not sure how anyone could have guessed that XML was going to be shunned in favor of JSON. It does have some pretty cool advantages, such as DTDs ensuring correctness. It's just that it got a lot of typing real fast (and probably was never supposed to be written by humans anyway) and the space sizes blew up too.
From personal experience I think the reason JSON won over XML is simply because it maps to OOP/dictionaries much better.
In most scripting languages reading JSON looks like this: x = json.parse(string) and saving it looks like this string = json.dump(x) ... The rest of the app looks the same as if JSON was never in the picture at all.
Now with XML, hah, in most scripting languages you have to instantiate a parser, then use strange xpath traversal methods that are different depending on which parser you use and os on. And don't even get me started on generating xml programmatically ...
Sure, I understand why handling XML has to be so much more complex, it's got attributes and a few other nifty things I will never have a use for.
JSON vs XML --> simplicity always wins over more features.
In most scripting languages reading JSON looks like this: x = json.parse(string) and saving it looks like this string = json.dump(x) ... The rest of the app looks the same as if JSON was never in the picture at all.
Now with XML, hah, in most scripting languages you have to instantiate a parser, then use strange xpath traversal methods that are different depending on which parser you use and os on. And don't even get me started on generating xml programmatically ...
Sure, I understand why handling XML has to be so much more complex, it's got attributes and a few other nifty things I will never have a use for.
JSON vs XML --> simplicity always wins over more features.
JSON won because there's a straightforward mapping between a JSON document and the language's native arrays, maps and values.
The remainder of the problems you list are not problems. There are one line parse and dump functions for XML. You don't need to use xpath to traverse a document. The result of parsing a document is the same across each OS (at least in Python, Ruby, Java, Go and other languages that I have used). An application shouldn't need to use more than one parse, so it does not matter that different parsers represent the DOM differently.
The remainder of the problems you list are not problems. There are one line parse and dump functions for XML. You don't need to use xpath to traverse a document. The result of parsing a document is the same across each OS (at least in Python, Ruby, Java, Go and other languages that I have used). An application shouldn't need to use more than one parse, so it does not matter that different parsers represent the DOM differently.
Back in the beginning some people did make the obvious observation that XML wasn't a good idea for data given that it was designed for documents. It's just taking a decade to roll back the excess hype.
Interesting summary and juxtaposition of various opinions on these technologies.
The fact that JSON has won over XML on the web says a lot about the technologies people are using to build websites these days. JavaScript client-side data processing seems to be a lot more popular than server-side processing with an XML-loving language like Java or C#.
The fact that JSON has won over XML on the web says a lot about the technologies people are using to build websites these days. JavaScript client-side data processing seems to be a lot more popular than server-side processing with an XML-loving language like Java or C#.
You're right, but I think it speaks less to client side and more to current trends in server side technologies. The #1 feature of JSON is that it maps directly into the dictionary and array built-in types of all the popular "scripting" languages (Perl, Python, Ruby, Javascript, PHP). This makes it absolutely trivial to work with in both the browser and the server unlike XML which demands either DOM traversal to extract data (yuck!), or XPath (cleaner, but yet another thing to learn).
I think this points to Ruby/Python/Perl/Javascript becoming more and more popular on the server side, even for "serious" apps.
I think this points to Ruby/Python/Perl/Javascript becoming more and more popular on the server side, even for "serious" apps.
I agree. JSON is inherently dynamically typed. Trying to make it work in a static language is a pain just like using XML in a static language.
Actually its more like on the server side the stacks are richer and more diverse so you can consume and easily use anything, XML, JSON, etc.... Whereas on the client you're pretty much stuck with Javascript. So the choice is obvious -- use the technology that works best with your most constrained environment.
Isn't the relative simplicity of JSON the bigger factor? It is for me.
You know, no one's saying this, but I'd just like to point to Douglas Crockford's accomplishment with JSON. Crocker thought of a really bare bones way to represent data that works in a huge number of cases people tried to use XML. XML has not lost, in any way, shape or form, and people will be using it to represent and structure data for a long time to come.
We can just be glad we have such a simple, common data structure before people start writing insanely complex data interchange formats on top of it. You could just as easily write SOAP in JSON.
We can just be glad we have such a simple, common data structure before people start writing insanely complex data interchange formats on top of it. You could just as easily write SOAP in JSON.
I'm slightly dubious as I'm reading on my mobile browser, but I'm getting a 404 for this page. Can anyone else see it?
I see it, but I clicked away as soon as I saw that the guy writes under the name "Carnage 4 Life" and is proudly quoting some ridiculous gangsta rappers in his headline. Whatever kind of content he then may have below that, then simply does not matter any more.
A guy posting as "muuh-gnu" responding to a post from "goopot" has no business complaining about the name someone else writes under.
And what was wrong with the quote? Do you disagree with it and think respect can be bought?
And what was wrong with the quote? Do you disagree with it and think respect can be bought?
Wow. Just wow. So ... the fact that he listens to hip-hop means he can't say anything of value? I guess you think Ben Horowitz (of Andreesen Horowitz) is an idiot too: http://bhorowitz.com/
You could have posted your comment as "NourielRoubini" or "LinusTorvalds" and it'd be just as xenophobic and worthless.
You could have posted your comment as "NourielRoubini" or "LinusTorvalds" and it'd be just as xenophobic and worthless.
It seems AtomPub isn't that terrible idea—you get uniform envelope format, so you can use feeds to represent your data in any way you like, because each entry has its own unique id. AtomPub let you use many conventions that web developers are already familiar with—e.g. <link rel="alt"/>, various extensions like OpenSearch etc.
If you are API author would you consider using AtomPub (or GData/OData)? Do you consider it too difficult/inconvenient to implement?
If you are using any AtomPub (GData/OData) based API what are your experiences? Do you consider it too difficult/inconvenient to consume?
[1] And provides JSON support: directly by translating XML tree as JSON structure (and I was sure they had something called cJSON, but now I can't find reference anywhere so I might have been dreaming that.)
[2] I don't know .NET world that good, so anyone please correct me if I'm wrong.