Twitter Goes JSON-Only With One API(blog.programmableweb.com)
blog.programmableweb.com
Twitter Goes JSON-Only With One API
http://blog.programmableweb.com/2010/11/10/twitter-goes-json-only-with-one-api-more-to-come/
48 comments
I disagree on readability, a big chunk of deeply nested JSON is not easy to read at all. Most smart XML viewers, including most browsers, have folding and coloring which make it pretty easy to read.
I agree about the smart xml viewers and the fact that most browsers support the friendly treeview's etc... However shouldn't browsers just as easily be able to do the same with JSON? And I believe most JSON viewers I've seen give you the same type of presentation of objects and lists...
The only thing I miss from XML is the ability to comment out something. I discovered the other day that it wasn't apparently possible.
Formats targeted strictly at machine/machine interchange ultimately get used for much more. JSON is no exception.
Configuration tends to be the first place that interchange formats are "abused"; even for such a mild abuse, comments are essential. (Hard-core abuses, like XML-based programming languages, are another thing entirely. :-)
To deal with JSON's lack of comments, I've considered writing services and apps that produce JSON, but consume YAML. Here, the interesting qualities of YAML are (1) it's a superset of JSON, and (2) it supports comments.
Configuration tends to be the first place that interchange formats are "abused"; even for such a mild abuse, comments are essential. (Hard-core abuses, like XML-based programming languages, are another thing entirely. :-)
To deal with JSON's lack of comments, I've considered writing services and apps that produce JSON, but consume YAML. Here, the interesting qualities of YAML are (1) it's a superset of JSON, and (2) it supports comments.
YAML isn't a strict superset of JSON, although since YAML 1.2 it is very nearly so; with just a little bit of care you can avoid the edge cases (eg. you can have hash keys longer than 1024 characters in JSON but not in YAML), provided you are cognizant of them.
[deleted]
I'm quite surprised. What did you need the comments for?
I had a chef json configuration file which I was creating and testing as I went. In this file I had to remove 3 "recipes" and one "attribute" temporarily.
Because of the lack of comments I copied the old version to a new file etc.
Because of the lack of comments I copied the old version to a new file etc.
If you don't care about the standard, you can use / * * / to comment out stuff in JSON. It's a shame this wasn't in the spec. Along with making quotes optional for keys.
It's a real shame that the native JSON object, and the spec missed out these useful features...
I love JSON, but making quotes compulsory for keys, and missing out comments are both a PITA.
It's a real shame that the native JSON object, and the spec missed out these useful features...
var obj = eval("({foo:3 /*,bar:7*/ })");
I love JSON, but making quotes compulsory for keys, and missing out comments are both a PITA.
Depends on your language - in Scala XML is so much easier than JSON, mostly because Scala allows you to do pattern-matching and do functional programming on Xml.
But if you are working in Javascript, then yeah it is properly easier.
But if you are working in Javascript, then yeah it is properly easier.
A little rude, isn't that? There's different fundamental metaphors: XML is a way of representing a tree with labeled nodes, and also a markup format; JSON is the representation of arrays, simple maps, and basic types in unicode. Each has its uses.
Their uses overlap in that they both broadly serve as a language for defining structured data being transfered between two different systems. For most use cases in this space, JSON is a) expressive enough and b) far easier to render, read and parse.
Very well put. Especially B.
Whatever the merits of the two methods, making the API a moving target really sucks for anyone might, say, be in the middle of writing to it.
Other than smaller bandwidth on each call, what advantages does JSON provide over XML?
Direct mapping to most (dynamic) programming languages' datatypes, such as arrays, dictionaries, numbers and strings.
Once you get used to working with a combination like JSON and Python, you really don't want to go back to the complexity and inconvenience of parsing and generating XML.
Once you get used to working with a combination like JSON and Python, you really don't want to go back to the complexity and inconvenience of parsing and generating XML.
But still, deprecating XML means lots of already functional apps won't work anymore. Maybe a "All new app registered after X date will only be able to use JSON formatted data" would be better.
They're not ditching XML for their regular APIs, just for the streaming API - which has been in beta for just a few months, and probably doesn't have that many XML clients (I imagine Twitter looked at the number of requests the XML endpoint was getting before deciding to deprecate it).
XML is actually very hairy and complicated. Consider entity encoding, whitespace (as in xml:space="preserve"), and namespaces. In contrast, the complete syntax of JSON is given in 5 simple flowcharts: http://json.org/
Nearly any competent programmer could hand-roll a tested, production-ready JSON parser in under a week. Try doing the same with XML. In practice there are plenty of JSON parsers out there to choose from, but being able to completely wrap your head around something is useful.
Nearly any competent programmer could hand-roll a tested, production-ready JSON parser in under a week. Try doing the same with XML. In practice there are plenty of JSON parsers out there to choose from, but being able to completely wrap your head around something is useful.
I upvoted your comment for the info, but I have no idea why this is an advantage for anyone except the handful of people who create an implementation. Hand-rolling an XML parser is a pretty sure sign of a certain kind of incompetence. Wouldn't the same be true of hand-rolling a JSON parser, unless you're one of the (rare) first guys to solve the problem for your language or language platform? To put it another way, wouldn't this only have been an advantage before XML libraries became commonplace? The last time I reached for an XML library and found there was none already installed with my platform, I was using Tcl (and it wasn't hard to find TclXML.)
Benefits JSON > XML: Less bandwidth for same data, Quick to load, quick to parse. Very web friendly.
Benefits of XML > JSON: Easy to validate structure and content (via schemas), and transform content in a language neutral manner (via xslt). Older, thus supported in more places.
Use what works - JSON is fine for web stuff, XML is better for things that need strict validation and/or long term data storage.
Benefits of XML > JSON: Easy to validate structure and content (via schemas), and transform content in a language neutral manner (via xslt). Older, thus supported in more places.
Use what works - JSON is fine for web stuff, XML is better for things that need strict validation and/or long term data storage.
Regarding schemas - there is JSON Schema:
http://json-schema.org/
http://tools.ietf.org/html/draft-zyp-json-schema-02
We've been using this pretty extensively at work. The Java guys wrote something to generate their model from it and on the JavaScript side we've used it to do all kinds of nice things like automagically generating forms that validate according to the schema and verifying that the data we receive from the backend conforms to it in our tests.
From what I can gather very few people are using it. It's not exactly fun to write but a co-worker of mine wrote a dsl to make that less painful.
Of course when you write JavaScript, it's a joy to work with.
On the transformation side there are a few ideas floating, notably JSON-T
http://goessner.net/articles/jsont/
Neither of these things are as mature as XSD and XSLT, but if you love JSON and JavaScript, they're definitely worth investigating.
http://json-schema.org/
http://tools.ietf.org/html/draft-zyp-json-schema-02
We've been using this pretty extensively at work. The Java guys wrote something to generate their model from it and on the JavaScript side we've used it to do all kinds of nice things like automagically generating forms that validate according to the schema and verifying that the data we receive from the backend conforms to it in our tests.
From what I can gather very few people are using it. It's not exactly fun to write but a co-worker of mine wrote a dsl to make that less painful.
Of course when you write JavaScript, it's a joy to work with.
On the transformation side there are a few ideas floating, notably JSON-T
http://goessner.net/articles/jsont/
Neither of these things are as mature as XSD and XSLT, but if you love JSON and JavaScript, they're definitely worth investigating.
Using a schema to frontend/backend form validation sounds totally awesome from a "don't repeat yourself" perspective.
If that code becomes worthy of public release, I'd bet it could dramatically raise the profile of JSON Schema.
If that code becomes worthy of public release, I'd bet it could dramatically raise the profile of JSON Schema.
I never really got the 'why' behind XML Schemas. Why would you validate a document with a schema vs actual code? It's just a bigger pain in the ass, and more limited.
If you have multiple applications using a document format, XML schemas are a good way to make sure they all agree on the semantics of the format. Recall that XML was originally intended as a means of bridging different environments and platforms, where code will not necessarily be portable.
I guess, but even if that's the case, who cares if the message is formatted right if it's still invalid.
I guess I haven't really worked in a heterogenous environment with a lot of XML before though, I can see the potential value there, but i'm still doubtful.
I guess I haven't really worked in a heterogenous environment with a lot of XML before though, I can see the potential value there, but i'm still doubtful.
The big win for schemas is being able to use them to make assumptions about input, which simplifies your code.
For example, if you have an RelaxNG schema (which has a great compact syntax) that says that an element has to have at least one child node and that child node contains an integer between a certain set of values. Once you have that schema, you could write code that could read in the XML file and validate it against the schema in 2 lines, then grab all the child node integers with one XPath expression.
The data might be junk (heck, I'm not aware of any format that is impervious to worthless data), but at least it's junk in the right format, and you never had to mess around with parsing the input.
Need to switch programming environments or languages because you're working on Unix/Windows/embedded system/mainframe/database/web browser ? The schema can move with you (or be converted to another schema format that does), and programming niceties SAX and XPath will often carry over too.
For example, if you have an RelaxNG schema (which has a great compact syntax) that says that an element has to have at least one child node and that child node contains an integer between a certain set of values. Once you have that schema, you could write code that could read in the XML file and validate it against the schema in 2 lines, then grab all the child node integers with one XPath expression.
The data might be junk (heck, I'm not aware of any format that is impervious to worthless data), but at least it's junk in the right format, and you never had to mess around with parsing the input.
Need to switch programming environments or languages because you're working on Unix/Windows/embedded system/mainframe/database/web browser ? The schema can move with you (or be converted to another schema format that does), and programming niceties SAX and XPath will often carry over too.
I can totally see the value now. I haven't used XML in heterogenous environments much, but I can see the value now.
Thanks for the careful response!
Thanks for the careful response!
Without a neutral schema language, the validation rules need to be reimplemented in every programming language used by anyone who works with the format. And there's almost no chance they will exactly agree on which documents are valid or not, because you're either playing telephone between one implementation and the next or just starting from a vague schema in prose.
When you say "every programming language" I assume you mean "JavaScript"?
I would not be surprised if embedding a javascript interpreter and running a validation script was probably already easier in most languages than embedding an XML parser & validator.
JavaScript has become the lingua franca of programming. I hope they teach that in schools. http://en.wikipedia.org/wiki/Lingua_franca
I would not be surprised if embedding a javascript interpreter and running a validation script was probably already easier in most languages than embedding an XML parser & validator.
JavaScript has become the lingua franca of programming. I hope they teach that in schools. http://en.wikipedia.org/wiki/Lingua_franca
I would expect a JavaScript interpreter to be even slower than a streaming XML parser. And one good thing about XML Schema is that it doesn't require DOM, in fact it doesn't say anything about representing document data in memory. This is also true of the JSON Schema draft. But hand-coded JavaScript validation rules aren't likely to be context-free, and we'd constantly be fighting people who assume everyone can afford to store the entire document as one huge DOM-like map of arbitrary keys and values of unknown types. It's possible to build a validator on top of something like http://jackson.codehaus.org/, but I don't believe the industry has the diligence to do it that way.
"Worse is Better" says it much better than I can, the dynamics of success are not what we may first think they are.
From one point of view, JavaScript is the "Worse is Better" version of Lisp. Guess who's winning?
Ditto XML vs JSON
From one point of view, JavaScript is the "Worse is Better" version of Lisp. Guess who's winning?
Ditto XML vs JSON
Also, writing validation code by hand leads to a lot of duplication and room for error.
[deleted]
Here's a piece I wrote about JSON v.s. XML back in 2006: http://simonwillison.net/2006/Dec/20/json/
Hooray! Let's hope other websites follow their lead and ditch the huge SOAP/XML overhead and stick with the ease and simplicity of REST/JSON.
At Yell we're currently working on our first public API, and only on Friday did we have the conversation and decided that doing JSON well was far more important than supporting XML at all.
The decision was a quick one, we just asked the iPhone, Android and web devs where we should focus, what they wanted most. The answer was unanimously JSON.
The decision was a quick one, we just asked the iPhone, Android and web devs where we should focus, what they wanted most. The answer was unanimously JSON.
Twitter's XML API does not use SOAP.
Sigh
I just implemented my project's API in XML. Time to switch I guess...
I just implemented my project's API in XML. Time to switch I guess...
Since the 1st line of the article indicates this is the streaming API only, I will presume that's the one your project is written against.
On XML vs. JSON, I used to hate XML until I started programming in ActionScript which has a really nice syntax for it (except a few oddities here and there). I use Enunciate at work ( http://enunciate.codehaus.org/ ) which generates XML, JSON, AMF, SOAP/REST endpoints all by itself from base Java services and classes, which works well too.
Can't say I'm disappointed, back when I did twitter dev I used JSON for everything, plus it was really easy to interface with our JavaScript client (duh).
Can't say I'm disappointed, back when I did twitter dev I used JSON for everything, plus it was really easy to interface with our JavaScript client (duh).
I haven't used Action script but Scala has really nice support for Xml - it is properly the only language that allows you to do pattern matching on XML.
Scala's creator is on record as regretting the inclusion of XML literals in the syntax of the language:
http://www.codecommit.com/blog/scala/working-with-scalas-xml...
http://www.codecommit.com/blog/scala/working-with-scalas-xml...
Impedance mismatch between XML and JavaScript is killing XML.
What's next? HTML? {"head":{"title": "JOHN"},"body:"{"div": "Javascript Object Hypertext Notation"}}
What's next? HTML? {"head":{"title": "JOHN"},"body:"{"div": "Javascript Object Hypertext Notation"}}
The article on Programmable Web doesn't add any value to the original press release:
http://groups.google.com/group/twitter-development-talk/brow...
XML has its uses but honestly when it comes down to it working with JSON is a lot easier. When picking a data interchange format you need to consider indirect advantages as well. JSON is easier to read on a screen, type out, and overall skim. Skimming XML is a pain.