Twitter API: With Great Power Comes Great Responsibility(zetetic.net)
zetetic.net
Twitter API: With Great Power Comes Great Responsibility
http://www.zetetic.net/blog/2009/04/09/twitter-api-with-great-power-comes-great-responsibility/
2 comments
Summary: Twitter changed their API, we didn't check properly for duplicate tweets and screwed up our database.
You're misreading the article. The problem has to do with uncoordinated breaking changes to a public API. There were no problems for our apps.
Let me draw an analogy: imagine that one day before release MySQL (or pick your favorite DB) decided to deprecate and ignore the the greater than > operator on queries. No errors - it would just pretend that query clause didn't exist and return all rows for queries that used a >. How would that affect your application?
The point is that when you expose a public API it becomes a contract with your developer community. If you make a breaking change to the API you have a fundamental responsibility to effectively communicate the change and ensure that it doesn't create massive and unpredictable results for clients. In this case Twitter didn't do either.
Let me draw an analogy: imagine that one day before release MySQL (or pick your favorite DB) decided to deprecate and ignore the the greater than > operator on queries. No errors - it would just pretend that query clause didn't exist and return all rows for queries that used a >. How would that affect your application?
The point is that when you expose a public API it becomes a contract with your developer community. If you make a breaking change to the API you have a fundamental responsibility to effectively communicate the change and ensure that it doesn't create massive and unpredictable results for clients. In this case Twitter didn't do either.
Let me draw an analogy: imagine that one day before release MySQL (or pick your favorite DB) decided to deprecate and ignore the the greater than > operator on queries. No errors - it would just pretend that query clause didn't exist and return all rows for queries that used a >. How would that affect your application?
Well, most people would run their test suite, notice that MySQL broke, and then revert that MySQL update. Completely different.
When you are accepting untrusted data from the Internet, you need to validate it. If Twitter sends you invalid data, it is your loss, not theirs. Code accordingly.
Well, most people would run their test suite, notice that MySQL broke, and then revert that MySQL update. Completely different.
When you are accepting untrusted data from the Internet, you need to validate it. If Twitter sends you invalid data, it is your loss, not theirs. Code accordingly.
I got bitten by this as well, even though I'm checking for duplicates like you suggest -- it just jacked the total bandwidth usage way up, because calls previously returning 1 or 2 messages were suddenly returning the max every time.
Actually, we had 24 hours to change our code to adapt to the API change, and we were able to handle it quickly before it became a problem, we didn't end up with any duplicate tweets. The point of the article is that anybody who didn't see the announcement in time didn't really have a chance. It's a public API, there's a reasonable expectation here that a bit more notice would be given to such a change.
Definitely. I was bitten by this, and had my Twitter stream flooded with old tweets this morning from bots due to the removal of "since" in the API call. No notice, nothing. In fact, after it happened, I signed up for the API Google Group announcement list and only WAY after the fact did I get the official "we broke the API soz" email. Very very poor form.
Twitter doesn't have hundreds of millions of users.