One long, intimate month with the Facebook API(anilchawla.org)
anilchawla.org
One long, intimate month with the Facebook API
http://anilchawla.org/2011/02/23/one-intimate-month-facebook-api/
12 comments
I should say that I've been lurking on HN for quite some time and that this is my first post :-) One motivation for writing this post was that I was surprised to see how often the Facebook API will drop/omit data that exists and that you should be able to access. I specifically call out the Breakup Notifier app (which recently got a ton of buzz) because I don't see how apps like that can report accurate information when the underlying data is unreliable. Any thoughts or similar experiences with the Facebook API?
So this is how Breakup Notifier does it. It checks to see that the data is there. If at any point in time, the data I'm looking for is unavailable, I mark that field as "stale", don't vouch for the validity of its contents, and remove the ability for the user of the app to view that data.
When it comes back online, I mark it as not stale, and if it differs from the previous (non-stale) version, I send out an email.
When it comes back online, I mark it as not stale, and if it differs from the previous (non-stale) version, I send out an email.
Gotcha. I thought you might report updates only. The problem is that people can also remove their status instead of setting it back to "Single". Focusing on value updates only is probably fine and I'm sure nobody is going to freak out if the app misses a breakup :-P In my case, I was trying to create a more comprehensive notification that covered events such as new relationships (i.e. the user does not have a relationship status and one day finally sets it) and I couldn't afford the false positives.
We just work with the API within the usual scope (fb connect, etc) but it breaks regularly and things sometimes change without notice. It's really a joke for a company of that size. It has become kind of a running gag in the office (like: "Skip the tests, just release it. Today we do it facebook-style")
One motivation for writing this post was that I was surprised to see how often the Facebook API will drop/omit data that exists and that you should be able to access.
I loved the article, but I'm a bit surprised that you're surprised. What else would you expect from a massive NoSQL installation? It seems to me a given that they are going to trade some data integrity (in the form of missing attributes) for speed of access. I'd assume that was a fundamental design goal of the architecture.
I loved the article, but I'm a bit surprised that you're surprised. What else would you expect from a massive NoSQL installation? It seems to me a given that they are going to trade some data integrity (in the form of missing attributes) for speed of access. I'd assume that was a fundamental design goal of the architecture.
We run a program that monitors a significant number of Facebook Pages. It drives me crazy because every once in a while a certain attribute, like the author, is just omitted. It'll usually show up again later on.
Well, the graph API reports things as JSON. Your app has to page through the resulting json and see what information is relevant and meaningful.
I'm barely starting the Facebook API but the solution I'll use is to put the code for interpretating a given query into a series of configuration files using a DSL to interpret the Json. It's mostly a series of entries like:
I'm barely starting the Facebook API but the solution I'll use is to put the code for interpretating a given query into a series of configuration files using a DSL to interpret the Json. It's mostly a series of entries like:
foo = [bar or baz or "default"]
Regarding the Breakup Notifier reliability:
"I had my boyfriend test it out by changing his status several times waiting about 10 minutes in between, and I never received an email notification. The site supposedly crawls Facebook every ten minutes for changes so it shouldn’t take long to be notified."
http://datingdissection.com/2011/02/21/breakup-notifier-make...
http://datingdissection.com/2011/02/21/breakup-notifier-make...
Thanks for sharing. Your insights & warnings have been most helpful.
I'm the developer of Clarity (clarity-app.com), a Facebook Client for the Mac and I've had a similar experience. Here's another couple of things that are weird.
- you can't like pictures or even see likes for pictures
- you can't post on your wall if the attached image is located in your fcebook album: so if you want to post an image you have to upload it somewhere else.
- you can't rename friend lists
- lots of things are inaccessible if your friends have strict security settings (like their birthday, or their name)
- as outlined in the article: the notification data structure is a train wreck
- you can't approve friend requests
- some parts of the API are not documented at all, like the 'attachment' field on the posts table: it contains a dictionary, but in order to understand the keys and values you have to read the raw data from lots of posts, and even then you never know if one future entry breaks your app.
- and much more
One problem I had was that users expected much of this to work because it works on the Facebook iPhone app too. But that one is official and does not need to use the limited app API.
- you can't like pictures or even see likes for pictures
- you can't post on your wall if the attached image is located in your fcebook album: so if you want to post an image you have to upload it somewhere else.
- you can't rename friend lists
- lots of things are inaccessible if your friends have strict security settings (like their birthday, or their name)
- as outlined in the article: the notification data structure is a train wreck
- you can't approve friend requests
- some parts of the API are not documented at all, like the 'attachment' field on the posts table: it contains a dictionary, but in order to understand the keys and values you have to read the raw data from lots of posts, and even then you never know if one future entry breaks your app.
- and much more
One problem I had was that users expected much of this to work because it works on the Facebook iPhone app too. But that one is official and does not need to use the limited app API.
Great list of additional restrictions/shortcomings. I forgot to mention that you can't post a photo that is hosted on the Facebook CDN - that is really annoying.
Great stuff, I'll keep this in mind as I build out the Appleseed Graph API. Especially good that you opened up with what you like about the Graph API, often-times articles like this skip right to the criticisms.
I recently spent some time integrating with the Facebook Credits API, and even though it's smaller, it's a similarly frustrating experience.
The documentation is sort of there, sometimes contradictory, and I always have to resort to reading their code examples in PHP to really figure out what data is expected, how to format it, and what I get back. None of their APIs are really well thought-out, they're all slapped together to sort of have all the relevant data. I understand that it's better to ship than not, but a little thought. Please?
The greatest irony though is that if you want to make a post asking for help in the official developer forum, you're met by this page:
http://forum.developers.facebook.net/register.php?agree=Agre...
The documentation is sort of there, sometimes contradictory, and I always have to resort to reading their code examples in PHP to really figure out what data is expected, how to format it, and what I get back. None of their APIs are really well thought-out, they're all slapped together to sort of have all the relevant data. I understand that it's better to ship than not, but a little thought. Please?
The greatest irony though is that if you want to make a post asking for help in the official developer forum, you're met by this page:
http://forum.developers.facebook.net/register.php?agree=Agre...
I won't go into the specifics of the article, because there are people far more qualified than I - I did enjoy it.
There's a lesson here in marketing: can you predict which name will get more buzz, 'Breakup Notifier' or 'Friend Mail'?
There's another lesson in marketing - you sell the sizzle, not the steak, but if the steak is awful you won't sell very many for very long regardless of how sizz-tastic your marketing is.
I can't compare both these products - and I hope they're both solid.
There's a lesson here in marketing: can you predict which name will get more buzz, 'Breakup Notifier' or 'Friend Mail'?
There's another lesson in marketing - you sell the sizzle, not the steak, but if the steak is awful you won't sell very many for very long regardless of how sizz-tastic your marketing is.
I can't compare both these products - and I hope they're both solid.
Good read. And a good reminder that you shouldn't (or can't) rely on documentation or a single request here and there when you decide to work with an API.
And that the amount of support and 'release cycles' (frequency of updates and fixes) is something that you should always consider when integrating another piece of software into your own system.
And that the amount of support and 'release cycles' (frequency of updates and fixes) is something that you should always consider when integrating another piece of software into your own system.
Anil, why didn't you just ignore fields and attributes that were missing? So if education:concentration returns "CompSci" on the first call, but then is missing on the second call, don't treat that as a change. Don't consider the attribute changed until the it is present and different.
Right, I also thought I could work around the issues by ignoring fluctuations in the data. The problem is knowing what is the correct baseline data. In your example, the education:concentration might be missing when you first start checking and only start appearing a long time later. Do you ignore the data when it first appears or report that the person just added another major at college (which may be true!)? This is a bigger problem with fields that people might completely add/remove instead of simply updating (e.g. relationship status). I tried baking in a time period (e.g. 24 hours) in which I ignored all changes and waited for the baseline data to become obvious, but even that didn't work because some data took even longer to re-appear. At some point I had to admit to myself that - with all these work-arounds - my simple feature had turned into a hack.
I understand that having to work around fluctuating data might produce less than desirable results. But your users might find the feature useful anyway.
Thanks. I'm definitely keeping the code around since it's fully implemented. I might either reduce the scope of the alerts (for example, ignore add/remove and focus on changes only) or just bring it back when I have more confidence in the API.
So what if it is missing on the first call? How many calls until you get the complete picture? And which of those results is 'the truth' and reflects the current status?
Also, some things like relationship status might go from 'dating' to blank after a breakup if the person doesn't want to scream that they're single right away
I can understand all the complains about the facebook api, and they're all very valid, but my use of the facebook javascript API's have been relatively easy, to be honest. The documentation is far from ideal, and you have to figure stuff out by reading the actual response codes instead of trusting the documentation, but in general it all seems to work.
I am just wall posting and uploading photos though, and not doing any real data retrieval. Maybe facebook is more focussed on API's that put stuff in instead of taking things out? ;)
I am just wall posting and uploading photos though, and not doing any real data retrieval. Maybe facebook is more focussed on API's that put stuff in instead of taking things out? ;)
Having built a Facebook application that uses FB video I can understand where he's coming from. The Graph API is a great concept but like so many things Facebook, only halfway executed. It always amazed me that there are so many differences between a video in FQL and a video in the Graph API. In the end I figured out that "begin ... rescue ... end" was my friend when dealing with the Facebook apis. And about any call I make to the OpenGraph API is now wrapped in them.
Nice post.
Remember ChromeSpeed? Fancy seeing you here!
http://www.cs.columbia.edu/~neeraj/projects/chromespeed/
Remember ChromeSpeed? Fancy seeing you here!
http://www.cs.columbia.edu/~neeraj/projects/chromespeed/
Ha - ChromeSpeed! Glad to see you have the project page hosted and the video on YouTube. Maybe we should port to iPhone? Definitely an Angry Birds killer... OK, sorry to go off topic, we should catch up offline (or online elsewhere)!
How do you deal with spoofed emails?
The API had the exact same problems then... unanswered questions, confusing documentation, complex code examples. Something that seemed so simple--post a link to someone's wall, with their permission--turned into a multi-day codefest. But I finally got it working!
Until one day a few months later, when the API changed without warning. And that's what I think the biggest problem with the FB API is... they (at least back in the day) change it constantly, without much warning, and don't update documentation in a sane way. It's impossible to build once and forget.
I finally got a complaint from a user one day that the FB integration wasn't working. When I went in to check the code, I found that FB had almost completely changed their JS libraries and made significant API changes. Instead of going through that hell again, I just removed the feature.