Protocols over APIs(blog.superfeedr.com)
blog.superfeedr.com
Protocols over APIs
http://blog.superfeedr.com/protocols-over-api/
4 comments
Using duplicity is almost as simple as using rsync, and it gives you incremental backups as well. And encryption if you need it. And can back up over SSH like Rsync, or to a local path, or even to Amazon S3.
[edit] I wrote up how I do this, with examples and cron entries a few months ago: https://grepular.com/Secure_Free_Incremental_and_Instant_Bac...
[edit] I wrote up how I do this, with examples and cron entries a few months ago: https://grepular.com/Secure_Free_Incremental_and_Instant_Bac...
They don't want me switching away, but they should also recognize that programmers like me are lazy/smart enough that I'm much more likely to switch to their service if I know I can easily switch away.
(But being lazy, I'm not going to switch away from them on a whim. Even if the interface is the same, there's still a cost to the testing and such that would be needed. I mostly want to know that there would be a reasonable migration path.)
Joel Spolsky made this point about Excel once ("Strategy Letter III: Let Me Go Back!"): its adoption really took off once people could write 123 files.
(But being lazy, I'm not going to switch away from them on a whim. Even if the interface is the same, there's still a cost to the testing and such that would be needed. I mostly want to know that there would be a reasonable migration path.)
Joel Spolsky made this point about Excel once ("Strategy Letter III: Let Me Go Back!"): its adoption really took off once people could write 123 files.
That's an interesting point but I think it works more in favor of small providers trying to gain market share rather than incumbents.
Amazon's strategy with AWS seems to be to hype it so much that anybody who isn't using it feels like they are some sort of neanderthal since everyone talks about how everything is moving "to the cloud".
Interesting that he uses Excel as an example since I imagine (I haven't used Excel for a while) that saving data in a modern version of Excel makes it incompatible with anything else (at least it would do if OpenOffice didn't reverse engineer the format).
Amazon's strategy with AWS seems to be to hype it so much that anybody who isn't using it feels like they are some sort of neanderthal since everyone talks about how everything is moving "to the cloud".
Interesting that he uses Excel as an example since I imagine (I haven't used Excel for a while) that saving data in a modern version of Excel makes it incompatible with anything else (at least it would do if OpenOffice didn't reverse engineer the format).
The format didn't need to be reversed, it's (almost) open. Additionally there are plugins for the older versions of Office to let them work with the newer format, and it's much much easier to write tools to do things with Word files now - just crack one open and take a look (it's a .zip file full of XML files).
Iris Couch's first blog post was about how to leave them for other providers http://www.iriscouch.com/blog/2011/05/how-to-bail-out-on-iri...
Well done.
It would also be nice not to have to use large install scripting languages (Perl, Python, Ruby), Java or Go, in order to communicate with AWS, e.g., EC2.
Let us be thankful for XML and the popular idea of "it's already done for you, just install this bloated library and read these 320 pages of documentation".
If API's mean you can't debug something without having to install Java or some huge, complex scripting language, I'm not sure they've simplifed anything. They've just added another layer of abstraction. More work should anything ever go wrong. And eventually, something always does go wrong.
Maybe S3 should just be rysnc servers running in Xen. Just cut to the chase.
It would also be nice not to have to use large install scripting languages (Perl, Python, Ruby), Java or Go, in order to communicate with AWS, e.g., EC2.
Let us be thankful for XML and the popular idea of "it's already done for you, just install this bloated library and read these 320 pages of documentation".
If API's mean you can't debug something without having to install Java or some huge, complex scripting language, I'm not sure they've simplifed anything. They've just added another layer of abstraction. More work should anything ever go wrong. And eventually, something always does go wrong.
Maybe S3 should just be rysnc servers running in Xen. Just cut to the chase.
Doesn't Google Storage for developers also have the same protocol as AWS S3?
And I heard there are ope-source S3 clones as well, e.g. http://www.pdl.cmu.edu/pWalrus/index.shtml#hdfs
And I heard there are ope-source S3 clones as well, e.g. http://www.pdl.cmu.edu/pWalrus/index.shtml#hdfs
Do you have an example of a stateful protocol you prefer over REST?
Depends on what you are doing, there are plenty of times where REST is fine.
What people seem to forget however is that there are other mature protocols on the internet besides HTTP.
Let's say for example you want to send some information between two servers. You decide to do this with REST, so you make an HTTP request. What happens if the REST server goes down?
You either want to failover to a different host or you want to it to wait and retry later. You could use some custom async queuing system like RabbitMQ etc but another thing to bear in mind is that postfix supports both of these things. You can simply send an email with your message to a target mailbox that the other server has access too and use multiple MX records for the failover.
What people seem to forget however is that there are other mature protocols on the internet besides HTTP.
Let's say for example you want to send some information between two servers. You decide to do this with REST, so you make an HTTP request. What happens if the REST server goes down?
You either want to failover to a different host or you want to it to wait and retry later. You could use some custom async queuing system like RabbitMQ etc but another thing to bear in mind is that postfix supports both of these things. You can simply send an email with your message to a target mailbox that the other server has access too and use multiple MX records for the failover.
[deleted]
It's not about the Protocols over APIs. It's not even about open-web over walled-gardens.
Take a quick look back: Google Wave wasn't a safe bet, even though it was a federated protocol. On the other hand Facebook's Open Graph is a safe bet, even though it is hardly a protocol, but rather FB's API.
If you'd like to be sure something doesn't get deprecated, make sure it is widely adapted before you use it. In this case you would be a part of community, whose opinion has a leverage over technology providers.
But this way you may miss an early opportunity. It's all about choices. Luckily, we have plenty.
Take a quick look back: Google Wave wasn't a safe bet, even though it was a federated protocol. On the other hand Facebook's Open Graph is a safe bet, even though it is hardly a protocol, but rather FB's API.
If you'd like to be sure something doesn't get deprecated, make sure it is widely adapted before you use it. In this case you would be a part of community, whose opinion has a leverage over technology providers.
But this way you may miss an early opportunity. It's all about choices. Luckily, we have plenty.
I want to assume that if we had versioning built into more of our common languages and runtimes (some do), than this would not be an issue.
Also, I hate to nitpick, but protocols usually describe stuff like how to drink tea formally in a garden with a king or queen (workflow), not API's (implementation). I think what they mean is URL naming scheme, but I could be wrong.
Also, I hate to nitpick, but protocols usually describe stuff like how to drink tea formally in a garden with a king or queen (workflow), not API's (implementation). I think what they mean is URL naming scheme, but I could be wrong.
What if the two services have different features, even slightly? The protocols are instantly incompatible. The problem is that the feature set, and thus the protocol, is strongly tied to the provider, and decoupling the two doesn't make much sense.
Well, I'd argue that the protocol hasn't been designed correctly. Don't forget that every single website out there use HTTP as their main protocol, yet, they have very different feature sets, capabilities and even goals!
We're reinventing the wheel, yet again.
Wasn't this what CORBA, XML-RPC, or SOAP was meant to solve? And then we all happily ditched them for loosely defined RESTy JSON because it was so goddamn easier.
I think TFA is describing a legitimate complaint; but I don't think the RESTy CRUD twitter exposes is all that difficult to abstract.
Amazon on the other hand…
Wasn't this what CORBA, XML-RPC, or SOAP was meant to solve? And then we all happily ditched them for loosely defined RESTy JSON because it was so goddamn easier.
I think TFA is describing a legitimate complaint; but I don't think the RESTy CRUD twitter exposes is all that difficult to abstract.
Amazon on the other hand…
Amazon's protocols were clearly designed by a Java EE developer. Not that there isn't anything inherently wrong with them, and they are very flexible. They're also somewhat stuck with them based around the huge amount of code which interfaces with S3/AWS.
There are some problems with no good solution, namely the signed parameters in Amazon's case. How do you sign JSON? Is there a "canonical" JSON format that will allow signing consistently, and verification? All the approaches I've seen are incompatible in some way.
Client-side SSL is another solution, but its much less flexible (i.e., token-generated requests wouldn't be easy)
There are some problems with no good solution, namely the signed parameters in Amazon's case. How do you sign JSON? Is there a "canonical" JSON format that will allow signing consistently, and verification? All the approaches I've seen are incompatible in some way.
Client-side SSL is another solution, but its much less flexible (i.e., token-generated requests wouldn't be easy)
Stuck with Java. Exactly.
They want you to write your app so that all of your code is so hopelessly coupled to their service and API that it may as well just be an extension of it.
I had a similar revelation when I was writing an S3 backup system recently, I was writing hundreds of lines of Java that was catching exceptions that I hadn't even heard of while calling libraries in the namespace of a company that is known for selling books. What on earth was I doing?
So I wrote a 3 line bash script to rsync to a bunch of places and added it to crontab. Now that means I can change backup providers whenever I want with a single code change.
As a separate point there is something to be said for stateful protocol (especially open ones) instead of stateless by design API stuff like REST.
I am considering the use of an IRC server as a backend for a messaging/logging system.
I like building systems that can be debugged with telnet.