Sammy.js, CouchDB, and the new web architecture(quirkey.com)
quirkey.com
Sammy.js, CouchDB, and the new web architecture
http://www.quirkey.com/blog/2009/09/15/sammy-js-couchdb-and-the-new-web-architecture/
40 comments
The reasons to stick with DHTML/JS as a "thick client" development platform are simple: decent, free development tools, the ability to interpose proxies and application middleware between the client and server transparently, and a very, very large install base for the client runtime.
That being said, I share much of your amusement at/disdain for the attitude of "OMG, look at this brand new thing I have made that is completely different from all those old things!"
That being said, I share much of your amusement at/disdain for the attitude of "OMG, look at this brand new thing I have made that is completely different from all those old things!"
I'm not sure about these reasons.
decent, free development tools
Well, you can have these for any mature platform. Be it QT, Flex or even Java. I'm not sure what's more decent about the HTML toolchain. The often cited FireBug is, to me, merely an emergency bandaid that makes development at least possible but still far from enjoyable.
the ability to interpose proxies and application middleware between the client and server transparently
There's nothing stopping any fat-client to speak HTTP. You don't need to a browser for that.
and a very, very large install base for the client runtime.
I'd say installation base is a red herring when talking about cross-platform apps but I'll give you that "instant-on" is an argument. For that I'd point to Air, which can run hybrid in and outside the browser.
I'm not arguing that HTML doesn't have it's place, mind you. But the extent to which it is abused nowadays is absurd.
decent, free development tools
Well, you can have these for any mature platform. Be it QT, Flex or even Java. I'm not sure what's more decent about the HTML toolchain. The often cited FireBug is, to me, merely an emergency bandaid that makes development at least possible but still far from enjoyable.
the ability to interpose proxies and application middleware between the client and server transparently
There's nothing stopping any fat-client to speak HTTP. You don't need to a browser for that.
and a very, very large install base for the client runtime.
I'd say installation base is a red herring when talking about cross-platform apps but I'll give you that "instant-on" is an argument. For that I'd point to Air, which can run hybrid in and outside the browser.
I'm not arguing that HTML doesn't have it's place, mind you. But the extent to which it is abused nowadays is absurd.
> and a very, very large install base for the client runtime.
You are underetimating this. For evrything else, (air, java, Qt, C#) I have to start a new app. My firefox is always on.
You are underetimating this. For evrything else, (air, java, Qt, C#) I have to start a new app. My firefox is always on.
In most cases, we are just swapping some startup time (for java, qt, etc.) for download time in the heavy web 2.0 apps.
Tools for developing Qt and Java applications are also free and are much much better than anything that has been created for DHTML/JS so far.
Java too has a very large install base. It is a shame that SUN didn't do much to promote it but you can still take advantage of the fact that many many people have the JVM installed.
Java too has a very large install base. It is a shame that SUN didn't do much to promote it but you can still take advantage of the fact that many many people have the JVM installed.
I hear all the negatives of why it wouldn't work, and everything that's wrong with it. And it's certainly healthy to be skeptical. But I don't hear anyone talking about what its possibilities are.
While it might be the new old thing, it's happening in the context of a different tech env, different market. The limitations of the old old thing might be different for the new old thing.
While it might be the new old thing, it's happening in the context of a different tech env, different market. The limitations of the old old thing might be different for the new old thing.
Modern standards compliant browsers are usually cross-platform so HTML is the new GUI toolkit, scripted by JS. Especially with WebGL and O3D in the works we will see the browser become the delivery mechanism for the new-old-school fat client. The difference between the new and old is network awareness. New style apps are often mashups.
Programs are temporarily installed for a session by visiting a specific URL (Chrome's app shortcut even makes it seem like a real installation). You update to new versions by refreshing the page.
Old school, new school, like it or not, this is the foreseeable future.
Programs are temporarily installed for a session by visiting a specific URL (Chrome's app shortcut even makes it seem like a real installation). You update to new versions by refreshing the page.
Old school, new school, like it or not, this is the foreseeable future.
Especially with WebGL and O3D in the works we will see the browser become the delivery mechanism for the new-old-school fat client.
I'm sorry but I fail to see what 3d technology in the browser has to do with application UIs. The two fundamental problems with HTML are the complete failure that is the box model, and the W3Cs complete ignorance of real world requirements.
As an example for the latter just look at the pityful HTML widget-set that hasn't changed one bit over the last 15 years. Every OS GUI toolkit in existence has native widgets for comboboxes, date pickers, sliders and various other amazingly useful primitives. I don't see any plans for finally bringing any of that to HTML. Instead we'll continue to see every new javascript framework reinvent them to varying degrees of success, along with a long list of other baseline functionality (client-side form validation, anyone? WebSocket really soon now?).
All the while the W3C invents fancy audio/video tags, 3d widgets, a questionable canvas API and lots of other stuff that hardly solves any real problem we developers are fighting every day, in any app.
I don't think the evolution of RIAs will wait for the W3C to get their act together. I rather think the alternatives to the browser will see a raise in popularity in the foreseeable future, as the HTML/JS combo gets maxed out and more people start looking for ways to notch the expirience up further than possible within the browser constraints...
I'm sorry but I fail to see what 3d technology in the browser has to do with application UIs. The two fundamental problems with HTML are the complete failure that is the box model, and the W3Cs complete ignorance of real world requirements.
As an example for the latter just look at the pityful HTML widget-set that hasn't changed one bit over the last 15 years. Every OS GUI toolkit in existence has native widgets for comboboxes, date pickers, sliders and various other amazingly useful primitives. I don't see any plans for finally bringing any of that to HTML. Instead we'll continue to see every new javascript framework reinvent them to varying degrees of success, along with a long list of other baseline functionality (client-side form validation, anyone? WebSocket really soon now?).
All the while the W3C invents fancy audio/video tags, 3d widgets, a questionable canvas API and lots of other stuff that hardly solves any real problem we developers are fighting every day, in any app.
I don't think the evolution of RIAs will wait for the W3C to get their act together. I rather think the alternatives to the browser will see a raise in popularity in the foreseeable future, as the HTML/JS combo gets maxed out and more people start looking for ways to notch the expirience up further than possible within the browser constraints...
Because the HTML/JS platform is only getting nicer. Almost everyone is already dumping fat clients for web clients, even in the enterprise, and it started way before AJAX was even heard of.
The advantages of a fat-client client/server app are low latency of the UI, maybe development speed given the right toolset, plus definitely flexibility of the UI for graphics and complicated interactions. Disadvantages though are difficulty of deployment, client platform restrictions, distributing updates/patches.
Whereas the traditional web thin client is easy deployment to your users across different platforms, via your single installation on the server. The disadvantage of traditional web clients is that you're restricted in UI complexity and latency is high. But this hasn't stopped so many enterprise and web apps being written (and rewritten) for the web, just to get deployment/update simplicity in place of a fat client.
So if you can build a complex UI fat client with low latency and the deployment ease of a web app, you're having your cake and eating it too. I think it really could be different, this time.
The advantages of a fat-client client/server app are low latency of the UI, maybe development speed given the right toolset, plus definitely flexibility of the UI for graphics and complicated interactions. Disadvantages though are difficulty of deployment, client platform restrictions, distributing updates/patches.
Whereas the traditional web thin client is easy deployment to your users across different platforms, via your single installation on the server. The disadvantage of traditional web clients is that you're restricted in UI complexity and latency is high. But this hasn't stopped so many enterprise and web apps being written (and rewritten) for the web, just to get deployment/update simplicity in place of a fat client.
So if you can build a complex UI fat client with low latency and the deployment ease of a web app, you're having your cake and eating it too. I think it really could be different, this time.
Well in this case the client side isn't the interesting part. The interesting part is the compaction of the server part into the database. The one language to rue them all is also interesting to a point but I assume you can do that with a Java app too.
I can see productivity gains in that more can be done without messing with a server side web framework, so there could be less maintenance.
There is an important aspect that not many "fat clients" have solved except the iPhone (except for the fact many iPhone apps are pirated and end up torrented): a reliable way to charge users... 50% of Photoshop installs are pirated; I wonder how many users of Salesforce are using pirated copies?
My thoughts exactly.
When all you have is a hammer ...
jQuery is not a good architecture for a full client application. jQuery is meant for flipping some dom attributes here and there. It does a poor job of modeling the UI state. Sinatra is also a poor architecture choice for the client since the client has persistent state.
We already have a well designed client architecture. It's called Cocoa and its implemented in the browser as Cappuccino.
jQuery is not a good architecture for a full client application. jQuery is meant for flipping some dom attributes here and there. It does a poor job of modeling the UI state. Sinatra is also a poor architecture choice for the client since the client has persistent state.
We already have a well designed client architecture. It's called Cocoa and its implemented in the browser as Cappuccino.
Interesting point. I tend to agree with you. I find myself hesitant about putting my business logic essentially on the front-end. With the current model we can hide some of the "secret sauce" that makes our applications unique on the server end. With something like Cappuccino (or Sammy.js, etc), how do we keep our secret sauce secret?
> I find myself hesitant about putting
> my business logic essentially on the front-end
I think that's the core argument, if you omit the "CouchDB as an app server" part (which is orthogonal).
This was exactly the approach I took with an app about 4 - 5 years ago. Back then XML not JSON was the data transport format, but I kept it lightweight. The UI was built in the browser using XSLT to generate HTML. Additional transforms tied to UI events allowed filtering, sorting, etc. all on the client. IE, for all its other flaws, actually supported this much better than any other browser at that time and we were developing in an IE-only environment.
Business logic was on the server in stored procedures, which were exposed in a simple web service api so they could be called directly by the browser script logic. This also made it very testable because the stored procedures could be invoked by testing scripts that did not depend on the presence of any middle tier or UI at all. It also would have been a simple matter to build other client front-ends, as long a platform could work with XML and HTTP it could in theory be a client.
The XSLT to render the UI from XML could get a bit verbose but despite that it turned out to be a very productive way to work.
The downsides at the time were that IE was slow to render when you injected the HTML output of the transforms into the body.innerHTML property; this became noticeable if the generated page was large, but on the plus side this was a constraint against developing overly-complex pages.
Business logic was on the server in stored procedures, which were exposed in a simple web service api so they could be called directly by the browser script logic. This also made it very testable because the stored procedures could be invoked by testing scripts that did not depend on the presence of any middle tier or UI at all. It also would have been a simple matter to build other client front-ends, as long a platform could work with XML and HTTP it could in theory be a client.
The XSLT to render the UI from XML could get a bit verbose but despite that it turned out to be a very productive way to work.
The downsides at the time were that IE was slow to render when you injected the HTML output of the transforms into the body.innerHTML property; this became noticeable if the generated page was large, but on the plus side this was a constraint against developing overly-complex pages.
I find the grafting of cocoa onto javascript a bit stomach turning. Its like watching a man read the news to you on TV. Different mediums require different methods. I don't think anyone has really gotten this right yet. I agree jQuery isn't the whole solution, I don't think Sammy.js is the whole thing either. But I do think there will be something built on top of this stack. I have started to try and create something like this http://github.com/voidfiles/BHive .
You didn't explain why the Cocoa model doesn't work on top of the DOM.
With the power of CouchApp and _attachments, the javascript and HTML files needed to serve the application are served directly out of the database.
That statement kind of troubles me. It shows that you're really just offloading all of the work from a normal app server to the data storage layer. From a "Mom look what I can do!" perspective this is cool, but, again, you're not eliminating your app server. Couchdb allows this because it uses HTTP as it's communication protocol, but one could do the same with MySQL, via an HTTP communication frontend, which sounds a lot like what Rails, Django, etc effectively are with some optimizations to make other non-data-storage operations faster.
That statement kind of troubles me. It shows that you're really just offloading all of the work from a normal app server to the data storage layer. From a "Mom look what I can do!" perspective this is cool, but, again, you're not eliminating your app server. Couchdb allows this because it uses HTTP as it's communication protocol, but one could do the same with MySQL, via an HTTP communication frontend, which sounds a lot like what Rails, Django, etc effectively are with some optimizations to make other non-data-storage operations faster.
Good, except that you still have to deal with authentication, authorization, and other security aspects of a server-side component. Server-side code won't go away no matter how fat the client gets.
In this case it will just move into the database. You'll have to start writing procedures and rules in the database instead of in your web framework and only expose those instead having only trusted code interact with the DB.
You'll have to start writing procedures and rules in the database instead of in your web framework
This will not fool anyone. You can't fold the business logic into the database and then pretend that you've eliminated the business-logic layer on the server. It's still there. You've just driven it into hiding.
This will not fool anyone. You can't fold the business logic into the database and then pretend that you've eliminated the business-logic layer on the server. It's still there. You've just driven it into hiding.
Exactly. I don't think there is an equivalence between server code and client code. There are some things you just can't trust the client to do, so there will always have to be server code to do it. Whether that code is in a web server, a framework, a database, it doesn't matter. Someone has to write it.
However, I could see some of the code, such as the code that actually builds the page the users sees, being pushed into the browser. But then, don't we already have that with XML and XSLT (or whatever?)
However, I could see some of the code, such as the code that actually builds the page the users sees, being pushed into the browser. But then, don't we already have that with XML and XSLT (or whatever?)
couchDB folks are trying to create auth in client, they haven't succeeded yet, the crypto algorithms are a bit taxing on the browser, but they are trying.
Honest question: How would one go about implementing authentication/security into such a setup? Maybe it could work for 1-writer, many reader apps (blogs) but how do you handle user signups, accounts, and security with this?
I think CouchDB might have built-in authentication coming soon, but I'm not sure about that.
There's always the option of using HTTP authentication.
There's always the option of using HTTP authentication.
Is that really good enough, though? For example, suppose that I write a blogging platform, and I want to ensure that a user can only query their own blog entries. How would I do that if the query is coming from the client (which always has to be untrusted) directly to the database? What prevents someone from rewriting the js client side so that it queries blog posts from other users? Or to prevent it from just sucking back all the blog posts in the DB and essentially DOSing the whole server?
Doing per-user security directly in the DB with simple CRUD permissions per table would be enough of a headache, but most applications eventually require finer-grained security than that, and for performance reasons you also don't want a client to be able to execute just any arbitrary query.
It seems like this is pretty close to same trap a lot of people fall into of only enforcing data validation client-side, or only enforcing things like view permissions client-side (by not rendering links), which leaves all sorts of holes open.
Doing per-user security directly in the DB with simple CRUD permissions per table would be enough of a headache, but most applications eventually require finer-grained security than that, and for performance reasons you also don't want a client to be able to execute just any arbitrary query.
It seems like this is pretty close to same trap a lot of people fall into of only enforcing data validation client-side, or only enforcing things like view permissions client-side (by not rendering links), which leaves all sorts of holes open.
Well, all queries are accessed through URLs. There are two types of queries: temp views and permanent views. You can use apache/nginx to redirect all query through URLs for temporary views. Permanent views are stored on the db. That way, your client can't execute arbitrary queries. Then, the client can only be limited to views/queries that the web dev set.
http://wiki.apache.org/couchdb/Nginx_As_a_Reverse_Proxy
Then as long as the client can keep its identification token secret from others, you can implement "user can only query their own blog entry"
I suggest you look into couchDB a bit further. CouchDB is a different beast altogether from relational databases. Therefore, the assumptions you make about what a database can and cannot do coming from a relational world doesn't always apply.
http://wiki.apache.org/couchdb/Nginx_As_a_Reverse_Proxy
Then as long as the client can keep its identification token secret from others, you can implement "user can only query their own blog entry"
I suggest you look into couchDB a bit further. CouchDB is a different beast altogether from relational databases. Therefore, the assumptions you make about what a database can and cannot do coming from a relational world doesn't always apply.
I kept hoping the next paragraph would address this, but it didn't.
It's cool and all, but it sounds like you're still going to basically need a server stack there to enforce permissions and some logic and consistency things. For example, say users have some private data. What stops me from modifying the javascript to insert a record that points to someone else's email address in my record? Unless the database can see this and prevent it I might then be looking at or even able to change someone else's email on their account. It can get very hairy if you don't have full control over the database interaction because you have to start guarding against arbitrary database operations.
It's cool and all, but it sounds like you're still going to basically need a server stack there to enforce permissions and some logic and consistency things. For example, say users have some private data. What stops me from modifying the javascript to insert a record that points to someone else's email address in my record? Unless the database can see this and prevent it I might then be looking at or even able to change someone else's email on their account. It can get very hairy if you don't have full control over the database interaction because you have to start guarding against arbitrary database operations.
[deleted]
Check out jchris' toast, which exposes CouchDB directly to the world, and runs a Comet realtime chat/post app.
Example of how he does authentication: http://github.com/jchris/toast/blob/master/validate_doc_upda...
http auth + document update validation function
It's unintuitive, but this setup actually supports a huge number of connections and runs really fast. Go figure.
Example of how he does authentication: http://github.com/jchris/toast/blob/master/validate_doc_upda...
http auth + document update validation function
It's unintuitive, but this setup actually supports a huge number of connections and runs really fast. Go figure.
[deleted]
This might work for the simplest of sites like some have pointed out (e.g. a blog) but what I can't accept is having application logic on the frontend. Beyond authentication (let's say OAuth et all solve that issue down the road), one of the benefits of having serverside code is that when it comes down to it, you as a developer don't care if the user messes with the frontend (exposed) business logic (usually form validation of some type) because you're always checking it again / cleansing it on the serverside. That's the tip of the iceberg...ranking algorithms, limits on certain activities (rules in general).
Now, that serverside code can be javascript serverd on something akin to 10gen if javascript is their tool of choice, but exposing any of those to the user so that they may edit them without any secondary sanity check is unacceptable.
Now, that serverside code can be javascript serverd on something akin to 10gen if javascript is their tool of choice, but exposing any of those to the user so that they may edit them without any secondary sanity check is unacceptable.
I like this idea quite a bit. No application server means less maintenance and less administrative overhead.
Of course, it also means that the newly-empowered database has to scale out just as easily as an application server. For starters, request authentication becomes the responsibility of the database, and that could mean heavy volume.
Of course, it also means that the newly-empowered database has to scale out just as easily as an application server. For starters, request authentication becomes the responsibility of the database, and that could mean heavy volume.
learning JavaScript & prototype i wrote a Twitter clone around this idea. I put up a number of textfiles in different user directories, and turned apaches directory listings on, and voilà a very basic restful interface. I also added post somehow, maybe a php script.
The client-side script could read who to follow from the server and by itself performed the task of assembling the timeline which was the CPU and IO extensive task that twitter had problems scaling with. Not sure the extra bandwidth cost of performing this client side would be worth it, but there's almost no logic at all server side so there's not much scaling of processor power at least. Programming with prototype was fun, but then I found jQuery...
The client-side script could read who to follow from the server and by itself performed the task of assembling the timeline which was the CPU and IO extensive task that twitter had problems scaling with. Not sure the extra bandwidth cost of performing this client side would be worth it, but there's almost no logic at all server side so there's not much scaling of processor power at least. Programming with prototype was fun, but then I found jQuery...
How on earth are these guys going to prevent a user from reading all the credit cards from that couch-db via its json interface?
Is anyone actually using this architecture?
Stop the madness. Just stop it.
So, here we have the return of the fat client, episode 20.
And as with all the other episodes the standard question remains: If you're going fat-client then why on earth stick to a platform as horrible as HTML/JS?