Opa: A new language aiming to make web programming transparent(lambda-the-ultimate.org)
lambda-the-ultimate.org
Opa: A new language aiming to make web programming transparent
http://lambda-the-ultimate.org/node/4336
14 comments
I've never seen the model work before where you "don't even need to think about what's running on the client or server." I like features like sharing code (same language) and easy transport (serializable data structures), but frameworks that blur the client/server distinction usually download all of your code, or none of it, or little snippets on demand, or some other strategy that's worse than a manual distinction.
There are two things here. For prototyping, Opa will automatically decide what goes to the client and what goes to the server. We are very much aware that this may not yield the best performance. So, when you move from prototyping to optimization, using your knowledge of the application security policy, you can add security hints that the compiler will use to optimize client-server placement, knowing that this does not break the policy. And finally, if you want even better control, you can annotate any piece of code to instruct the compiler to place it on the client, or on the server, or both. In any case, the compiler will inform you if the set of constraints you put has no solution.
Tons of dependencies (a personal gripe). Coming from the Java world, where lots of things like reading images are built-in, it's comical to see that Opa requires Java, AND libjpeg and a dozen other things (it doesn't actually run on the JVM I don't think).
I concur that the dependencies are a (minor) annoyance. Just one point: Java is used only to build Opa from source. More precisely, during Opa compilation, we run the (very nice) Java-based Google Closure Compiler as a measure of the quality of our JavaScript libraries. If I recall correctly, that build step is optional, too.
There are two things here. For prototyping, Opa will automatically decide what goes to the client and what goes to the server. We are very much aware that this may not yield the best performance. So, when you move from prototyping to optimization, using your knowledge of the application security policy, you can add security hints that the compiler will use to optimize client-server placement, knowing that this does not break the policy. And finally, if you want even better control, you can annotate any piece of code to instruct the compiler to place it on the client, or on the server, or both. In any case, the compiler will inform you if the set of constraints you put has no solution.
Tons of dependencies (a personal gripe). Coming from the Java world, where lots of things like reading images are built-in, it's comical to see that Opa requires Java, AND libjpeg and a dozen other things (it doesn't actually run on the JVM I don't think).
I concur that the dependencies are a (minor) annoyance. Just one point: Java is used only to build Opa from source. More precisely, during Opa compilation, we run the (very nice) Java-based Google Closure Compiler as a measure of the quality of our JavaScript libraries. If I recall correctly, that build step is optional, too.
Partial answers/tidbits:
The JS code is not as optimized as what you can write manually, although a lot of work was put in making it work "fast enough" (thanks Mathieu). However, we have many ideas on how to reach native performance. Look for instance at how Java got sometimes even faster than C!
There are hints @server and @client to specify a side for running a bit of your code. Since, as you say, there are things you know and the machine doesn't.
Opa being a compiler, updated projects have to be recompiled. You gain a lot in quality, consistency with this (especially associated to the very advanced type system of Opa). And as we know that compilers are annoying, we made our best to make the compile time short. You probably won't have to take a coffee, sorry.
The chat example does not rely on predefined libraries -- that's the difference. You can also look at bigger chat applications like https://github.com/hbbio/OpaChat and compare with https://github.com/ry/node_chat.
Opa requires Java for building from source - if the package does, that's something you can report at https://github.com/mlstate/opalang/issues?sort=created&d...
The JS code is not as optimized as what you can write manually, although a lot of work was put in making it work "fast enough" (thanks Mathieu). However, we have many ideas on how to reach native performance. Look for instance at how Java got sometimes even faster than C!
There are hints @server and @client to specify a side for running a bit of your code. Since, as you say, there are things you know and the machine doesn't.
Opa being a compiler, updated projects have to be recompiled. You gain a lot in quality, consistency with this (especially associated to the very advanced type system of Opa). And as we know that compilers are annoying, we made our best to make the compile time short. You probably won't have to take a coffee, sorry.
The chat example does not rely on predefined libraries -- that's the difference. You can also look at bigger chat applications like https://github.com/hbbio/OpaChat and compare with https://github.com/ry/node_chat.
Opa requires Java for building from source - if the package does, that's something you can report at https://github.com/mlstate/opalang/issues?sort=created&d...
>Look for instance at how Java got sometimes even faster than C!
I don't think browsers are going to let you build a hot-spot compiler in Javascript. Nor do I think that an online, optimizing interpreter in javascript stands much of a chance of improving performance.
I don't think browsers are going to let you build a hot-spot compiler in Javascript. Nor do I think that an online, optimizing interpreter in javascript stands much of a chance of improving performance.
Certainly. However, in many cases, the high-level information that Opa has can be used to produce highly-efficient JavaScript code and/or highly-efficient distribution layout, that would be quite hard, time-consuming and error-prone to replicate by hand. On the JavaScript side, this is already the case e.g. for our implementation of pattern-matching or tail-recursive functions. We have numerous additional such ideas that we have not found time to implement.
And then, if NaCl/NativeClient becomes mainstream, that's yet another avenue worth exploring.
And then, if NaCl/NativeClient becomes mainstream, that's yet another avenue worth exploring.
There are ways this story fails in practice, like the "language mismatch" I mentioned.
For example, in Google Web Toolkit, which compiles Java to JavaScript, theoretically the static type information should be a huge boon to optimization. And indeed, the compiler does some fancy stuff, including static inlining, and segmenting the code so that chunk 2 is only downloaded when a method call crosses the boundary of chunk 1.
However, consider something like a HashMap. A JavaScript programmer might use a single JS object as a hash, whereas the GWT programmer has the luxury of using a java.util.HashMap, which is theoretically compiled down to something efficient, maybe just as efficient. In practice, this is hard or impossible for the compiler, and HashMaps are compiled to a system of objects that's much more complex. There are great reasons for this -- primarily, HashMaps can have any object as a key, not just a string; also the nature of the HashMap API and behavior under subclassing -- but the result is if you want something fast, you need to use a FastStringMap or some such class, backed by a "native" JS object. But in general, when you're not worrying about performance, you might find yourself using many HashMaps, even parsing JSON into nested HashMaps! So now each JS object is represented by a bunch of JS objects needed to make the mechanics of the HashMap abstraction work.
I worry that not having arrays leads to a similar problem. In magic happy land, a call like List.pop that removes the last element of a list would be implemented as a tail-recursive operation on heads and tails, which the compile would then recognize and turn back into a call to Array.pop! In practice, this kind of thing is unlikely. As with HashMaps, the programmer is forced (or told) to use heavyweight data structures to simulate lightweight ones, with the hope that the compiler will recognize that this is a case where the lightweight structure applies and transform it back!
As for "online" compilation, I guess this is easier on the JVM. Clojure, Rhino, Scala, and Java are all compiled and can be set up to recompile on the fly after every change to the program. Node.js too. Having developed this way, I don't think I could go back to having to do anything after changing the code besides reloading the page.
For example, in Google Web Toolkit, which compiles Java to JavaScript, theoretically the static type information should be a huge boon to optimization. And indeed, the compiler does some fancy stuff, including static inlining, and segmenting the code so that chunk 2 is only downloaded when a method call crosses the boundary of chunk 1.
However, consider something like a HashMap. A JavaScript programmer might use a single JS object as a hash, whereas the GWT programmer has the luxury of using a java.util.HashMap, which is theoretically compiled down to something efficient, maybe just as efficient. In practice, this is hard or impossible for the compiler, and HashMaps are compiled to a system of objects that's much more complex. There are great reasons for this -- primarily, HashMaps can have any object as a key, not just a string; also the nature of the HashMap API and behavior under subclassing -- but the result is if you want something fast, you need to use a FastStringMap or some such class, backed by a "native" JS object. But in general, when you're not worrying about performance, you might find yourself using many HashMaps, even parsing JSON into nested HashMaps! So now each JS object is represented by a bunch of JS objects needed to make the mechanics of the HashMap abstraction work.
I worry that not having arrays leads to a similar problem. In magic happy land, a call like List.pop that removes the last element of a list would be implemented as a tail-recursive operation on heads and tails, which the compile would then recognize and turn back into a call to Array.pop! In practice, this kind of thing is unlikely. As with HashMaps, the programmer is forced (or told) to use heavyweight data structures to simulate lightweight ones, with the hope that the compiler will recognize that this is a case where the lightweight structure applies and transform it back!
As for "online" compilation, I guess this is easier on the JVM. Clojure, Rhino, Scala, and Java are all compiled and can be set up to recompile on the fly after every change to the program. Node.js too. Having developed this way, I don't think I could go back to having to do anything after changing the code besides reloading the page.
> I worry that not having arrays leads to a similar problem.
Good point. We have actually been working on this specific problem of lists vs. arrays. The results have not been put on code yet, but we hope that the problem can be solved in many cases.
> Having developed this way, I don't think I could go back to having to do anything after changing the code besides reloading the page.
We have also been working on that. Future versions of Opa might offer something along these lines.
Good point. We have actually been working on this specific problem of lists vs. arrays. The results have not been put on code yet, but we hope that the problem can be solved in many cases.
> Having developed this way, I don't think I could go back to having to do anything after changing the code besides reloading the page.
We have also been working on that. Future versions of Opa might offer something along these lines.
Seems like an interesting stab at fixing the drudgery that web app development has become. Some questions, however:
1. When you go through the pains of integrating everything else, why not go for gold and exclude x/html and css as well? These are as "glue, antagonistic" as the other bits that make up the current webapp quagmire.
2. When ideas like PaaS are emerging, wouldnt it be better to bundle everything that makes up the app separate from everything that makes up the platform? I know that this sounds like the antithesis of your "one executable" idea - and I love single executables where appropriate - but it seems like webapp deployment could do with some economies of scale on the underlying container in the sense that the difference between App A and App B is primarily in the functionality, not in the way those functions are accessed, used, etc.
1. When you go through the pains of integrating everything else, why not go for gold and exclude x/html and css as well? These are as "glue, antagonistic" as the other bits that make up the current webapp quagmire.
2. When ideas like PaaS are emerging, wouldnt it be better to bundle everything that makes up the app separate from everything that makes up the platform? I know that this sounds like the antithesis of your "one executable" idea - and I love single executables where appropriate - but it seems like webapp deployment could do with some economies of scale on the underlying container in the sense that the difference between App A and App B is primarily in the functionality, not in the way those functions are accessed, used, etc.
> 1. When you go through the pains of integrating everything else, why not go for gold and exclude x/html and css as well? These are as "glue, antagonistic" as the other bits that make up the current webapp quagmire.
Actually, the x/html and (most of the) css that you see in our examples is syntactic sugar for higher-level constructions.
> 2. When ideas like PaaS are emerging, wouldnt it be better to bundle everything that makes up the app separate from everything that makes up the platform? [...]
Certainly feasible, but I'm not sure what would be gained, and it would make deployment a little harder.
Actually, the x/html and (most of the) css that you see in our examples is syntactic sugar for higher-level constructions.
> 2. When ideas like PaaS are emerging, wouldnt it be better to bundle everything that makes up the app separate from everything that makes up the platform? [...]
Certainly feasible, but I'm not sure what would be gained, and it would make deployment a little harder.
Opa translated from Dutch to English means Grandpa ...
Just in case anybody wonders: roughly half of the Opa team speaks German, at least one speaks Argentinian Spanish, at least one speaks Greek. So, yeah, we are aware of most of the puns.
Guess what? We like them :)
Guess what? We like them :)
Opa is Argentinian slang for "retarded" :-(
Opa translated from Portuguese to English means wow, or whoops ...
Also grandpa in german.
It apparently means Ulcer in "Lithuanian". http://translate.google.com/#lt|en|Opa
Also a greek exclamation common in songs, e.g. http://www.youtube.com/watch?v=bmeCC53FKes
It means "older brother" in Korean.
I thought this OPA thing was closed already.
I have quit java/j2ee to avoid recompiling/redeploying all the time ! So the compiling part is a no go for me !
I have quit java/j2ee to avoid recompiling/redeploying all the time ! So the compiling part is a no go for me !
really like the web page design (especially the 'secure' tab). was it made at MLstate?
Made internally: Congrats go to Ida.
I hope I'm not the only one that is now picturing a bunch of Greeks destroying plates...
[deleted]
The project itself looks great. Novel idea, excellent documentation, etc. But the title of the OP seems a bit off: how does adding an extra layer between the developer and the application code (server code, client-side js) create more transparency? It's kind of like saying Coffeescript makes Javascript more transparent.
Now functional programming languages are scary to any student who sees them for the first time, but I admit, this kind of thing really scared me. Took a CS Introduction to Functional Programming course at university, taught using Clean, a pure and lazy functional language developed at the university. Most of their research currently is related to iTasks (see http://wiki.clean.cs.ru.nl/ITasks) which also does web apps, but focused on workflows. As a web dev however, it totally stripped the layers between model and browser, which is actually the harder part most of the time where the customization happens, and became this (all too simple) mess of models and functions. It was too early stage software too, various server bugs in the built-in HTTP server, and lacking JS/frontend implemantation. Only one of the 15 weeks needed iTasks, which was a good thing, for students and TA sanity.
Interesting story, though it's tough to gather what your overall point is. You're saying that abstracting server and client logic into monolithic functional code gave you headaches?
Sorry, it was late. I think my point was that you can't reduce the "science" of a web app to pure functions...
Unfortunately it's AGPL licensed so it can't be used for commercial development (without obtaining a commercial license). It looks neat though, and it's great to see people rethinking the normal way of doing things on the web.
s/commercial/closed source/...
Also, I may be wrong, but I don't think that the code run by a GPLed interpreter has to conform to the license. Native libraries linked to the runtime would have to, though.
Also, I may be wrong, but I don't think that the code run by a GPLed interpreter has to conform to the license. Native libraries linked to the runtime would have to, though.
Caveat: Opa is not an interpreter, it's a native code compiler, with native libraries. Consequently, the previous poster is right: AGPL is fully contagious here.
Note: I'm part of the Opa team.
Note: I'm part of the Opa team.
I was going to give a quick glance to Opa, but the AGPL license fills me with confidence. Your team is confident that the language is awesome enough that the license won't be a problem even for proprietary/commercial users of it.
Good choice on the license!
(However if you encounter problems with adoption/funding, it might be time to start thinking about paid dual-licensing heh)
Good choice on the license!
(However if you encounter problems with adoption/funding, it might be time to start thinking about paid dual-licensing heh)
I wouldn't even bother to look at it without knowing that, if I mange to make something interesting, I have a way to control my code.
AGPL is a non-starter.
EDIT: There is a non-AGPL license that can be purchased. Non-starter has passed start. :)
AGPL is a non-starter.
EDIT: There is a non-AGPL license that can be purchased. Non-starter has passed start. :)
But no published price list. So, no sale here.
Well, if you guys ever rethink this, I'll be interested. Until then, I'll just keep plugging along with what I have.
Can you give insight into this strategy? It seems... limiting. For startups, at least.
Well, the general idea is to ensure that whoever is using Opa somehow contributes to the FOSS community, either by contributing source code or by funding the development of Opa itself.
How is that limiting for startups? Most startups are built on execution and network-effects and I don't see how the license is a hindrance to that.
Really? Source-code is not part of the execution?
A vast majority of start-ups, talented developers see afterwards and go "Oh, I could have coded that", but perhaps not built the business or had the drive to go after the users. They can see the site and imagine how they would reverse engineer it. The value is in the user-base and execution of the business. Only a few start-ups rely on what might be a non-trivial new technology that was invented in house, for which closed source is necessary.
At least superficially, this seems close to UR/Web. (http://www.impredicative.com/ur/). I assume you guys are aware of each other. Can you tell us something about the advantages of Opa?
Of course, we know each other.
Ur/Web for instance allows metaprogramming when Opa doesn't.
But Opa handles distribution natively when Ur/Web doesn't.
Some users say that both projects are of great quality, so I invite you to try both and make your own judgment. Some bits at http://www.reddit.com/r/programming/comments/icp4u/yet_anoth...
Thanks for the information!
Ur/Web syntax seems closer to OCaml syntax, so it might be a bit easier to learn. However, Opa documentation seems more polished and complete so it might even the odds.
Nice work, and thanks for bringing static guarantees to web development. It is much needed.
Ur/Web syntax seems closer to OCaml syntax, so it might be a bit easier to learn. However, Opa documentation seems more polished and complete so it might even the odds.
Nice work, and thanks for bringing static guarantees to web development. It is much needed.
The LtU thread now contains the start of a discussion on the topic.
You can't even open source your code using a license that doesn't restrict the rights of others. You would have to pay for a commercial license to be able to release your code under WTFPL, zlib, or MIT license.
Uh, I think this is wrong. Your code is not a derivative work of Opa, you can release it under whatever permissive license you want. (Of course, if users run it using Opa, they might have obligations with respect to Opa's license, but this is a different matter.)
The Opa web site says that I cannot: "Firstly, `‘if I'm using AGPL Opa to develop an app does it need to be AGPL, too?’'. Long story short — yes, it does." It would probably be better for me to consult the license, but I don't plan on reading the license just to make sure that I really can't use a language whose developers don't want me to be able to use it
I confirm (I'm the one who wrote that FAQ entry).
However, if you wish to make your code open-source with a different license, don't hesitate to contact [email protected], he'll certainly be happy to arrange something.
However, if you wish to make your code open-source with a different license, don't hesitate to contact [email protected], he'll certainly be happy to arrange something.
I agree that under the right circumstances, the code would have to be released under the AGPL. I disagree with the fact that you could not also release the code (your code, not including any of MLstate's code) under a permissive license like MIT or WTFPL. If you think you are in a position to forbid this, I would be interested to know your reasoning.
I actually have no clue. Perhaps hbbio can respond to this question. He's our CEO, btw.
Otherwise, you can try and ping our COO at [email protected] .
here is a blog post about the license:
http://blog.opalang.org/2011/08/opa-license-contributions.ht...
Regardless of license requirements, I think the following built in feature is interesting:
"way for your users to download the sources of the application. In fact Opa facilitates that by automatically enriching the server (in release mode) to serve the source code of the application at a special /_internal_/src_code URL."
"way for your users to download the sources of the application. In fact Opa facilitates that by automatically enriching the server (in release mode) to serve the source code of the application at a special /_internal_/src_code URL."
I do not believe "automatically generating client-side Javascript and handling communication and session control", in short: hiding redundancy and complexity, can "make web programming transparent".
IMHO, making something as complex as web programming transparent requires a lot of knowledge transfer. Making web development more easily accessible would be a fitting description. The idea is not new - see Ruby on Rails, even though it takes a different approach.
There is a lot of transparency in web programming if you are willing to learn. Roy Fielding's dissertation comes to mind as well as the book "RESTful Web Services", RFC 2068, RFC 2616 and ECMA-262. In the end it's "some server stuff" -- HTTP -- "some client stuff" where "some client stuff" is mostly Javascript.
Off topic: Does Blizzard's World of Warcraft client count as web development or is web development limited to the way Joel Spolsky describes it? [1]
PS: This comment is based solely on the introductory paragraph of OPA's home page.
[1] https://plus.google.com/117114202722218150209/posts/KhMRNmgG...
There is a lot of transparency in web programming if you are willing to learn. Roy Fielding's dissertation comes to mind as well as the book "RESTful Web Services", RFC 2068, RFC 2616 and ECMA-262. In the end it's "some server stuff" -- HTTP -- "some client stuff" where "some client stuff" is mostly Javascript.
Off topic: Does Blizzard's World of Warcraft client count as web development or is web development limited to the way Joel Spolsky describes it? [1]
PS: This comment is based solely on the introductory paragraph of OPA's home page.
[1] https://plus.google.com/117114202722218150209/posts/KhMRNmgG...
Fully agreed that current web programming is complex and that was the driving force behind making Opa.
What might not be clear in the presentation of Opa is that it's not some new glue on top of existing components.
It's instead a whole new, clean, technology for writing and running web applications.
And given that the language is statically typed (with inference in most cases, so it's easy for the programmer) with first-class web values, the whole point of Opa is that the compiler knows quite a lot about your application when generating the binaries. The typing system determines wether part of the code will run on the client side, on the server side (or both) for instance. This mechanism is very different from Ruby on Rails for instance, although both approaches share the goal of making the programmer's life easy.
Building RESTful services is exactly where Opa excels. Have a look at http://doc.opalang.org/index.html#_hello_web_services and the following chapter.
And given that the language is statically typed (with inference in most cases, so it's easy for the programmer) with first-class web values, the whole point of Opa is that the compiler knows quite a lot about your application when generating the binaries. The typing system determines wether part of the code will run on the client side, on the server side (or both) for instance. This mechanism is very different from Ruby on Rails for instance, although both approaches share the goal of making the programmer's life easy.
Building RESTful services is exactly where Opa excels. Have a look at http://doc.opalang.org/index.html#_hello_web_services and the following chapter.
Yeah, 'transparent' might be the wrong formulation. Still, Opa is great :)
Note: I'm part of the Opa team.
Note: I'm part of the Opa team.
Does the chat work?
http://chat.opalang.org/
http://chat.opalang.org/
[deleted]
Similar to GWT?
There are common points. However, Opa goes much further than Gwt (and its toolchain supersets) in terms of automation, concurrency, distribution and security.
Automation: by default, you do not have to give the compiler any hint as to whether code should be compiled towards the client, the server, the database, etc. The compiler solves all of this automatically, inserting whichever Ajax/Comet calls wherever needed. You can, however, provide annotations for optimization and/or security purposes.
Concurrency/distribution: at its heart, Opa is based on an actor model (although you do not need to use actors for simple applications). This actor model lets you connect easily clients, servers, etc. or subsystems through a very simple paradigm.
Security: every Ajax/Comet call inserted is checked for a number of criteria. In particular, the structure of messages is checked automatically, entry points generated dynamically for one single client can only be used by that client, etc.
Note: I'm one of the authors of Opa.
Automation: by default, you do not have to give the compiler any hint as to whether code should be compiled towards the client, the server, the database, etc. The compiler solves all of this automatically, inserting whichever Ajax/Comet calls wherever needed. You can, however, provide annotations for optimization and/or security purposes.
Concurrency/distribution: at its heart, Opa is based on an actor model (although you do not need to use actors for simple applications). This actor model lets you connect easily clients, servers, etc. or subsystems through a very simple paradigm.
Security: every Ajax/Comet call inserted is checked for a number of criteria. In particular, the structure of messages is checked automatically, entry points generated dynamically for one single client can only be used by that client, etc.
Note: I'm one of the authors of Opa.
I am the music man, I come from down your way and I can plaaay (what can you play?)
I play the trombone.
Opa opa opa-pa opa-pa opa-pa
Opa opa opa-pa opa opa-pa
I play the trombone.
Opa opa opa-pa opa-pa opa-pa
Opa opa opa-pa opa opa-pa
Opa is definitely something, but isn't going anywhere without the support of existing database servers. I don't really need that much of automation to create a prototype - and that's what Opa is only good at in it's current form, unfortunately.
Only a few obvious downsides/open questions to me:
* License prohibits making closed-source things in it.
* How efficient is the generated client-side code? Language mismatch is a major cause of slow generated code (see GWT). In Opa's case, having no arrays (only linked lists) sounds like a disaster for compiling to JS.
* I've never seen the model work before where you "don't even need to think about what's running on the client or server." I like features like sharing code (same language) and easy transport (serializable data structures), but frameworks that blur the client/server distinction usually download all of your code, or none of it, or little snippets on demand, or some other strategy that's worse than a manual distinction.
* What's development like? Do you really have to recompile your app after every change? My favorite frameworks just let you change the code and reload the web page.
* I've seen many, and even written one, tutorial that goes: "Making a chat room in a web browser is hard. But here's a demo that does it in 20 lines!" With all the frameworks and libraries out there these days, it's not that hard.
* Tons of dependencies (a personal gripe). Coming from the Java world, where lots of things like reading images are built-in, it's comical to see that Opa requires Java, AND libjpeg and a dozen other things (it doesn't actually run on the JVM I don't think).