Programming Language Development: The Past 5 Years(blog.fogus.me)

284 pointsby jashkenasvor 15 Jahren53 comments

blog.fogus.me

Programming Language Development: The Past 5 Years

http://blog.fogus.me/2011/10/18/programming-language-development-the-past-5-years/

56 comments

hammerdrvor 15 Jahren

I spent some time doing several small programming exercises in Ioke. I've also done similar exercises in several programming languages ranging from C to Java to Haskell to Ioke.

Ioke was ridiculously clean in almost every scenario. The only part I missed was that I didn't delve deep enough to really use the macros to their fullest extent. Something that would take 50 lines of idiomatic Ruby took 10 lines of Ioke. Ruby is already a very expressive language and yet Ioke could express the same thing in half the amount of code.

The author of this post is not kidding when he says that Ola designed the language with no regard for performance. The language is slow.

However, Ola is working on a language that learns from the expressiveness of Ioke but is a bit more practical. It's called Seph and is at http://www.seph-lang.org

And, finally, a small story: I was spending a few nights a week writing Ioke and trying it out about a year ago. Ola spoke about Ioke at an internal company presentation (sort of a mini-conference) and afterwards I started to talk about the language with him and Brian Guthrie. We talked about the language constructions and how we solved problems in the language, etc. etc. Finally, I get around to asking the question, "So, after writing Ioke for these past few weeks, I feel like I have no idea if I'm writing idiomatic Ioke!" Both of them look at me as if I'm a crazy person and finally Ola smiles and says, "There need to be more than 10 developers writing in a language for there to be idiomatic anything." Playing with these languages are fun but messy! Don't be afraid to make mistakes and just dive in.

beagle3vor 15 Jahren

You (and Ola ...) really should take a look at the APL / J / K family. All are 10-100x times more expressive than Ruby/Python, and 100x-1000x times more expressive than C and Java.

They are all quite fast, with K being super-blazing-fast.

I can't do these languages justice in a comment -- but have a look at https://github.com/kevinlawler/kona/wiki/Idioms and https://github.com/kevinlawler/kona/wiki - you are in for a shock about how unexpressive everything you've used so far is (and you don't have to give up speed to get that)

skrebbelvor 15 Jahren

Short isn't equal to expressive. Making every operation a single character does not really help understandability, IMO.

beagle3vor 15 Jahren

APL/K's expressiveness does not come from making every operation a single character (although they do work together nicely).

It comes from selecting a good set of primitive elements and their rules of interaction. It turns out (for K) that ~40 primitives are sufficient to cover what you need, and it is therefore possibly to assign these to single ascii characters (20 unary, 20 binary; e.g. "xy" is multiplication, but "x" is "first element of x". Most languages overload unary/binary on the same symbol).

Take, for example, the "maximum substring sum" problem (see link in the K idioms above). Using K syntax, it's

    |/0(0|+)\

Using Q syntax (K with words instead of characters), it becomes

    max over 0 (0 max +) scan

Three times longer, hardly using any non-alphabet characters. Just as unreadable to the uninitiated, and still 10 times shorter than a comparable Python or ruby implementation.

Once you do get used to thinking in APLish or Kish, then the first version to the second version is like "a+b*c-d" to "a added to b times c and then d is subtracted". I know which one I prefer.

Short isn't equal to expressive, but your comment is irrelevant to APL or K.

hvsvor 15 Jahren

The APL-family of languages aren't just short. Those single (and double) characters are each very powerful. APL was not based on the von Neumann style of programming. It is much more comparable to Haskell and the like.

[deleted]vor 15 Jahren

chubotvor 15 Jahren

But aren't these just domain-specific languages? It doesn't make sense to compare them to Ruby/Python or C/Java, because they don't solve the same problem. It's like saying that sed is more expressive than C.

Can you write a web server or a source control system in APL/J/K? I don't think so. You can compare Python and C because you can write a source control system in both (e.g. Mercurial and git).

beagle3vor 15 Jahren

> But aren't these just domain-specific languages?

No. Why would you think that?

> Can you write a web server or a source control system in APL/J/K?

Yes.

I'm not aware of a source control system written in any of them, but K's standard web server is implemented in K; there are several databases written in each of them - flipdb in APL, kdb+ in K, JDB in J. There are compilers written in each. There are raytracers, spreadsheets and sudoku solvers.

http://nsl.com/ contains a lot of K examples, some of them annotated. jsoftware.com is your source for J examples.

riffraffvor 15 Jahren

could you share an example of a ioke program with a 5x ratio to ruby code? I can think of many things that could be way less verbose in ruby, but I wonder where from does a a fivefold improvement come.

hammerdrvor 15 Jahren

I fear I am going to do a bad job of explaining this, but here goes:

You'll get the typical Lisp-like terseness and folding that'll make your complex code much smaller. This includes things like currying and passing multiple functions as parameters (I know both of these are possible in Ruby, but it really isn't idiomatic). This one, I think, is easier to understand especially if you've done some work in those types of languages.

The other benefit is more difficult to see at face value. What Ioke does is encourages you to write expressions and to build up your own language. This is a DSL and can be built in Ruby and Lisp and Javascript and many other languages. However, Ioke makes the DSL writing the "Pit of Success" that a developer encounters. It isn't a conscious choice, it is how you write your code. Simplistic example:

    square = lambda { |x| x * x }
    puts (1..10).inject(0) { |sum, x| sum + square.(x) }

In Ioke:

    square = method(x, x * x)
    (1..10) map(x, square(x)) sum println

This is obviously a trivial example that doesn't save any lines. They look very similar except that ruby has a bit more "noise". First, we're using the inject function. I love the inject function but it is a significant amount of noise in the ultimate goal of a readable line of code. Ioke has 4 words that are all significant: map square sum println. They are used once and only once. Ioke's space a function applicator cannot be overlooked, though. It is syntax, yes, but it really encourages "sentence writing" a la Smalltalk. And, finally, Ioke reads left-to-right unlike the C-inspired languages which read inside-out. In fact, if you were to describe this problem in English, you would probably write something similar to:

For the range 1 to 10, square each number and then sum the results.

Ioke maps very well to this.

            (1..10) map(x, square(x))        sum          println

The problem with this trivial example is that this expressiveness is just slightly off for Ruby. But when we grow the program to scale, in Ruby we lean on the classical inheritance-based Object Oriented model that is available in Ruby. In Ioke, we continue to build a DSL.

I hope that makes a little sense?

Edit: Let me also say that I'm madly in love with Ruby and think its a great language :) By no means does "Less Expressive than Ioke" mean "Not Expressive Enough" or "Bad Language"!

rauljaravor 15 Jahren

But you can do the same map square, then sum in ruby 1.9:

puts (1..10).map{|x| x*x}.inject(:+)

And if you don't like writing inject(:+) all the time, it's pretty easy to monkey patch in a sum method to either array or enumerable.

draegtunvor 15 Jahren

And Perl6 can also provide an expressive example:

  say [+] (1..10).map: {$_ * $_};

JadeNBvor 15 Jahren

No need to wait for Perl 6:

    my $tot = 0; map { $tot += $_ } map { $_ ** 2 } 1 .. 10; say $tot;

EDIT: Or, regarding List::Util (http://search.cpan.org/~gbarr/Scalar-List-Utils-1.23/lib/Lis...) as part of Perl 5,

    say sum map { $_ ** 2 } 1 .. 10;

EDIT 2: (Golfing is fun!) Or an Applescript-y

    sub the { @_ } sub of { @_ }
    sub squares { map { $_ ** 2 } @_ }

    say the sum of the squares of 1 .. 10;

draegtunvor 15 Jahren

No need to wait for Perl 6

Absolutely :)

  use autobox::Core;  # or perl5i::2

  [1..10]->map(sub{ $_ ** 2 })->sum->say;

Or if you like a double helping of sugar in your tea!

  use Whatever;

  [1..10]->map( &_ ** 2 )->sum->say;

JadeNBvor 15 Jahren

I thought `Whatever` was a joke, and had to go look it up on CPAN (http://search.cpan.org/~asg/Whatever-0.21/lib/Whatever.pm). Not to belabour a joke, but the examples there suggest that the map should be

    map( $* ** 2 )

not

    map( &_ ** 2 )

draegtunvor 15 Jahren

No it's quite a serious & well written module. Hopefully it will become part of perl5i - https://metacpan.org/module/perl5i

  &_ is an explicit $_   You use $* or &* when you need to check $_[0] first.
  
  So &_ is more correct with general map but $* (and &*) will work also in 
  this autobox example (in fact i had used it in my first edit but preferred 
  aesthetics of &_ especially with previous usage of $* only being deprecated 
  in 5.10).

NB. Above in code just for aesthetics!

JadeNBvor 15 Jahren

> In Ioke:

    square = method(x, x * x)
    (1..10) map(x, square(x)) sum println

> Ioke has 4 words that are all significant: map square sum println. They are used once and only once.

Sorry to be that guy, but `square` is clearly used twice above. Why not `(1..10) map(x, x*x) sum println`? (I don't have Ioke on this computer, so I can't easily check if this works.) Or are you assuming that `square` would probably have been defined somewhere already?

draegtunvor 15 Jahren

Not surprising its nearly identical to how you would write it in Io (http://iolanguage.com):

    square := method(x, x * x)
    1 to(10) map(x, square(x)) sum println

and also the variation:

    1 to(10) map(x, x * x) sum println

hvsvor 15 Jahren

I went to a Chicago ACM presentation of Ola's a couple of years ago talking about Ioke. It was really pretty impressive.

Seph (on the surface) actually reminds me a lot of Smalltalk.

necubivor 15 Jahren

For anyone interested in AGDA, I found this [0] paper really helpful for getting my head around dependent typing. I sort of doubt that dependently typed languages are ever going to catch on, but they embody some seriously cool ideas about how to verify program correctness.

Also, AGDA has probably the most flexible syntax for defining function fixiness, letting you easily define new "syntax":

   if_then_else_ : {A : Set} -> Bool -> A -> A -> A
   if true then x else y = x
   if false then x else y = y

[0] http://www.cse.chalmers.se/~ulfn/papers/afp08/tutorial.pdf

ehsanu1vor 15 Jahren

fogus mentions OMeta, which is amazing in my own humble opinion. You can, extremely concisely, create a parser for a language you dream up. The author of OMeta has an example of parsing JavaScript in 200 lines of OMeta. OMeta can also parse itself in about 40 LOC. This conciseness is the reason Alan Kay's Viewpoints Research Institute is using it for their STEPS project (attempting a full GUI system from "scratch" in 20k LOC). They use it to compile all the languages in the project, and even make use of it in their 200 LOC TCP implementation.

The example gist provided really doesn't do it justice (partly because there's a variant with slightly lighter syntax). Check out the sandbox for OMeta/JS (there are several OMetas, each for its own host language), which has several projects you can check out: http://tinlizzie.org/ometa-js/#Sample_Project

alexandrosvor 15 Jahren

Let me add to the OMeta love. I did my PhD using OMeta/JS, and I'm using it now in my startup. I find it's a perfect companion to CoffeeScript, both of them compiling to JavaScript as they do.

ehsanu1vor 15 Jahren

Awesome, would you have a link to your PhD thesis? Couldn't find it on your blog. Also wondering how you're using OMeta for your startup?

Since you mention CoffeeScript, since I've found out about OMeta last week, I've started a little work on a CoffeeScript compiler in OMeta. Have some basic stuff working, but error messages for parsing kinda suck: the best I can do with OMeta/JS as currently implemented seems to be noting the position at which there's a parsing error.

jashkenasvor 15 Jahren

I'd love to see a CoffeeScript compiler implemented in OMeta -- please post it to the GitHub Issues if you come up with something neat.

ehsanu1vor 15 Jahren

Sure thing, glad to see the interest.

FraaJadvor 15 Jahren

I'm surprised that Factor was not featured.

[1]: http://factorcode.org/ [2]: http://concatenative.org/wiki/view/Factor/Features/The%20lan...

technomancyvor 15 Jahren

This article is specifically about languages created in the past 5 years, so Factor doesn't qualify.

FraaJadvor 15 Jahren

Ah, yes. Technically that is a good reason.

However, Factor has gotten a lot of new features in the recent years. For a relatively young language (7 years) the last 5 years are quite significant.

wgrovervor 15 Jahren

Cheers for including Scratch (a graphical programming language aimed at kids). Scratch is worth playing with, if only to witness the potential (and drawbacks) of graphical programming languages. Now back to my LabVIEW code...

julesvor 15 Jahren

Magpie: http://magpie.stuffwithstuff.com/

Had a very interesting approach to its type system, but that has been temporarily disabled because the language was changed to use multiple instead of single dispatch.

jxcolevor 15 Jahren

So he seems to not like Go or Arc. I like Go a lot but I am always interested to hear other opinions. While I'm sure he has good reasons for this, I wish he would have included them. It seems like he's saying "I don't really like this, here's a link!"

Detrusvor 15 Jahren

Those languages were meant to stretch your mind

Go is not in the mind stretching camp, it's just practical. It's made from existing ideas that you would have seen in other languages that came and went. So in that sense it's not interesting and yet it is.

adeelkvor 15 Jahren

Actually, the comment about mind-stretching was referring to the languages in his “Perlis Languages” post.

urielvor 15 Jahren

Go is mind-stretching in its practicality and pragmatism.

Most (but not all) its ideas have been implemented before elsewhere (but then, that is almost bound to be true of any new language).

But the real 'innovation' is in the careful selection of those ideas and in how well they work together _in practice_.

This puts off people who are used to this or that feature in other languages, Go is not about checklists of features or abstract arguments about how languages should be, it is about what works really well when you sit down to write (or read) code.

gtanivor 15 Jahren

His next book:

    No Joy in Go ;-}

william42vor 15 Jahren

Yeah, but why would you want to implement Joy[1] when Factor[2] exists?

[1]http://en.wikipedia.org/wiki/Joy_(programming_language) [2]http://factorcode.org/

mark_l_watsonvor 15 Jahren

A good read. Michael is one of the few people who I read just about everything that they write.

Using alternative languages is refreshing, both new and old. After many days of doing Java server side (and some web dev in SmartGWT) for a customer, I cleared out the cobwebs tonight by installing both SBCL and Clozure Common Lisp on a new MBA, configured Quicklisp for both, and started updating some of my old utilities and programs to build and run with the Quicklisp package manager (that I wish I had ten years ago), tried hacking a bit with weblocks and clouchdb (yeah, I spelled that right). I have done a lot of Lisp development since 1980, but not so much this year except for some Clojure for a customer, so I had fun, even if Common Lisp is a lot older than 5 years old!

My favorites on his list of new languages: Clojure and CoffesScript. CoffesScript is a good, practical idea. Although I have never used Scratch, I have sat with my granddaughter while she uses it.

jfbvor 15 Jahren

Good stuff. It makes me long for more time in the day and more neurons in the skull.

[deleted]vor 15 Jahren

(1)

djhworldvor 15 Jahren

Just reading the Shen example has made me question my choice of learning Clojure, with the simple example of partial application.

I dream for the day when you can just do (map (* 2) [1 2 3]) in Clojure

ataggartvor 15 Jahren

Why wait?

  user=> (defmacro mapp [part coll & colls]
          `(map (partial ~@part) ~coll ~@colls))
  #'user/mapp
  user=> (mapp (* 2) [1 2 3])
  (2 4 6)

fogusvor 15 Jahren

Clojure provides function arity overloading and varargs, so automatic partial application is out. It's a tradeoff that Clojure alleviates by providing a lightweight function literal:

    (map #(* % 2) [1 2 3])

An added advantage of Clojure's approach is that it's not limited to partial application on the last argument.

To provide implicit partial application or arity overloading and varargs is a fundamental shift in the way that the language operates. However, the gap between (in the direction Clojure->Shen only) one and the other is 2 characters, or a mapp macro like Alex showed. I'd make that trade any day of the week.

paulkoervor 15 Jahren

On one hand I feel the same - Shen looks incredibly nice. And it features a lot of things that I hope Clojure will have in the future (such as an optional static type-systems and high portability due to Clojure in Clojure). Then on the other hand a language's usefulness is to a large degree determined by having a minimal viable userbase - you just cannot write every library yourself. And by being highly pragmatic, running on the JVM and so forth (and probably many other things I don't comprehend) Clojure has achieved just this. So I think right now its the most practical Lisp. If Shen has a lot of nice features maybe we can learn from it!

bitcrackervor 15 Jahren

You don't need Shen features in Clojure because the Shen team aims to provide a compiler for KLambda which is a very small subset of Lisp. It should be no problem to implement KLambda in Clojure.

Quote from https://groups.google.com/group/qilang:

"Much of the work necessary for converting to Clojure or Python (Or any other non-tco language) is being done as a pass over KLambda right now in my JS port. Once this is finalized, it should be relatively easy to port the transformed KLambda code to any architecture that supports exceptions or labels."

nathanwdavisvor 15 Jahren

I'm disappointed F# was not on his list. Not surprised though.

eddieplan9vor 15 Jahren

To be fair, F# is, instead of a new language, mostly a re-implementation of OCaml for .Net platform, in the same camp as IronRuby and IronPython. They even refer it as Caml.net sometimes:

http://research.microsoft.com/en-us/um/cambridge/projects/fs...

MichaelGGvor 15 Jahren

F# has been around for well over 5 years, although the last 5 years have seen many new features to it.

piranhavor 15 Jahren

It's 9 years old, which is more than 5 years.

grayrestvor 15 Jahren

Is F# in the past 5 years? I remembered it as being a bit older.

ThaddeusQuay2vor 15 Jahren

In addition to my list of eight, in another comment here, I am throwing in Logtalk. I wouldn't label it as "9", given that it came out in 1998, but it's worth noting, because it is still developed, and because its depth fits with the spirit of the original post.

"Logtalk is an object-oriented logic programming language that can use most Prolog implementations as a back-end compiler. As a multi-paradigm language, it includes support for both prototypes and classes, protocols (interfaces), component-based programming through category-based composition, event-driven programming, and high-level multi-threading programming." - http://logtalk.org

"Logtalk is an object-oriented logic programming language that extends the Prolog language with a feature set suitable for programming in the large. It provides support for encapsulation and data hiding, separation of concerns and enhanced code reuse. Logtalk uses standard Prolog syntax with the addition of a few operators and directives. Logtalk is distributed under an open source license and can run using ISO-compliant Prolog implementations as the back-end compiler." - http://en.wikipedia.org/wiki/Logtalk

ThaddeusQuay2vor 15 Jahren

Here's my partial list, of what I can remember at the moment, and in no particular order.

1) Opa

"Opa is a concise and elegant language for writing distributed web applications." - http://opalang.org

"Opa is an open source programming language for web applications." - http://en.wikipedia.org/wiki/Opa_(programming_language)

There is much to say about Opa. I recommend going through the references in the Wikipedia article.

2) BaCon

"BaCon is a free BASIC to C converter for Unix-based systems. BaCon intends to be a programming aid in creating tools which can be compiled on different platforms (including 64bit environments). It tries to revive the days of the good old BASIC." - http://basic-converter.org

BaCon is interesting because: 1) all you need is a shell and a C compiler, 2) it was created by Peter van Eerten, who also made GTK-server, and 3) it converts lazily, effectively making a BASIC-like wrapper for C, thereby allowing easy access to some of the more interesting aspects of C, which would not normally be available in BASIC.

3) Spin

"Spin is a multitasking high level computer programming language created by Parallax's Chip Gracey, who also designed the Propeller microcontroller on which it runs, for their line of Propeller microcontrollers." - http://en.wikipedia.org/wiki/Parallax_Propeller#Built_in_SPI...

4) Agena

"Agena provides you with all the means you need to implement your ideas quickly: fast real and complex arithmetics, efficient text processing, graphics, flexible data structures, intelligent procedures, simple package management, plus various configuration facilities in multi-user environments. The syntax resembles very simplified Algol 68 with elements taken from Maple, Lua and SQL. Agena is based on the ANSI C source code of Lua." - http://agena.sourceforge.net

5) Monkey

"Monkey is a brand spanking new programming language that allows you to create apps on multiple platforms with the greatest of ease. Monkey works by translating Monkey code to one of a different number of languages at compile time - including C++, C#, Java, Javascript and Actionscript." - http://monkeycoder.co.nz

"In 2011, BRL released a new cross-platform programming language called Monkey and its first official module called Mojo. Monkey has a very similar syntax to BlitzMax, but instead of compiling direct to assembly code, translates Monkey source files into source for a chosen language, framework or platform." - http://en.wikipedia.org/wiki/Blitz_BASIC#Monkey_and_Mojo

6) LOLCODE

"LOLCODE is an esoteric programming language inspired by the language expressed in examples of the lolcat Internet meme. The language was created in 2007 by Adam Lindsay, researcher at the Computing Department of Lancaster University." - http://en.wikipedia.org/wiki/LOLCODE

7) Neko

"Neko is a high-level dynamicly typed programming language. It can be used as an embedded scripting language. It has been designed to provide a common runtime for several different languages. Learning and using Neko is very easy. You can easily extend the language with C libraries. You can also write generators from your own language to Neko and then use the Neko Runtime to compile, run, and access existing libraries." - http://nekovm.org

"Neko is a high-level dynamically typed programming language developed by Nicolas Cannasse as part of R&D efforts at Motion-Twin." - http://en.wikipedia.org/wiki/Neko_(programming_language)

Neko is one of the targets for haXe, and was created by the same guy. Neko is six years old, but given its importance, I think that's close enough.

8) Piet

"Piet is a programming language in which programs look like abstract paintings. The language is named after Piet Mondrian, who pioneered the field of geometric abstract art." - http://dangermouse.net/esoteric/piet.html

"Piet is an esoteric programming language designed by David Morgan-Mar, whose programs are bitmaps that look like abstract art. The compilation is guided by a "pointer" that moves around the image, from one continuous coloured region to the next. Procedures are carried through when the pointer exits a region. Piet was named after the Dutch painter Piet Mondrian." - http://en.wikipedia.org/wiki/Piet_(programming_language)

I don't know the age of Piet, but the 99 Bottles of Beer example is from just over five years ago.

http://99-bottles-of-beer.net/language-piet-1269.html

--------------------------------------------------

Before anyone complains, the article says:

"In this post I will provide a list of fairly new languages (let’s say 5 years with a little flex) that display interesting features and display higher-order thinking in the way that they tend toward an evolution of past learnings in programming language thinking. Not all of these languages enjoy active development, but the more important point is that they represent in some way "new language thinking". Remember that this goal does not necessarily mean "innovative"."

I think that each of the languages I've specified applies, although, for some, my interpretation of "evolution" may have been a bit loose.

minikomivor 15 Jahren

Do you know any more about how monkey came about? The about page is pretty sparse.. but it seems to be a pretty vibrant community.

ThaddeusQuay2vor 15 Jahren

I don't have any special knowledge of any of these languages, but the full version of Monkey costs $120. If you are willing to spend that much, then you might want to first look at a more mature competitor, NS Basic, which has been around, in one form or another, since 1998, and which costs about the same as Monkey. I'm not sure what all of the differences and similarities are, but I notice that they released an update to their "NS Basic/App Studio" today.

http://nsbasic.com

http://en.wikipedia.org/wiki/NS_Basic

schiptsovvor 15 Jahren