If you want to learn Lisp, be ready to meet the scowling faces (and don't even think about mentioning PG in #lisp)(classbug.com)

22 pointsby palishvor 19 Jahren51 comments

If you want to learn Lisp, be ready to meet the scowling faces (and don't even think about mentioning PG in #lisp)

http://classbug.com:3000/post/main.html

52 comments

Chapter 3 of The Structure and Interpretation of Computer Programs is a great discussion of mutable state (make sure you read the previous chapters, though, or at least the first chapter and The Little Schemer).

A functional-ish way to run a game world simulation:

(define (run-game)

  (define (make-initial-world) ...)

  (define (get-input) ...)

  (define (wait-for-next-frame) ...)

  (define (update-world world input) ...)

  (define (game-over? world) ...)

  (define (run-step world)

    (display-world world)

    (wait-for-next-frame)

    (if (game-over? world)

      'bye

      (run-step (update-world world (get-input)))))

  (run-step (make-initial-world)))

It's not purely functional: the calls to display-world and wait-for-next-frame are made for their side effects. But they are not mutating any program state, so this is good enough. You _can_ try and get more 'pure' with Haskell and monads when you feel brave, but there is plenty to grok before going there.

Let's study the parts:

make-initial-world: self-explanatory; I'll only add that this object is held 'in the stack' and (conceptually) never overwritten.

get-input returns the player actions, game time, and whatever other 'delta' data is relevant for the current update. You can think of the input over a whole game session as an indefinitely long list which you iterate over, one element (game update) at a time. You can indeed run automated tests replacing this data with an actual list; the program shouldn't notice the difference.

wait-for-next-frame causes a time delay that is conceptually irrelevant to the rest of your logic (you deal with system time, if at all, in the body of (get-input)). In test situations, you should be able to just replace this with a no-op.

game-over? tells you whether the player has quit. I make it a function of the world and not the input because the input is meaningless without the context that the world object represents.

(By this time you may have noticed that I'm being a bit sloppy with terminology, for color and simplicity. 'world' is not only the game world, but also the program state (e.g., is the user in a configuration screen?). Similarly, 'game over' is not the end of one game/'match' but of the playing session. The principles are the same.)

run-step is just a loop that keeps updating and displaying the world state until the game is over. Don't be scared by the recursive call to run-step itself; that's the idiomatic way to loop in Scheme, and implementations are required to handle it efficiently (google for 'tail recursion').

The crux of the problem is in update-world. This is an ordinary function that takes the world and the input for this game tick, and returns the world corresponding to the next tick.

You'd think there are only two options here: either modifying the existing world in place- definitely non-functional!-, or building a new one from scratch, which sounds inefficient.

Actually, there is much more to be said about purely functional data structures. In a nutshell, you can often make a new world that shares most of the data with an old one, so only the delta is created anew. References to the old world will not be affected at all by the new one, so you can keep as much history of your game as you have memory for. As a basic example, in Lisp you do this with lists all the time. But it can get more complex:

http://www.cs.cmu.edu/~rwh/theses/okasaki.pdf

(This thesis was expanded upon and published as a book by the same name, with examples in ML and Haskell. If someone knows of more resources about this, please let me know.)

Depending on the ratio of changing-data-per-frame versus total data in your application, this may be an interesting option. The learning curve is considerable, though.

But note that when update-world is called the old world will never be used again (most games don't keep the state of every past frame). So in this case it's OK to modify the old world in place and use that as the return value. The rest of the code doesn't care; again, you might as well be iterating over a stream of worlds (with the same desirable implications for testing).

If you do mutate the world, though, you may want to make it explicit by saying something like

(define (run-game)

  ...

  (define (run-step)

    (update-world! world (get-input))))

    (display-world world)

    ...)

  (define world (make-new-world))

  (run-step))

This is harder to test and more work if you want to switch to a purely functional update-world later, but there's less risk you or others will try and use (or modify!) a world after it's been updated.

Part of the calculations _within_ update-world can still be done in a functional way, and where you need to mutate state, you can often hide that behind a functional interface, with varying degrees of hygiene.

The bottom line is you don't need religious avoidance of mutable state to benefit from functional programming at all; you strive for stateless code by default, you compromise where it makes sense, and you try to minimize the impact of such sins by wrapping functional interfaces, or placing warning signs, around them. The mere habit of paying attention to these matters will improve your taste.

There is a lot more to be said about this, but I hope you can already see how this is not an easy topic to introduce, out of the blue, via IRC. In all fairness to the Lisp folks, I have to say the question about Python packages was much more concrete and easier to address without both sides losing patience.

euccastrovor 19 Jahren

I just noticed that although I said you can treat input as a stream, the way you consume the input in the example doesn't reflect that. If you have a primitive (get-input) function like the one I assume above, you can wrap it in a more functional interface like this:

(define (new-input)

  (cons 'input (get-input)))

(define (input? x)

  (and (pair? x)

       (eq (car x) 'input)))

(define (input-val input)

  (cdr input))

(define (current-input input)

  (if (input? input)

    (input-val input)

    (car input)))

(define (future-input input)

  (if (input? input)

    (new-input)

    (cdr input)))

...

(define (run-game initial-world input)

  (define (run-step world input)

    ...

      (run-step (update-world world (current-input input))

                (future-input input)))

  (run-step initial-world input))

(run-game (make-initial-world) (new-input))

Now you can just pass a list as the input, to test stuff or to run a demo. Note that current-input behaves much like car; you can call it as many times as you want and you get the same result (this is called referential transparency). You can wrap your random number generator and other stuff in a similar way.

This can be further refactored so you can get out a stream of worlds too, so you can compare it to pre-recorded or hand authored test data. For practical purposes, you'll normally be testing smaller parts of your system; the same principle works for those.

palishvor 19 Jahren

Great info, great detail. Thanks so much for your time. I've never even considered looping through recursion. The construct-a-new-world vs modify-the-current-world situation was exactly the problem I was thinking of. And you followed it up with a paper, if I wanted to learn more. It's hard to emphasize how perfect stuff like this is for a newcomer to Lisp. There should be more snippets like this. You should write a tutorial somewhere.

henningvor 19 Jahren

Scheme has kind of a different culture from Common Lisp. Most of the material is oriented towards students in order to teach the timeless stuff in computer science.

I learned C before I learned any functional languages and I regret it to this day. It took me _years_ to undo the damage. I really dug myself into a hole, too, by learning all about pointers and low-level stuff like that before I ever knew what Python, Ruby, etc. were.

euccastrovor 19 Jahren

Why, thanks! I myself reinforced some concepts while writing it, since I have more of a Python background. I like the idea of writing a tutorial about this, but I'd rather experiment more first.

euccastrovor 19 Jahren

Argh! Anyone knows if/how can I introduce a pre block?

ralphvor 19 Jahren

Yes, you cannot.

gyro_robovor 19 Jahren

Pretty ironic considering just about everyone here is a programmer, and it's run by someone famous for being a programmer and who is a language designer.

jlfvor 19 Jahren

I was lurking on #lisp for most of today and it seems to me you're being a bit oversensitive. You asked about packages at 14:23. Several people tried to figure out exactly what you were asking, including gigamonkey (the author of PCL!). At 14:26 rahul mentioned "asdf" which is probably what you are looking for. If you had at that point Googled 'asdf lisp' and felt lucky, you would have gone straight to this page: http://www.cliki.net/asdf , which contains a script to automatically generate packages as well as some other Windows-specific information that you were asking about. Otherwise, at 14:39 in the complete log, jamesjb pointed you to a tarball containing an example system. Not too shabby IMHO.

Incidentally, http://www.cliki.net/IRC mentions #cl-gardeners and #lispcafe as more novice-friendly than #lisp, though I've never been too troubled by my own experiences there as a newcomer to Lisp.

Lastly, you might want to check out http://www.google.com/coop/cse?cx=012679172708151972086%3Aeg1jtvm_dlk (hopefully the link did not break) -- it's a custom Google search that searches a set of 20 Common Lisp resources. It's pretty useful.

Good luck.

palishvor 19 Jahren

Thanks for the awesome resources. I'm honored that the author of a book took the time to speak to me. If I was being oversensitive though, the average newcomer has no chance.

rzwitserlootvor 19 Jahren

Just to play devil's advocate here:

I'm a frequent contributor on The GWT newsgroups and it's very easy to get annoyed. Specifically, java suffers from a lot of wannabe programmers. For those of you liking the ambiance amongst Ruby and Python groups, you should do yourself a favour and stop advocating the use of Ruby/Python in colleges.

The influx of those idiots will kill your newsgroups and your IRC channels -very- quickly.

Now, as to the LISP community, they aren't usually bothered too much, I'd gather, with wannabees. However, where they do waltz I'm, I can imagine they'll be mentioning Paul Graham's essays. Result: short fused posters take the piss out of you.

And, you have to admit, Paul Graham's ramblings on LISP are a bit out of left field. The problem with promising magical performance with any one programming tool is that you create expectations. I'm fairly sure the expectations created by PG are waaay out of proportion of what you can reasonably expect a functional programming newbie to do in the span of a couple of months.

This isn't an excuse or anything. The chat logs are certainly horrible. Just trying to figure out if I can explain why this is happening.

pgvor 19 Jahren

Can you give me an example of a specific paragraph of rambling you consider to be out of left field?

rzwitserlootvor 19 Jahren

I'm referring to the tales of orders-of-magnitude flexibility and productivity enhancement, mostly the "Beating the Averages" essay, in other words. Here's a fragment that seems to promise almost magical productivity:

It must have seemed to our competitors that we had some kind of secret weapon-- that we were decoding their Enigma traffic or something. In fact we did have a secret weapon, but it was simpler than they realized. No one was leaking news of their features to us. We were just able to develop software faster than anyone thought possible.

pgvor 19 Jahren

That's out of left field? We were writing code in Lisp, with a read-eval-print loop to test stuff, and they were writing code in C++ with "builds."

There's nothing preposterous in claiming we could implement a serious feature in a couple hours. A lot of people could today with Ruby or Python. But there were no Ruby or Python at the time I'm describing. (An early form of Python existed, but few used it.)

far33dvor 19 Jahren

Yes. But I bet a lot of the consternation around the LISP literati circles about PG's essays is because they like LISP the way it is: obscure.

If LISP became popular, they wouldn't be as special anymore. It's like the punk rock syndrome - as soon as a band is popular, the original listeners pretend they don't like it anymore (and look down on anyone who isn't "old school")

gyro_robovor 19 Jahren

Don't forget

http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/88a36fb2c239a44e/89405e202723a377

palishvor 19 Jahren

Holy jeez, that's way worse. I guess I should consider myself lucky.

Alex3917vor 19 Jahren

You should pick up a copy of The Little Schemer. It's entirely Socratic, assumes no background knowledge, and is probably one of the best books ever written on any subject. And it just happens to be on Scheme, so all the better.

And while Scheme varies in syntax, the concepts are the same.

herdrickvor 19 Jahren

Great call, Alex. That's one of my favorite books ever. I blogged this here: http://herdrick.blogspot.com/2006/03/what-all-books-ought-to-be.html

Alex3917vor 19 Jahren

Good post. The only other Socratic book I own is "What is Calculus about" by W.W. Sawyer. It's more loosely Socratic though. There are paragraphs of text interspersed with questions that either ask you to look for a pattern, or else follow a statement to its logical conclusion.

Not as good as The Little Schemer, but it's still cool that you can learn traditional (non-lambda) calc knowing only addition and subtraction.

[deleted]vor 19 Jahren

palishvor 19 Jahren

Cool. Does Scheme work on Windows?

Tronvor 19 Jahren

Yep. Check out DrScheme: http://www.plt-scheme.org/software/drscheme/

palishvor 19 Jahren

Wow; what an amazing breath of fresh air.

Alex3917vor 19 Jahren

Yes. Also, SICP uses Scheme for its examples, so that's the other reason you want to understand Scheme code.

(If you look up SICP on Amazon you'll see PG's review on page one)

euccastrovor 19 Jahren

Peter Norvig's review was on the front page too. I found that one even more helpful.

willarsonvor 19 Jahren

It often feels like the lisp way is to figure it out yourself, at any cost, and without any external help. For whatever reason the community feels skewed towards egotistical hackers without much sympathy.

I have enjoyed working with CL, and built several large projects in it, but the packaging is a serious pain. At some point you accumulate sufficient knowledge that you can use the different packaging systems, but until then you are essentially sol.

A number one complaint about CL is the lack of libraries, but I think the real issue isn't that the libraries don't exist, but that installing them often isn't worth the effort. C'est la vie, but its still a pity.

vikramvor 19 Jahren

Can you clarify why you think it's not worth installing the libraries?

ryantmulliganvor 19 Jahren

Palish- I sympathize you. I recently read PCL and started to code a project with it. I have a pretty firm idea of where I wanted to go and ran into the same issues with ASDF that you mention. Fortunately I was on linux so I was able to fix them quickly. Still I was just fed up with a lot of the unclean syntax that I had to write to make stuff work. No one was sympathetic with my viewpoint on #lisp and they all said there was no such thing as "clean and consistent" code when I argued that I would use Arc if it was more clean and consistent than ANSI CL. I came to the conclusion that the community sucks and I don't want to be apart of it, even if their language is better. Anyway to celebrate my decision I went over and coded some rubini.us . Cheers.

willarsonvor 19 Jahren

So you went into a programming language's IRC chatroom and began arguing that a non-existent language is better then theirs. I don't really envision many language enthusiasts responding well to that. Is the next step is for us to debate with pg that Fortress is better than Arc?

ryantmulliganvor 19 Jahren

I referenced that point out of context.

Someone asked, "Is there any demand for Arc?", After people were joking about the 'Invisible Arc.' I said, "If it is more clear and consistent, I demand it." I was merely stating my demand, not arguing with them that Arc is better. I don't know if Arc is better, but I know that if the Community of Arc is better than #lisp that would be a great start.

willarsonvor 19 Jahren

Sorry, it just seemed a mite too easy. :) Its pretty hard to find a community worse than lisp, but a big part of that is that lisp is at a very different stage of its lifetime than almost any other language out there.

Python/Ruby/Perl/TclTk et al are very young, and tend to be much more friendly (interestingly many feel that the Ruby crowd is getting a bit antsy these days, an acceptable lisp afterall? never). I'm hard pressed to explain why C is still relatively friendly, but I suspect it is in large part because there are many more C programmers, many more new C programmers, and that C finds people, whereas people find Lisp.

omousevor 19 Jahren

Non-existent? Then what are you using right now? Invisible news.yc?

jeyvor 19 Jahren

How To Ask Questions The Smart Way

http://www.catb.org/~esr/faqs/smart-questions.html

I didn't read the whole transcript, but the above advice is a nearly foolproof formula for getting good answers.

dpapathanasiouvor 19 Jahren

Well, you're not the first one to complain about it: http://social-problems-of-lisp.blogspot.com/

But personally, I've found that if you post a question to the c.l.l. newsgroup about how to do something specific, you'll get good replies.

It's the more theoretical and philosophical type of posts which wind up attracting the trolls en masse.

ricky_clarksonvor 19 Jahren

You made assertions that possible things were impossible, and asked how to do intrinsically non-functional things in a functional way.

I think you experienced a typical IRC reaction of "annoy the newbie because it's easier than answering him". It seemed to work. I've seen it in non-Lisp channels too.

You need to store things on the heap or on the stack, there is nowhere else. When you're thinking in objects, program in objects. To think in objects but then try to write functionally is going to cause you difficulty. You can write procedurally in Lisp, without OOP, it's not just a choice between OOP and FP. Lisp programmers treat these as tools, not lifestyle choices.

palishvor 19 Jahren

I think you experienced a typical IRC reaction of "annoy the newbie because it's easier than answering him". It seemed to work. I've seen it in non-Lisp channels too.

Hell yes it worked; that's why I wrote this article. The problem with the Lisp community is that it's the first reaction, not the last. You won't get this on #gamedev. You'll either get a "Google it" or a helpful response.

I also asked if Lisp constructs could be stored on the stack, instead of the heap; not whether they could be stored in some magical place that doesn't exist.

shirovor 19 Jahren

Yeah, I've seen the pattern. Part of the problem is, I guess, that Lisp has long history and stable user base so that the newbie/expert ratio is much lower than other languages; and (a) experts tend to forget how they were like when they were newbies, or (b) even they remember, there weren't such a thing like irc back then and they figured out most by themselves, so they expect a newbie nowadays can do the same thing.

However, another part of the problem, which I think is larger, is that the way of functional thinking is so different than the way of procedural thinking, and it is simply impossible to tell concisely "this is the functional way" (except very abstract statement like "avoid side effects"). If the asking one only has procedural background, his very question often doesn't make sense when viewed from the functional world; and we cannot answer a question which is not really a question.

Take one example: You asked about stack and heap. Probably you assumed a specific storage model, used mainly in C, C++, etc. However, in many functional languages, the extent of objects are just infinite by default, conceptually---and you just let the compiler to figure out which objects can be stack allocated and which should be heap allocated. Or, even a compiler can allocate everything on stack first, and when stack is popped or becomes full it moves live objects into heap, effectively using stack as a nursery of objects. It all depends on implementations, not the language. See, just asking "how to put an object on the stack instead of a heap" doesn't really make sense, except you are talking about a specific optimization technique on a specific implementation (in which case you should've already known the language and quite deeply about the implementation.)

This is just an example. Treating states, when to use mutation, how to model I/O, ... all require very different views.

I program in Lisp/Scheme both for work and hobby, and I feel sorry about your experience. If you still think giving a try to the functional way, another path may be to taste a few other functional languages (Haskell seems to get quite attention recently, and ML claims very fast); then you'll get the perspective of procedural vs functional ways, instead of (all other languages) vs Lisp.

ricky_clarksonvor 19 Jahren

If you look at the relevant logs, and see what Peter Seibel said to you, your question was answered immediately. I'm not sure that you noticed though.

diviavor 19 Jahren

Has anyone here had any experience using newlisp? There are things I don't love about it, like the lack of backquote syntax, but the community seems a lot friendlier than what palish has had to deal with. That being said, my experience with comp.lang.lisp hasn't been bad. I posted a simple question about destructive functions there a while ago and got a helpful response pretty quickly.

nostrademonsvor 19 Jahren

NewLisp is dynamically scoped. That's a showstopper for many people who have any experience with dynamically-scoped languages (eg. Emacs Lisp, Maclisp).

awtvor 19 Jahren

I had much the same experience myself in the channel, though after much brow beating I did eventually get my questions answered.

Shorelvor 19 Jahren

The famous books are not good for beginners.

This one is very good for beginners: http://www.cse.buffalo.edu/pub/WWW/faculty/shapiro/Commonlisp/ In fact, that one is the one that finaly teached me Lisp.

adamdoupevor 19 Jahren

I think the way to do this functionally is the Haskell way with Monads: http://en.wikipedia.org/wiki/Monads_in_functional_programming

Having used Haskell (which is purely functional) to write a compiler for a CS class, state is mostly kept by passing it around and using recursion. It's hard to explain if you don't know Haskell, however the other way is to use Monads. I can't explain them, because I don't understand them completly myself. Anyhoo, good luck.

weelvor 19 Jahren

If you want to learn a functional programming language that comes with a friendly on-line community, try Haskell. The people at the Haskell IRC channel and haskell-cafe mailing list will criticize you if you say something that just isn't right, but they are also relentlessly friendly to newcomers, they like to spend some time working on educational materials that explain their latest type system hacks, and there are always some of them around.

slobodanvor 19 Jahren

There's a nice group called a comp.lang.lisp also accessible through google groups http://groups.google.com/group/comp.lang.lisp/topics if you have any newbie question ask it there and mention that you're newbie. There's a lot of people who will answer, sometimes even in a less than minute.

Slobodan Blazeski

malkiavor 19 Jahren

Hi fireblade!

Yes comp.lang.lisp was very useful to me for the last year. Too bad the group is not available on gmane, and the full archive is only in google, but if you have an NNTP provider (www.motzarella.org for example) you can read it with your newsgroup reader, although you get only the messages from the last month.

Dimiter "malkia" Stanev.

machinevor 19 Jahren

I have to say, I'm not that sympathetic. If you are using open source software, you can't expect free support, and you definitely can't expect immediate free support. Plus, it actually looks like you got each of your questions answered within an hour, which is pretty impressive actually.

palishvor 19 Jahren

No problem. If you're using Google, you can't expect to get your results in 0.13 seconds. If you're playing a free game, you can't expect it to be fun. If you're in News.YC, you can't expect to interact with smart people, like yourself. If you use software, you can't expect things to work.

With that said, I wasn't expecting free support, nor immediate free support, and the answers I received were extremely helpful. There are wonderful, nice, and extraordinarily talented people on #lisp who are more intelligent than I could ever hope to become in my wildest dreams. However, there's a good deal of elitism, too. Statements like "There is no Lisp community, this is just an IRC channel" just reinforce that. You end up not being attached to the place you live in. You don't even feel like you live there; you just happen to be there. It's kind of like working at a company where you don't own any of the software you write (in the sense of, this is my module; if it breaks, I'm responsible, and its design choices are my call). If you think of your car as a hunk of metal, you're far less likely to oil its engine regularly.

I'm contrasting the Lisp experience with the Ruby and Python experience. In the Lisp world, things aren't obvious. It's not the programming language that's at fault, it's the interfaces. Things can make a great deal of sense when designed a certain way but require several steps and much thought. Or things can be simpler and require little thinking so people can focus their thoughts on what matters to them: creating the systems they want to create. Top that off with a community that might throw malicious code at a newbie or throw a patient response and you have a tough situation. It's the difference between a website's creators feeling they're doing users a favor by letting them visit versus feeling proud that they're making people happy.

Things don't need to be this way, and I'm confident that there will be changes for the better. Those wonderful, intelligent souls I referred to earlier will help turn the tide. Secretly, everyone wants to be perceived as a nice person, and building a community in which you can improve the life of just one person is a fabulous experience.

omousevor 19 Jahren

The point of open source software is to act as an alternative to the proprietary stuff out there. Therefore, you can and should expect free support. It's in the best interests of the project to have more people use it.

machinevor 19 Jahren

I understand that it is in the best interests of an open source developer to give support for his software (assuming he wants people to use it), but he's in no way obligated to do so. Some people just develop software for fun and make it available to the public as an afterthought in the hopes like minded people will find it useful. I know with the personal projects I've released as open source, although I fixed bugs and responded to requests when I could, it was my goal to give prompt technical support.

omousevor 19 Jahren

Some people just develop software for fun and make it available to the public as an afterthought in the hopes like minded people will find it useful.

That's true. But this is a programming language we're talking about. The goal is to create a good implementation and part of that is having good support. Does a programming language exist if no one uses it?