The Emacs Problem (February, 2005)(steve.yegge.googlepages.com)
steve.yegge.googlepages.com
The Emacs Problem (February, 2005)
http://steve.yegge.googlepages.com/the-emacs-problem
4 comments
In general I like this essay, but I do see how it left the impression that Lisp is about making all your data into executable programs. Most data would be read and written with the same functions that read and write programs, but the data would not be evaluated as a program. Among his examples, only configuration files seem suitable for that.
That being said, even if you did make log files into programs, there's never an issue of escaping text properly. The function that writes data always escapes strings so they'll read back in as the same string, not as any kind of structure.
That being said, even if you did make log files into programs, there's never an issue of escaping text properly. The function that writes data always escapes strings so they'll read back in as the same string, not as any kind of structure.
From the article:
"But Lisp is directly executable, so you could simply make the tag names functions that automatically transform themselves."
I don't think we're talking about taking raw strings and doing evals. You control the function implementations so you have control over any danger. Writing the data as a Lisp program saves you the overhead of writing a parser and lexer and all the other tedium associated with building a compiler.
Anyway, languages that can be interpreted really illustrate that it's a blurry line between code and data. Consider Actionscript. It's a scripting language interpreted by Flash, but it's nothing but XML. You could consider it marked up data describing a specific Flash application rather than code. When the interpreter sees the code, it knows how to use the description to generate the app on the fly.
The thing that's really nice about Lisp in this respect is that it makes defining your own domain-specific language (and Actionscript is really a DSL for flash apps) about as easy as it can be. The only downside - if it is a downside at all - is that you will start creating DSL's for every app you build. Maybe that's as it should be. I think so.
"But Lisp is directly executable, so you could simply make the tag names functions that automatically transform themselves."
I don't think we're talking about taking raw strings and doing evals. You control the function implementations so you have control over any danger. Writing the data as a Lisp program saves you the overhead of writing a parser and lexer and all the other tedium associated with building a compiler.
Anyway, languages that can be interpreted really illustrate that it's a blurry line between code and data. Consider Actionscript. It's a scripting language interpreted by Flash, but it's nothing but XML. You could consider it marked up data describing a specific Flash application rather than code. When the interpreter sees the code, it knows how to use the description to generate the app on the fly.
The thing that's really nice about Lisp in this respect is that it makes defining your own domain-specific language (and Actionscript is really a DSL for flash apps) about as easy as it can be. The only downside - if it is a downside at all - is that you will start creating DSL's for every app you build. Maybe that's as it should be. I think so.
I'm not disagreeing with you, but I think it's important not to conflate the concepts of Lisp's ease of reading/writing arbitrary data and Lisp's ability to read/write programs using the same facilities.
Using Lisp to read and write a configuration file means you don't need to worry about lexing and parsing either. If you don't use Lisp, or if you don't use S-expressions, you will need to lex and parse the configuration file just to read it, compiler or no compiler.
Depending what domain you program in, a Lisp programmer may not ever need to think about lexing and parsing. I tried a little googling to make sure I was remembering their definitions right and actually didn't come up with anything. From memory, lexing is splitting the input stream up into tokens, and parsing is structuring those tokens into a parse tree. If I'm remembering wrong, re-read the prior paragraph temporarily using those definitions and it will make sense.
Using Lisp to read and write a configuration file means you don't need to worry about lexing and parsing either. If you don't use Lisp, or if you don't use S-expressions, you will need to lex and parse the configuration file just to read it, compiler or no compiler.
Depending what domain you program in, a Lisp programmer may not ever need to think about lexing and parsing. I tried a little googling to make sure I was remembering their definitions right and actually didn't come up with anything. From memory, lexing is splitting the input stream up into tokens, and parsing is structuring those tokens into a parse tree. If I'm remembering wrong, re-read the prior paragraph temporarily using those definitions and it will make sense.
>Using Lisp to read and write a configuration file means you don't need to worry about lexing and parsing either.
That was the point I was trying to make, apparently unsuccessfully.
>If you don't use Lisp, or if you don't use S-expressions, you will need to lex and parse the configuration file just to read it, compiler or no compiler.
Any time you build a lexer and a parser, you are building a compiler (I'm using a broader meaning of compiler - not necessarily one that outputs bytecode). There are many XML-to-HTML compilers, for example.
Lisp macros compile DSL's to Lisp. That's why a Lisp programmer rarely needs to think about lexing and parsing. It is automatic and implied if you address the problem in the right way. It's also why the concepts of data and programs are necessarily conflated in Lisp.
That was the point I was trying to make, apparently unsuccessfully.
>If you don't use Lisp, or if you don't use S-expressions, you will need to lex and parse the configuration file just to read it, compiler or no compiler.
Any time you build a lexer and a parser, you are building a compiler (I'm using a broader meaning of compiler - not necessarily one that outputs bytecode). There are many XML-to-HTML compilers, for example.
Lisp macros compile DSL's to Lisp. That's why a Lisp programmer rarely needs to think about lexing and parsing. It is automatic and implied if you address the problem in the right way. It's also why the concepts of data and programs are necessarily conflated in Lisp.
That is a valid concern, however it doesn't mean you have to worry about escaping all text properly. You can simply handle the elements as lists, and code won't get executed.
As an exercise, I tried parsing a few entries in that java log format from the article, but changed a few things. In particular, I replaced the message string with one written with unescaped text that the user might have been responsible for.
I suck at Lisp, but in 28 lines (6 defuns and 1 defparameter) here's what I got for output. Note that (FORMAT T "injection") gets printed directly is not executed.
As an exercise, I tried parsing a few entries in that java log format from the article, but changed a few things. In particular, I replaced the message string with one written with unescaped text that the user might have been responsible for.
I suck at Lisp, but in 28 lines (6 defuns and 1 defparameter) here's what I got for output. Note that (FORMAT T "injection") gets printed directly is not executed.
Entry:("2005-02-21T18:57:39")('SEVERE)("A pretty bad thing has happened! The user did this: " (FORMAT T "injection") ".")
Entry:("2007-09-11T18:57:39")('SEVERE)("A very very bad thing has happened! The user did this: ...")
Entry:("2907-09-11T18:57:39")('WARNING)("A sorta bad thing has happened! The user did this: ...")
Here's the code, with "jlog" instead of log. I must have log defined somewhere (like I said, I suck at Lisp).
(defparameter *handlethese*
'(date level message))
(defun date (obj)
(prin1 obj))
(defun level (obj)
(prin1 obj))
(defun message (obj)
(prin1 obj))
(defun record (record-data)
(if (car record-data)
(progn
(if (member (car (car record-data)) *handlethese*)
(funcall (car (car record-data)) (cdr (car record-data)))
nil)
(record (cdr record-data)))
nil))
(defun process-jlog(record-list)
(if (car record-list)
(progn (format t "~%Entry:")
(funcall (car (car record-list)) (cdr (car record-list)))
(process-jlog (cdr record-list)))
t))
(defun jlog (&rest record-list)
(process-jlog record-list))
In the meantime I made up my mind: presumably the only way the text would be written would be through some kind of serializer for LISP, so it would be easy to ensure that everything is escaped properly.
A slight uneasy feeling still remains, but I guess it could be overcome...
A slight uneasy feeling still remains, but I guess it could be overcome...
Well, you probably would have to watch out if you are EVALuating some user input... other than that i don't understand your doubt.
Lovely dig. Greenspun's_Tenth_Rule is also encompassed as a point to top it all off :)
I love it.
Also, there is something like XPath for Java classes, I think it is called JXPath, but I am not sure. I guess one could also use the query language from Groovy as an alternative.