Lightening run-time code generation(wingolog.org)
wingolog.org
Lightening run-time code generation
https://wingolog.org/archives/2019/05/24/lightening-run-time-code-generation
18 comments
What I love about the guile optimizer (and some lisp optimizers in general) is that you usually have a first step of source to source optimization. That means I can write some code and see what gets inined, hoisted, unrolled and removed without having to read the bytecode assembly.
That is especially nice when writing complex macros. I remember writing a loop utility and realized I couldn't run the optimizer on code where everything was known at compile time, since guile usually optimized the loop away. (for/sum ((i (in-range 5)) i) expanded into 20 lines of code, but optimized to the integer 15 :)
That is especially nice when writing complex macros. I remember writing a loop utility and realized I couldn't run the optimizer on code where everything was known at compile time, since guile usually optimized the loop away. (for/sum ((i (in-range 5)) i) expanded into 20 lines of code, but optimized to the integer 15 :)
The tricky think about a source-to-source transformation, is where do you store the meta-data to map the new source back to the old source, for the purposes of error reporting, debugging, and deoptimisation? You can't just map from one list to another as your optimisation function.
I think it is the same system as used when doing macro expansion. Macros use syntax objects with embedded metadata.
Both chez and guile usually produces proper error sites, so it is done in some fashion.
Both chez and guile usually produces proper error sites, so it is done in some fashion.
> I wonder why Andy didn't chose to write the JIT in Guile itself, with a binding to Lightening, rather than in C. Guile is designed to be embedded and have bindings written for it.
OP states "eventually I should be migrating this code over to Scheme anyway. And, of course, simple can mean fast, and I needed fast code generation.". That does not anwer you actual question (and neither does the article), but I gather this was to make the project finite (solve one and only one problem only at a time), not because the author was not aware.
OP states "eventually I should be migrating this code over to Scheme anyway. And, of course, simple can mean fast, and I needed fast code generation.". That does not anwer you actual question (and neither does the article), but I gather this was to make the project finite (solve one and only one problem only at a time), not because the author was not aware.
Yes that’s the statement I’m responding to “I didn't want to add more complexity to the C side of Guile“ but this is more C code. Maybe speed of generation was more important than anything else then.
It makes me obscurely happy to see Guile evolving to remain useful, despite my misgivings about obligate-GC languages in general.
despite my misgivings about obligate-GC languages in general
In terms of performance, you have a point. However, there are many ways to set the performance dial, depending on your particular cost/benefit situation.
In terms of performance, you have a point. However, there are many ways to set the performance dial, depending on your particular cost/benefit situation.
Am building Guile 2.9.2 right now, to try it out.
When Racket (nee PLT Scheme) added JIT on x86, via GNU Lightning, there was a very noticeable practical performance bump for some of my work (dynamic Web serving).
When Racket (nee PLT Scheme) added JIT on x86, via GNU Lightning, there was a very noticeable practical performance bump for some of my work (dynamic Web serving).
For 2.9.1 I saw a performance improvement of between 1.5 and 4x across all workloads. It is still not as fast as racket, but iirc Andy said there are some low hanging fruit left.
The peg parser and my implementatio of clojure's transducers get the biggest performance improvements. My simple CSV parser got over 3x faster without any changes on my part.
The peg parser and my implementatio of clojure's transducers get the biggest performance improvements. My simple CSV parser got over 3x faster without any changes on my part.
Hi! I'm struggling to lean peg in guile - care to share csv-parsing code please?
Sure. Ill put it online whenever I have access to my computer next. Are you in #guile on freenode? Ivan ping you there when it's up.
It is slow as molasses though. A 5mb CSV file takes roughly 12s on 2.2. A properly implemented streaming parser should be several orders of magnitude faster. I want to see, just for shits and giggles if it is possible to beat pythons CSV parser with a pure guile module. I think it is. With guile 2.9 it should be a walk in the park.
It is slow as molasses though. A 5mb CSV file takes roughly 12s on 2.2. A properly implemented streaming parser should be several orders of magnitude faster. I want to see, just for shits and giggles if it is possible to beat pythons CSV parser with a pure guile module. I think it is. With guile 2.9 it should be a walk in the park.
Sorry for delay - will raise hand on #guile when online next
I have a CSV parser that was originally portable R5RS+SRFI, and probably still is given minor editing. I don't know how the performance compares. (Paying work has had me focused on Racket, so I stopped doing portable packagings, but I might be going back to releasing portable packagings in R7RS.) https://www.neilvandyke.org/racket/csv-reading/
"It's written in C and not C++" hope to stay this way. I really like guile. My problem is that I am on windows.
I wonder why Andy didn't chose to write the JIT in Guile itself, with a binding to Lightening, rather than in C. Guile is designed to be embedded and have bindings written for it.