Thinking Forth(thinking-forth.sourceforge.net)
thinking-forth.sourceforge.net
Thinking Forth
http://thinking-forth.sourceforge.net/
9 comments
Here is a gem from this gem of a book - surely one of the best descriptions ever of what are now called DSLs:
Forth programming consists of extending the root language toward the application, providing new commands that can be used to describe the problem at hand. Programming languages designed especially for particular applications such as robotics, inventory control, statistics, etc., are known as “application-oriented languages.” Forth is a programming environment for creating application-oriented languages. (That last sentence may be the most succinct description of Forth that you’ll find.) In fact, you shouldn’t write any serious application in Forth; as a language it’s simply not powerful enough. What you should do is write your own language in Forth (lexicons) to model your understanding of the problem, in which you can elegantly describe its solution.
Forth programming consists of extending the root language toward the application, providing new commands that can be used to describe the problem at hand. Programming languages designed especially for particular applications such as robotics, inventory control, statistics, etc., are known as “application-oriented languages.” Forth is a programming environment for creating application-oriented languages. (That last sentence may be the most succinct description of Forth that you’ll find.) In fact, you shouldn’t write any serious application in Forth; as a language it’s simply not powerful enough. What you should do is write your own language in Forth (lexicons) to model your understanding of the problem, in which you can elegantly describe its solution.
I've heard similar descriptions of LISP, actually.
They are actually very much alike. Personally I prefer Forth for having less "magic"- a basic Forth kernel is much smaller and simpler than a bare-bones Lisp system. Postfix notation also seems a more elegant (if initially less familiar) solution to a homoiconic language than Lisp's nested parentheses.
Not entirely on-topic, but since you're our local Forth enthusiast: can you tell us anything about Moore's recent (say, last 10 years) chip and language designs? I look at the GreenArrays and ColorForth stuff and feel like Bob Dylan's Mr. Jones: there's something happening here and I don't know what it is.
My spidey sense tells me that it's disruptive enough to represent a major opportunity of some kind. But Spidey could be wrong.
My spidey sense tells me that it's disruptive enough to represent a major opportunity of some kind. But Spidey could be wrong.
Well, I can do my best to summarize what I understand.
Moore has always intended Forth as sort of a stone-soup language- a collection of ideas and techniques that can be taken from and recombined to suit different purposes and programmers. There were several efforts to standardize Forth and build it out into a language with formalized, extensive libraries (ANS-Forth, mainly), but they are largely antithetical to Moore's attitude of continual experimentation and refinement.
The current state of the art with ColorForth is pretty alien in comparison with ANS-Forth. It's an exercise in subtractive design. The most obvious change is that the ColorForth environment tokenizes input as it is typed. Tokens can have some metadata associated with them which is reflected in their color. ColorForth generally uses colors to eliminate the need for the "mode switching" words in a normal Forth- comments, [] "compile-time blocks" and colon definitions, for example. By pre-tokenizing source code, virtually all the string manipulation required in the compiler is removed and getting rid of mode-switching words makes source code more compact visually.
The F18 core and the GA4/GA144 chips based around it are incredible machines. In a nutshell, an F18 is a tiny Forth machine with 64 18-bit words of RAM which can be packed with up to four instructions apiece. The GA144 has 144 F18 cores which communicate with their neighbors via a blocking handshake mechanism. Thanks to both to Chuck's novel approach to chip layout and simulation and the fact that there is no global clock, the GA144 uses very little power compared to other CPUs and microcontrollers with comparable compute capacity, despite the fact that the GA144 uses essentially antique fab technology. With no changes to the design and the latest fabrication techniques you could (theoretically) get an order of magnitude more cores on the same sized die and drive power consumption down further.
Programming the beasts is another matter. Since each core has very limited space for code, programs tend to have the character of functional pipelines in which each core takes a few arguments from neighbors, possibly stores a small amount of local state and computes simple functions to pass down the chain. I think that if any language other than Forth could be compiled for the machines it might be an array processing language like APL. Even if the resource constraints weren't difficult, simply getting your program deployed onto the grid is tricky, since you can only directly talk to cores on the edges- you have to write code that "walks" through the grid of F18s, pushing code into neighbors and triggering them to do the same, establishing communication channels and then tearing them down as they are replaced by the final application code.
So where does this put us? GreenArrays has extremely energy-efficient, highly scalable supercomputers-on-a-chip, but programming them is very difficult and requires incredible attention to detail. There are a number of PhDs in the making solving the new problems in compiler technology this type of architecture raises. If GreenArrays was open to the idea (and they seem to be), a more portable software simulator of the GA144 and more accessible development tools (ie, tools that do not require booting a Windows VM and learning a new keyboard layout on top of the ColorForth language) could be a huge shot in the arm. If a killer-app emerges that can take advantage of the power consumption and the particular style of parallelism the chips are capable of, these designs could change everything, but the jury's still out.
...Does that help at all?
Moore has always intended Forth as sort of a stone-soup language- a collection of ideas and techniques that can be taken from and recombined to suit different purposes and programmers. There were several efforts to standardize Forth and build it out into a language with formalized, extensive libraries (ANS-Forth, mainly), but they are largely antithetical to Moore's attitude of continual experimentation and refinement.
The current state of the art with ColorForth is pretty alien in comparison with ANS-Forth. It's an exercise in subtractive design. The most obvious change is that the ColorForth environment tokenizes input as it is typed. Tokens can have some metadata associated with them which is reflected in their color. ColorForth generally uses colors to eliminate the need for the "mode switching" words in a normal Forth- comments, [] "compile-time blocks" and colon definitions, for example. By pre-tokenizing source code, virtually all the string manipulation required in the compiler is removed and getting rid of mode-switching words makes source code more compact visually.
The F18 core and the GA4/GA144 chips based around it are incredible machines. In a nutshell, an F18 is a tiny Forth machine with 64 18-bit words of RAM which can be packed with up to four instructions apiece. The GA144 has 144 F18 cores which communicate with their neighbors via a blocking handshake mechanism. Thanks to both to Chuck's novel approach to chip layout and simulation and the fact that there is no global clock, the GA144 uses very little power compared to other CPUs and microcontrollers with comparable compute capacity, despite the fact that the GA144 uses essentially antique fab technology. With no changes to the design and the latest fabrication techniques you could (theoretically) get an order of magnitude more cores on the same sized die and drive power consumption down further.
Programming the beasts is another matter. Since each core has very limited space for code, programs tend to have the character of functional pipelines in which each core takes a few arguments from neighbors, possibly stores a small amount of local state and computes simple functions to pass down the chain. I think that if any language other than Forth could be compiled for the machines it might be an array processing language like APL. Even if the resource constraints weren't difficult, simply getting your program deployed onto the grid is tricky, since you can only directly talk to cores on the edges- you have to write code that "walks" through the grid of F18s, pushing code into neighbors and triggering them to do the same, establishing communication channels and then tearing them down as they are replaced by the final application code.
So where does this put us? GreenArrays has extremely energy-efficient, highly scalable supercomputers-on-a-chip, but programming them is very difficult and requires incredible attention to detail. There are a number of PhDs in the making solving the new problems in compiler technology this type of architecture raises. If GreenArrays was open to the idea (and they seem to be), a more portable software simulator of the GA144 and more accessible development tools (ie, tools that do not require booting a Windows VM and learning a new keyboard layout on top of the ColorForth language) could be a huge shot in the arm. If a killer-app emerges that can take advantage of the power consumption and the particular style of parallelism the chips are capable of, these designs could change everything, but the jury's still out.
...Does that help at all?
Thank you, I enjoyed reading that very much. Actually, I enjoy reading all your Forth comments very much, so please write lots more. A few points that struck me:
getting rid of mode-switching words makes source code more compact
That's a big deal, given that source code size is the best measure of complexity we have and the best predictor of errors we have.
the GA144 uses essentially antique fab technology. With no changes to the design and the latest fabrication techniques you could (theoretically) get an order of magnitude more cores
A smart approach. Prove the thing useful first, with the knowledge that there's another order of magnitude waiting to be had.
programming them is very difficult and requires incredible attention to detail
Do we know any programs that have been written for them? What does Chuck write? Do they provide a foundation for building application-oriented languages in the classic Forth style?
I think that if any language other than Forth could be compiled for the machines it might be an array processing language like APL.
What I really want to know is whether this architecture would be suitable for dataflow, i.e. whether you could build a spreadsheet on top of it. It seems not unlike the systolic array architectures that were a hot research topic 25 years ago, and those were thought to be suitable for dataflow.
I work on the problem of data parallelism in spreadsheets, and the thought of being able to apply hardware like this to it makes me salivate. Okay, not just salivate, drool. But we're probably years away from being able to figure out how to do that. Capturing even the low-hanging data parallelism in spreadsheets with any generality has proven surprisingly difficult, and I don't have a good gut feeling for whether the software techniques we've developed to do it would translate naturally to the GA chips. If they could, it would be a dream come true, because we could first prove the software on traditional hardware and then "go native" later.
Obviously you couldn't just port already-written code to a platform that radically different. The question is whether the key concepts would survive or whether you'd have to rethink everything from scratch. The environment we're building is highly reactive. It doesn't just compile code and then run it; if the computations produce new formulae, those have to go back to the top level, get analyzed again, and then be computed in turn. That sort of re-entrancy turns out to be a big deal for spreadsheets, and most of the important constructs depend on it. I don't know if the GA chips would feel very good about that kind of dynamism; perhaps it would be more like "Damn, you just got all the code laid out and now you're shoving us all around again."
getting rid of mode-switching words makes source code more compact
That's a big deal, given that source code size is the best measure of complexity we have and the best predictor of errors we have.
the GA144 uses essentially antique fab technology. With no changes to the design and the latest fabrication techniques you could (theoretically) get an order of magnitude more cores
A smart approach. Prove the thing useful first, with the knowledge that there's another order of magnitude waiting to be had.
programming them is very difficult and requires incredible attention to detail
Do we know any programs that have been written for them? What does Chuck write? Do they provide a foundation for building application-oriented languages in the classic Forth style?
I think that if any language other than Forth could be compiled for the machines it might be an array processing language like APL.
What I really want to know is whether this architecture would be suitable for dataflow, i.e. whether you could build a spreadsheet on top of it. It seems not unlike the systolic array architectures that were a hot research topic 25 years ago, and those were thought to be suitable for dataflow.
I work on the problem of data parallelism in spreadsheets, and the thought of being able to apply hardware like this to it makes me salivate. Okay, not just salivate, drool. But we're probably years away from being able to figure out how to do that. Capturing even the low-hanging data parallelism in spreadsheets with any generality has proven surprisingly difficult, and I don't have a good gut feeling for whether the software techniques we've developed to do it would translate naturally to the GA chips. If they could, it would be a dream come true, because we could first prove the software on traditional hardware and then "go native" later.
Obviously you couldn't just port already-written code to a platform that radically different. The question is whether the key concepts would survive or whether you'd have to rethink everything from scratch. The environment we're building is highly reactive. It doesn't just compile code and then run it; if the computations produce new formulae, those have to go back to the top level, get analyzed again, and then be computed in turn. That sort of re-entrancy turns out to be a big deal for spreadsheets, and most of the important constructs depend on it. I don't know if the GA chips would feel very good about that kind of dynamism; perhaps it would be more like "Damn, you just got all the code laid out and now you're shoving us all around again."
I enjoy reading all your Forth comments very much, so please write lots more.
My pleasure. :)
Do we know any programs that have been written for them? What does Chuck write?
The largest available program listing for the GA144 that I'm aware of is an MD5 hasher: http://www.greenarraychips.com/home/documents/pub/AP001-MD5....
All I know of Chuck's current projects comes from his blog and the fireside chats he gives at the Silicon Valley Forth Interest Group's annual "Forth Day". They can be found here, along with many other fun presentations: http://www.forth.org/svfig/videos/index.html
It seems he's written some temperature-monitoring and signal processing code, a number of routines for generating VGA video signals and he has begun work on building a ColorForth compiler that can run on the GA144.
Do they provide a foundation for building application-oriented languages in the classic Forth style?
Based on my observations and my own experience tinkering with a softsim for a single F18, the resource constraints of the processing elements make it very difficult to write genuinely reusable code, beyond discovering idiomatic opcode combinations and design approaches. The only way to make your code fit is to code like Chuck- simplify brutally. It would be interesting to see how well an optimizing compiler could take more relaxed, modular Forth (or another language) and squeeze it onto an F18.
What I really want to know is whether this architecture would be suitable for dataflow, i.e. whether you could build a spreadsheet on top of it. It seems not unlike the systolic array architectures that were a hot research topic 25 years ago, and those were thought to be suitable for dataflow.
I concur with your comparison to systolic arrays- the GA144 is more flexible than most of those old designs, but has a very similar character. I think calculating something like a spreadsheet on a grid of F18s could work well, but would need a much larger array of processing elements to be effective. Interconnect is also an issue. The parallelism that can be extracted by dataflow architectures is often a product of the rate at which calculations can fan out, and a grid-based architecture enforces a strict cap. As I'm sure you are well aware, spreadsheet formulae frequently gather input from a large number of other cells and are in turn used in many places, often with only a tenuous relationship to the orthogonal neighbors of a cell.
Your discussion of re-entrancy in spreadsheets raises some interesting ideas. F18 cores in a GA144 are designed to very easily push code to their neighbors to execute- it's the basis of deploying code through the grid. The same capabilities mean that you can write code that dynamically reprograms various parts of the array at runtime. Extensive use of self-modifying code is widely regarded as deep magic nowadays, but I've often imagined programming for a huge FPGA where an OS can assemble new purpose-designed cores on the fly instead of spinning up new processes- these chips could be used in a similar way. Like I said, there's a great deal of new CS theory lurking in this design space.
My pleasure. :)
Do we know any programs that have been written for them? What does Chuck write?
The largest available program listing for the GA144 that I'm aware of is an MD5 hasher: http://www.greenarraychips.com/home/documents/pub/AP001-MD5....
All I know of Chuck's current projects comes from his blog and the fireside chats he gives at the Silicon Valley Forth Interest Group's annual "Forth Day". They can be found here, along with many other fun presentations: http://www.forth.org/svfig/videos/index.html
It seems he's written some temperature-monitoring and signal processing code, a number of routines for generating VGA video signals and he has begun work on building a ColorForth compiler that can run on the GA144.
Do they provide a foundation for building application-oriented languages in the classic Forth style?
Based on my observations and my own experience tinkering with a softsim for a single F18, the resource constraints of the processing elements make it very difficult to write genuinely reusable code, beyond discovering idiomatic opcode combinations and design approaches. The only way to make your code fit is to code like Chuck- simplify brutally. It would be interesting to see how well an optimizing compiler could take more relaxed, modular Forth (or another language) and squeeze it onto an F18.
What I really want to know is whether this architecture would be suitable for dataflow, i.e. whether you could build a spreadsheet on top of it. It seems not unlike the systolic array architectures that were a hot research topic 25 years ago, and those were thought to be suitable for dataflow.
I concur with your comparison to systolic arrays- the GA144 is more flexible than most of those old designs, but has a very similar character. I think calculating something like a spreadsheet on a grid of F18s could work well, but would need a much larger array of processing elements to be effective. Interconnect is also an issue. The parallelism that can be extracted by dataflow architectures is often a product of the rate at which calculations can fan out, and a grid-based architecture enforces a strict cap. As I'm sure you are well aware, spreadsheet formulae frequently gather input from a large number of other cells and are in turn used in many places, often with only a tenuous relationship to the orthogonal neighbors of a cell.
Your discussion of re-entrancy in spreadsheets raises some interesting ideas. F18 cores in a GA144 are designed to very easily push code to their neighbors to execute- it's the basis of deploying code through the grid. The same capabilities mean that you can write code that dynamically reprograms various parts of the array at runtime. Extensive use of self-modifying code is widely regarded as deep magic nowadays, but I've often imagined programming for a huge FPGA where an OS can assemble new purpose-designed cores on the fly instead of spinning up new processes- these chips could be used in a similar way. Like I said, there's a great deal of new CS theory lurking in this design space.
I've heard similar descriptions of LISP, actually.
Actually, Paul Graham said something similar about Lisp.
"...In Lisp, you don't just write your program down toward the language, you also build the language up toward your program..."
http://www.paulgraham.com/progbot.html
Actually, Paul Graham said something similar about Lisp.
"...In Lisp, you don't just write your program down toward the language, you also build the language up toward your program..."
http://www.paulgraham.com/progbot.html
Already mentioned on this site, but still good to advertise from time to time.
http://www.hnsearch.com/search#request/all&q=thinking+fo...
checkout also http://factorcode.org/ a forth-like system with batteries. (already mentioned here too)
http://www.hnsearch.com/search#request/all&q=thinking+fo...
checkout also http://factorcode.org/ a forth-like system with batteries. (already mentioned here too)
Just to add to your list:
Ben Stiglitz (Terminal.app, Apple employee) at RubyConf 2008 introducing Forth: http://rubyconf2008.confreaks.com/forth.html
I also recommend checking out Factor. It grabs ideas from EVERYWHERE (Forth, Common Lisp, Smalltalk, Joy, etc.) and wraps them all up into a nice package.
Ben Stiglitz (Terminal.app, Apple employee) at RubyConf 2008 introducing Forth: http://rubyconf2008.confreaks.com/forth.html
I also recommend checking out Factor. It grabs ideas from EVERYWHERE (Forth, Common Lisp, Smalltalk, Joy, etc.) and wraps them all up into a nice package.
... and jonesforth (google it 'coz the original site is down)
web archive digging -> http://web.archive.org/web/20100116031518/http://www.annexia...
The author seems creative and integrative, the included forth REPL/IDE also is neat. Another benevolent dictator in the making I guess.
While Forth is certainly a mind-expanding experience, I still feel that it suffers a bit from "Stackholm syndrome" when I see advice like that contained on page 144, under the section "An alternative to screens: source in named files":
Infnite-length files allow sloppy, disorganized thinking and bad factoring. Defnitions become longer without the discipline imposed by the 1K block boundaries. The tendency becomes to write a 20K file, or worse: a 20K definition.Or perhaps this advice is just ahead of its time or beyond the mainstream or however you want to put it. The idea of accepting any constraint on the amount of code we churn out is alien to us. We take for granted that complex systems must grow arbitrarily large. What if we're wrong?
I've long been a big fan of this book. It will make you a better programmer even if you don't program in Forth.
The only downside is that it will feel dated (because it is). Many of the ideas that were once progressive, original, and a bit revolutionary now seem obvious and a little trite.
The only downside is that it will feel dated (because it is). Many of the ideas that were once progressive, original, and a bit revolutionary now seem obvious and a little trite.
Forth is a beautiful language. Just like Lisp, Smalltalk, and others have a comeback, I won't be surprised if the same happens soon with Forth as well.
One of the best books ever!
An eye opener next to 'a little Smalltalk'
An eye opener next to 'a little Smalltalk'
This really takes me back to my RPL days on my old HP-48G.
typo on page 3: "Move the contents or Register B to Register A,"
s/r/f/ that string
s/r/f/ that string
If I had an opportunity to teach an introductory software engineering course, I would strongly consider using Thinking Forth as a textbook.