Why have Visual Programming Languages not gone mainstream?
14 comments
1. Diffs: There is no easy way to diff two versions. You need to diff the visual changes (what it looks like) and the semantic changes (what it means). How do you visually display the changes in a useful way? IE, what do you put into github?
2. Spaghetti code: Read the Dijkstra's paper on Goto, and mentally replace 'goto' with 'line'.
3. Text labels: Many graphical languages have lots of text labels in the display.
4. Limits: There is a human visual limit of about 50 items in a graphical picture, unless there is a underlying graphical semantic context such as geography or time series.
5. Too Big: Once the diagram is larger that one screen, navigation and comprehension become exponentially more complicated.
6. Graphical API: Not clear what is the graphical equivalent to modules, namespaces, packages, etc. How would you package reusable code and make an API available?
During the 1970's, I spent years working on CAD tools, including logic schematic entry and IC layout programs. This was using graphical tablets in the pre mouse era. Even simple 8 bit microprocessors were too complex to draw schematics on a display, and barely possible to layout by hand on a display. I note that today logic design is using text languages such as Veralog, and layout is mostly automated.
2. Spaghetti code: Read the Dijkstra's paper on Goto, and mentally replace 'goto' with 'line'.
3. Text labels: Many graphical languages have lots of text labels in the display.
4. Limits: There is a human visual limit of about 50 items in a graphical picture, unless there is a underlying graphical semantic context such as geography or time series.
5. Too Big: Once the diagram is larger that one screen, navigation and comprehension become exponentially more complicated.
6. Graphical API: Not clear what is the graphical equivalent to modules, namespaces, packages, etc. How would you package reusable code and make an API available?
During the 1970's, I spent years working on CAD tools, including logic schematic entry and IC layout programs. This was using graphical tablets in the pre mouse era. Even simple 8 bit microprocessors were too complex to draw schematics on a display, and barely possible to layout by hand on a display. I note that today logic design is using text languages such as Veralog, and layout is mostly automated.
> 1. Diffs: There is no easy way to diff two versions.
Any visual programming language can be serialized to and deserialized from a linear format, which can easily be subject to diff.
It should be fairly straightforward from that to also build a visual interpretation of the diff of the serialized forms (e.g., an animation of the transformation from the original to the new form.)
Visual vs. semantic changes is only an issue if the language admits multiple visual representations that are structurally equivalent rather than (e.g.) accepting different inputs and reducing them to a canonical representation. But even if you allow that, you should be able to use canonicalization of that type "behind the scenes" to generate diffs which are comprehensive or which only identify semantically-significant changes.
> 2. Spaghetti code: Read the Dijkstra's paper on Goto, and mentally replace 'goto' with 'line'.
As in text language, using block structure rather than arbitrary gotos addresses this. There is no reason a visual language can't be block structured. (If you are concerned about procedure/function-calls-as-lines rather than gotos-as-lines, there's no reason that calls need to be lines; callables can be shapes, and calls can simply be inclusion of the shape in the procedure being drawn.)
> 4. Limits: There is a human visual limit of about 50 items in a graphical picture, unless there is a underlying graphical semantic context such as geography or time series.
There are similar limits in other domains, which is one of the reasons for structured decomposition of code in general. I don't see why this would apply less to visual code.
> 6. Graphical API: Not clear what is the graphical equivalent to modules, namespaces, packages, etc. How would you package reusable code and make an API available?
In general, names are shapes -- I don't see why that would be any different for names of namespaces vs. names of individual items within a namespace. Sure, how you depict either "opening" a namespace or accessing a name from it in a qualified way would be something a graphical language with namespaces would have to decide.
Any visual programming language can be serialized to and deserialized from a linear format, which can easily be subject to diff.
It should be fairly straightforward from that to also build a visual interpretation of the diff of the serialized forms (e.g., an animation of the transformation from the original to the new form.)
Visual vs. semantic changes is only an issue if the language admits multiple visual representations that are structurally equivalent rather than (e.g.) accepting different inputs and reducing them to a canonical representation. But even if you allow that, you should be able to use canonicalization of that type "behind the scenes" to generate diffs which are comprehensive or which only identify semantically-significant changes.
> 2. Spaghetti code: Read the Dijkstra's paper on Goto, and mentally replace 'goto' with 'line'.
As in text language, using block structure rather than arbitrary gotos addresses this. There is no reason a visual language can't be block structured. (If you are concerned about procedure/function-calls-as-lines rather than gotos-as-lines, there's no reason that calls need to be lines; callables can be shapes, and calls can simply be inclusion of the shape in the procedure being drawn.)
> 4. Limits: There is a human visual limit of about 50 items in a graphical picture, unless there is a underlying graphical semantic context such as geography or time series.
There are similar limits in other domains, which is one of the reasons for structured decomposition of code in general. I don't see why this would apply less to visual code.
> 6. Graphical API: Not clear what is the graphical equivalent to modules, namespaces, packages, etc. How would you package reusable code and make an API available?
In general, names are shapes -- I don't see why that would be any different for names of namespaces vs. names of individual items within a namespace. Sure, how you depict either "opening" a namespace or accessing a name from it in a qualified way would be something a graphical language with namespaces would have to decide.
They're popular in some domains. Aside from the ones already mentioned in this thread, there's Softimage ICE which I understand lives on in other forms in 3D modeling tools.
I worked on a visual dataflow language for complex ETL between databases some time ago. My experience with that was I would rather write Scheme directly (which is basically what the pretty directed graphs turned into). Mostly out of familiarity, but I did find the textual representation was denser and so faster to work with once familiar with it; maybe it's all about having the right level of abstraction for the task at hand.
I suspect a successful general purpose visual programming language would have to have some really powerful abstractions for parallelism and concurrency to take off in place of languages that already exist. You also have to remember the problems people have had integrating Smalltalk's image-based persistence with non-Smalltalk parts of a system, and avoid that kind of insularity.
I worked on a visual dataflow language for complex ETL between databases some time ago. My experience with that was I would rather write Scheme directly (which is basically what the pretty directed graphs turned into). Mostly out of familiarity, but I did find the textual representation was denser and so faster to work with once familiar with it; maybe it's all about having the right level of abstraction for the task at hand.
I suspect a successful general purpose visual programming language would have to have some really powerful abstractions for parallelism and concurrency to take off in place of languages that already exist. You also have to remember the problems people have had integrating Smalltalk's image-based persistence with non-Smalltalk parts of a system, and avoid that kind of insularity.
Have you looked at non-trivial simulink models? The graph of connections between logical programming elements is too high to nicely represent in a single two dimensional plane. Once you render all the required connections, the gain of a clean, easy-to-follow visual schematic for the logic is lost.
There are ways to carefully clean it up and keep it logical - but once you get into all of that, you might as well have been working in text.
Look at Printed Circuit Boards (PCBs) for hardware. For complex hardware logic, you often end up with multi-layer boards. The boards carry higher complexity and cost, but designers use them because two-dimensions is simply not enough for the interconnects.
There are ways to carefully clean it up and keep it logical - but once you get into all of that, you might as well have been working in text.
Look at Printed Circuit Boards (PCBs) for hardware. For complex hardware logic, you often end up with multi-layer boards. The boards carry higher complexity and cost, but designers use them because two-dimensions is simply not enough for the interconnects.
I would use a visual programming language in a heartbeat if it provided a first-rate environment for solving problems that I care about.
Here are some challenges that need to be overcome: 1/ I agree that the inertia of history is against visual systems. There's a lot of catch-up required to reach parity. 2/ People get comfortable and are often reluctant to adopt something new, especially when it's a big change. 3/ Text files provide a simple program representation, a format that's standard across languages. I don't know of a comparable data format that might work across visual languages. E.g., we need something that supports good diffs in a language-agnostic way. 4/ There's perception that a visual language needs to be programmed with a mouse, and that it would cater to beginners but not experts. I see no reason why I shouldn't manipulate visual elements with keyboard shortcuts and will use anything that helps me get the job done better. 5/ There's a perception that programming in a visual language will be slower, but I don't think that need be the case.
When I write software using a powerful IDE I often feel pretty far removed from the experience of writing a text file. There's a sensation of taking abstractions out of my head and putting them into the computer, with the text-file being a rather ancillary element in the programming / runtime environment as a whole. My feeling is that the text file has come to be limiting and I'd love to see other formats emerge.
Here are some challenges that need to be overcome: 1/ I agree that the inertia of history is against visual systems. There's a lot of catch-up required to reach parity. 2/ People get comfortable and are often reluctant to adopt something new, especially when it's a big change. 3/ Text files provide a simple program representation, a format that's standard across languages. I don't know of a comparable data format that might work across visual languages. E.g., we need something that supports good diffs in a language-agnostic way. 4/ There's perception that a visual language needs to be programmed with a mouse, and that it would cater to beginners but not experts. I see no reason why I shouldn't manipulate visual elements with keyboard shortcuts and will use anything that helps me get the job done better. 5/ There's a perception that programming in a visual language will be slower, but I don't think that need be the case.
When I write software using a powerful IDE I often feel pretty far removed from the experience of writing a text file. There's a sensation of taking abstractions out of my head and putting them into the computer, with the text-file being a rather ancillary element in the programming / runtime environment as a whole. My feeling is that the text file has come to be limiting and I'd love to see other formats emerge.
My impression is that they try to re-create the text language programming experience, in part because that's what we know, in part because they access the same underlying machine as text languages.
When graphic programming languages provide an entirely new and better way of making what we today call a computer do what we want, then they'll catch on. In the current environment, I don't think visual languages can compete with the expressiveness and flexibility of text.
Plus I'd probably suffer from shoulder RSI injuries from pushing the mouse around.
When graphic programming languages provide an entirely new and better way of making what we today call a computer do what we want, then they'll catch on. In the current environment, I don't think visual languages can compete with the expressiveness and flexibility of text.
Plus I'd probably suffer from shoulder RSI injuries from pushing the mouse around.
Some "visal programming languages" are actually faster to develop, specifically where there are prebuilt components and the real ability to reuse existing work.
Another great advantage is for business people and BAs to be able to easily see and understand how the application works, and even contribute at a meaningful level to what business needs.It's easier for new people on a project to pick up an understand what has been written, rather than getting their head around thousands of lines of code.
While technically not a visual programming language, we're developing an ESB that works inside a visual IDE and uses a lot of flow based and visual programming principles to make it easy to learn and simple to use. It's an open source project, go check it out on GitHub - https://github.com/Warewolf-ESB/Warewolf-ESB or get a compiled version from http://warewolf.io
Of the three advantages listed, only the third has much weight because it's hard to see a pictographic language having simpler syntax than Lisp and the most useful software is rarely written by beginners because it usually takes time to develop skill of conjuring up useful abstractions.
Looking at the third advantage, finding logic errors, it is still a bit of a stretch since many pure logic errors can be caught by compilers and tests and the typical pernicious logical errors are related to modeling the domain rather than endless loops or dead code paths.
Finally, not only are words are a really compact way of communicating but the Turing Machine model of computation is based upon alphabets and input strings and thus text is closer to the metal than pictures.
That said, Drakon is an interesting language developed for the Russian space program that leverages graphics in conjunction with text.
http://sage.com.ua/en.shtml?e6l0
Looking at the third advantage, finding logic errors, it is still a bit of a stretch since many pure logic errors can be caught by compilers and tests and the typical pernicious logical errors are related to modeling the domain rather than endless loops or dead code paths.
Finally, not only are words are a really compact way of communicating but the Turing Machine model of computation is based upon alphabets and input strings and thus text is closer to the metal than pictures.
That said, Drakon is an interesting language developed for the Russian space program that leverages graphics in conjunction with text.
http://sage.com.ua/en.shtml?e6l0
> I have never heard of one gaining mainstrean acceptance.
Check out Max/MSP, Pure Data, and Apple's Quartz Composer.
https://en.wikipedia.org/wiki/Max_(software)
https://en.wikipedia.org/wiki/Pure_Data
https://en.wikipedia.org/wiki/Quartz_Composer
https://en.wikipedia.org/wiki/Visual_programming_language
I'd argue that some of the Microsoft stuff involving XAML and Blend and database tools could be considered visual programming, but I can see the other side of the argument.
Check out Max/MSP, Pure Data, and Apple's Quartz Composer.
https://en.wikipedia.org/wiki/Max_(software)
https://en.wikipedia.org/wiki/Pure_Data
https://en.wikipedia.org/wiki/Quartz_Composer
https://en.wikipedia.org/wiki/Visual_programming_language
I'd argue that some of the Microsoft stuff involving XAML and Blend and database tools could be considered visual programming, but I can see the other side of the argument.
Visual programming languages (VPL) attempt to reproduce the low level constructs of general purpose programming languages and thus the complexity of the diagram becomes such that it is hard to discern the logic.
In highly domain specific areas VPLs are actually widely used. Blender 3D modelling tool is basically a graphical way of editing and visualising the underlying data structures. Now if you replace abstract syntax graph you can see the similarity.
wrt @RNeff's comment, AutoDesk Revit is a great example of how it might be possible to create a more general purpose programming language. What sets Revit apart from Verilog & VHDL is that it presents the user with context specific editors by which to set the parameters for the many different components of the designed building.
In highly domain specific areas VPLs are actually widely used. Blender 3D modelling tool is basically a graphical way of editing and visualising the underlying data structures. Now if you replace abstract syntax graph you can see the similarity.
wrt @RNeff's comment, AutoDesk Revit is a great example of how it might be possible to create a more general purpose programming language. What sets Revit apart from Verilog & VHDL is that it presents the user with context specific editors by which to set the parameters for the many different components of the designed building.
I actually started "programming" back when I was a kid working on Starcraft maps. The Starcraft editor isn't quite graphical but it definitely isn't text documents (though in Starcraft 2, I believe it compiles down to a C variant).
Starcraft has one thing that modern graphical editors don't have: reactivity. State is synchronized behind the scenes between client and server (who is also a client). You "program" predominately in response to events, which can be initiated by the game itself, player action or by another function call.
Reactivity, which is programming in response to events, in my opinion, will lead to the first useful non-notepad editors.
Starcraft has one thing that modern graphical editors don't have: reactivity. State is synchronized behind the scenes between client and server (who is also a client). You "program" predominately in response to events, which can be initiated by the game itself, player action or by another function call.
Reactivity, which is programming in response to events, in my opinion, will lead to the first useful non-notepad editors.
It is mainstream in certain domains. At my work we use a visual programming tool (an Eclipse plugin) for mapping data elements between one format to another. So if we have a customer who sends us something like a CSV file that they want us to upload to our system, but we need to transform it to XML first, we do that transformation mapping using the visual tool.
The main disadvantage is that over the years, we have developed amazing set of tools that all presuppose source code is text (so it's not only we are used to it, it's stronger than that). Can you imagine what would it take today to reimplement all the editors, source control, half of unix basically...
I'll take "fighting with syntax" over fighting with a GUI any day.
There are big advantages to alphabetic texts.
The three advantages that you cite are not.
1) there is no less fighting with syntax when you use a graphical syntax than when you use a textual syntax. Now we can agree that a lot of textual programming languages have horrible syntax. I would advise you to try Common Lisp or Scheme. The use of S-exps (Symbolic Expressions) in those languages is a recognition that the syntax doesn't matter, and therefore the uniform and systematic use of the forms: (operator argument argument) for all the language elements is a big simplification at the lexical level. However, there still remain a syntactic element, an essential one, where you have to know that eg. the IF operator takes three arguments, a test, and then-expression and an else-expression, in that order:
2) It is not easier for beginners to pick up. Only to children who don't know how to read yet. But then, one may argue that they ought to learn reading and writing before trying to write programs. In anycase, at this age, you would rather use objects than graphics, to let children play, programming "robots" by putting operation blocs into slots in a specific order.
3) Ok, it's your opinion, but graphical forms do not make it easier to find logic errors. They may help in getting an impression, and forming a mental image. But it would be dangerous to rely on them in a rigorous reflection about the program properties, just like it is bad to use the graphical diagrams to reason in Euclidian geometry. As goes the saying, geometry is the art of making correct reasoning using wrong graphics. Actually, you can do all geoemetry purely in textual and logical form, without ever using graphics, because the geometrical objects are abstract objects, and the graphical forms are only coarse approximations of the geometrical concepts. Similarly, the graphical forms of algorithms and programs, are but coarse approximations of the program logic, which is better described using text.
There's also another inconvenient of the graphical forms: they lack abstraction. You can zoom in and zoom out, and sometimes, you can even hide subcomponents in closed boxes, into which you can navigate graphically to discover the inside diagram, but by definition an abstraction is naming a concept. And text is better for names, if only because of the combinatorials. Take a standard human, and try to determine how many gestures, or how many different ideograms he is able to create and later recognize to identify different concepts. Similarly, count the number of words or names he is able to create and use to identify those concepts.
Finally, there's the question of the complexity of the tools required to manipulate graphics (and animations at that, if you want to manipulate sophisticated graphic views), vs. the tools required to deal with text, with a clear advantage for the later.
And let's not forget, at the beginning was the Verb, not the Image, and it's the Verb who was Creator.
That said, graphical forms can still be useful in the context of programming, notably when you want to explain some concepts. You can use various kind of diagrams, to present facets of software (cf. eg. UML), and other free-form diagrams. But either you have some formalized graphical syntax like UML, and therefore you bring the complexity of reading and writing correct meaningful diagrams, or you are left with ambiguity usually found with graphical forms, and you can only use as an illustration to some textual description.
You could also compare a program drawn in Piet vs. the same written in any textual programming languages :-) Can you tell what this program does: http://www.dangermouse.net/esoteric/piet/Piet_hello_big.png http://www.dangermouse.net/esoteric/piet.html And what about this one: http://rosettacode.org/wiki/Hello_world/Text#Modula-2
That said, if you read Asimov's Foundation serie, you may remember the room where the psychohistorian mathematicians worked, where mathematical formulae were displayed on all the walls of the room and where they could explore and manipulate them graphically with gestures and other input. I'd agree that it is possible that eventually we have sufficiently smart AI programs to be able to write programs and other mathematical proofs in graphical or such dynamical modifiable forms with simple and easy input (ie. with the system basically guessing what transformation or addition you mean from your grunting and waving). But I don't see that practical before AI is achieved.
The three advantages that you cite are not.
1) there is no less fighting with syntax when you use a graphical syntax than when you use a textual syntax. Now we can agree that a lot of textual programming languages have horrible syntax. I would advise you to try Common Lisp or Scheme. The use of S-exps (Symbolic Expressions) in those languages is a recognition that the syntax doesn't matter, and therefore the uniform and systematic use of the forms: (operator argument argument) for all the language elements is a big simplification at the lexical level. However, there still remain a syntactic element, an essential one, where you have to know that eg. the IF operator takes three arguments, a test, and then-expression and an else-expression, in that order:
(IF (= 1 a)
(print (quote one))
(return (quote done)))
2) It is not easier for beginners to pick up. Only to children who don't know how to read yet. But then, one may argue that they ought to learn reading and writing before trying to write programs. In anycase, at this age, you would rather use objects than graphics, to let children play, programming "robots" by putting operation blocs into slots in a specific order.
3) Ok, it's your opinion, but graphical forms do not make it easier to find logic errors. They may help in getting an impression, and forming a mental image. But it would be dangerous to rely on them in a rigorous reflection about the program properties, just like it is bad to use the graphical diagrams to reason in Euclidian geometry. As goes the saying, geometry is the art of making correct reasoning using wrong graphics. Actually, you can do all geoemetry purely in textual and logical form, without ever using graphics, because the geometrical objects are abstract objects, and the graphical forms are only coarse approximations of the geometrical concepts. Similarly, the graphical forms of algorithms and programs, are but coarse approximations of the program logic, which is better described using text.
There's also another inconvenient of the graphical forms: they lack abstraction. You can zoom in and zoom out, and sometimes, you can even hide subcomponents in closed boxes, into which you can navigate graphically to discover the inside diagram, but by definition an abstraction is naming a concept. And text is better for names, if only because of the combinatorials. Take a standard human, and try to determine how many gestures, or how many different ideograms he is able to create and later recognize to identify different concepts. Similarly, count the number of words or names he is able to create and use to identify those concepts.
Finally, there's the question of the complexity of the tools required to manipulate graphics (and animations at that, if you want to manipulate sophisticated graphic views), vs. the tools required to deal with text, with a clear advantage for the later.
And let's not forget, at the beginning was the Verb, not the Image, and it's the Verb who was Creator.
That said, graphical forms can still be useful in the context of programming, notably when you want to explain some concepts. You can use various kind of diagrams, to present facets of software (cf. eg. UML), and other free-form diagrams. But either you have some formalized graphical syntax like UML, and therefore you bring the complexity of reading and writing correct meaningful diagrams, or you are left with ambiguity usually found with graphical forms, and you can only use as an illustration to some textual description.
You could also compare a program drawn in Piet vs. the same written in any textual programming languages :-) Can you tell what this program does: http://www.dangermouse.net/esoteric/piet/Piet_hello_big.png http://www.dangermouse.net/esoteric/piet.html And what about this one: http://rosettacode.org/wiki/Hello_world/Text#Modula-2
That said, if you read Asimov's Foundation serie, you may remember the room where the psychohistorian mathematicians worked, where mathematical formulae were displayed on all the walls of the room and where they could explore and manipulate them graphically with gestures and other input. I'd agree that it is possible that eventually we have sufficiently smart AI programs to be able to write programs and other mathematical proofs in graphical or such dynamical modifiable forms with simple and easy input (ie. with the system basically guessing what transformation or addition you mean from your grunting and waving). But I don't see that practical before AI is achieved.
I can think of a few advantages off the top of my head: 1)Less fighting with syntax 2)Easier for beginners to pick up 3)Easier to find logic errors (in my opinion)
The main disadvantages I see is: 1)that it is slower to develop in 2)historically everything has been text so people are used to it
Any other thoughts on advantages and disadvantages? What would make you use a visual programming language?