Eve Development Blog, from Light Table developers(incidentalcomplexity.com)
incidentalcomplexity.com
Eve Development Blog, from Light Table developers
http://incidentalcomplexity.com/
75 comments
So how should it be done?
The way we do scoping now is to rely on the programming language which is basically the stack.
But we can implement our own scoping mechanisms based on the domain we are developing a program in.
For example, Eve mentioned that they were looking at a sort of "Excel" spreadsheet type visual language.
The scoping mechanism of spreadsheets are a sheet with row/column. So, I would create objects that are able to quickly pull from a row/column and push into a row/column. For example, somewhere in their hierarchy they had an algorithm that required access to a cursor.
Since it is a spreadsheet we can have our global active spreadsheet which, somewhere, contains the cursor. We know it is there because we put it there.
Somewhere deep in the bowels of the data hierarchy we have this line of code to get a cursor:
There is no need to push that current cursor down the function calling train and, in the end, it is probably faster because we are doing a simple cell lookup.
But we can implement our own scoping mechanisms based on the domain we are developing a program in.
For example, Eve mentioned that they were looking at a sort of "Excel" spreadsheet type visual language.
The scoping mechanism of spreadsheets are a sheet with row/column. So, I would create objects that are able to quickly pull from a row/column and push into a row/column. For example, somewhere in their hierarchy they had an algorithm that required access to a cursor.
Since it is a spreadsheet we can have our global active spreadsheet which, somewhere, contains the cursor. We know it is there because we put it there.
Somewhere deep in the bowels of the data hierarchy we have this line of code to get a cursor:
var cursorCurrent = global.curSpread[34,65];
There is no need to push that current cursor down the function calling train and, in the end, it is probably faster because we are doing a simple cell lookup.
The cursors were long before we were thinking explicitly of spreadsheets. Our model now has no encapsulating scope, which is just like access to some global object like you're suggesting here.
Why not full on adopt dynamic scoping? This is what McCarthy did in the original LISP (some say it was an accident, but it has lots of advantages compared to lexical scoping). Global scope, in contrast, is like having no scope at all. There is also nothing really wrong with this if you can manage your namespace separately Subtext style, but the use cases are still different from dynamic scoping.
> if you can manage your namespace separately Subtext style
We do
> the use cases are still different from dynamic scoping.
What are the use cases? We so far haven't felt the need for any kind of scoping (although we admittedly have only written a few 'real' programs).
It's not clear what dynamic scope would even really mean in Eve - there is no control-flow stack to set the scope and no notion of computer-time to dictate when a nested scope exists.
A better analog might be ML functors. We could take a chunk of Eve code and parametrise it by the input/output tables. Then you could wire it into your dataflow graph in multiple places.
We do
> the use cases are still different from dynamic scoping.
What are the use cases? We so far haven't felt the need for any kind of scoping (although we admittedly have only written a few 'real' programs).
It's not clear what dynamic scope would even really mean in Eve - there is no control-flow stack to set the scope and no notion of computer-time to dictate when a nested scope exists.
A better analog might be ML functors. We could take a chunk of Eve code and parametrise it by the input/output tables. Then you could wire it into your dataflow graph in multiple places.
LISP being technically based on the lambda calculus doesn't have any control flow either, but it does have hierarchy defined by callee/caller relationships. If you have any kind of hierarchy at all in your program execution (even if its just rules being used to for other rules...), you can leverage that as a "scope."
Dynamic scoping is useful when you want to configure different executions with different behaviors without invasively passing down contexts to do that. There is a whole body of work on context-oriented programming which essentially leverages dynamic scoping to make code execution more adaptive without sacrificing viscosity.
Dynamic scoping is useful when you want to configure different executions with different behaviors without invasively passing down contexts to do that. There is a whole body of work on context-oriented programming which essentially leverages dynamic scoping to make code execution more adaptive without sacrificing viscosity.
> when you want to configure different executions with different behaviors without invasively passing down contexts
That is certainly something we want, but I don't know if scoping is definitely the right way to do it, especially since both our code and data are entirely flat at the moment. We will likely have some kind of tagging of tables/rules for organisation and those tags could be the unit of reuse. The mechanism I have in mind feels more like copy-and-paste (I want that chunk of code but with these changes) or ML functors than it does dynamic scoping. In particular, the ability to change the dataflow graph at runtime without recompilation prevents lots of important optimisations.
That is certainly something we want, but I don't know if scoping is definitely the right way to do it, especially since both our code and data are entirely flat at the moment. We will likely have some kind of tagging of tables/rules for organisation and those tags could be the unit of reuse. The mechanism I have in mind feels more like copy-and-paste (I want that chunk of code but with these changes) or ML functors than it does dynamic scoping. In particular, the ability to change the dataflow graph at runtime without recompilation prevents lots of important optimisations.
With ML functors, however, you are duplicating the code and managing separately; you could already do that for each scope (what I basically do in Glitch), or better yet, try and optimize duplication away (copy your data flow graph, but memoize and try to reuse).
Anyways, the right path to take will probably become apparent as you do more development...keeping it simple at first is a good idea.
Anyways, the right path to take will probably become apparent as you do more development...keeping it simple at first is a good idea.
> try and optimize duplication away (copy your data flow graph, but memoize and try to reuse)
We sort of do that by hand at the moment. Rather than parametrise code, we attach ids to the data that is pushed through it. For example, instead of having a reusable 'compile' function we have a dataflow that just compiles anything thats added to the code tables and anything that wants to compile pushes code in one end and looks for the matching id coming out the other end.
I'm not sure how far that will take us. It's certainly annoying when what you want is more like a function with a return value rather than a side effect. Still, we might be able to address that with a little sugar. We'll see.
We sort of do that by hand at the moment. Rather than parametrise code, we attach ids to the data that is pushed through it. For example, instead of having a reusable 'compile' function we have a dataflow that just compiles anything thats added to the code tables and anything that wants to compile pushes code in one end and looks for the matching id coming out the other end.
I'm not sure how far that will take us. It's certainly annoying when what you want is more like a function with a return value rather than a side effect. Still, we might be able to address that with a little sugar. We'll see.
> but it does have hierarchy defined by callee/caller relationships. If you have any kind of hierarchy at all in your program execution (even if its just rules being used to for other rules...), you can leverage that as a "scope."
This is so true. You can take advantage of having access to a "parent" to change context. It allows you to re-use your code by just accessing information from your parent. It does lead to some coupling (it limits what types of parents you can have) but it does let you re-use code.
And hay Sean!
This is so true. You can take advantage of having access to a "parent" to change context. It allows you to re-use your code by just accessing information from your parent. It does lead to some coupling (it limits what types of parents you can have) but it does let you re-use code.
And hay Sean!
Interesting, do you have any reference about origin of COP in relation to dynamic scoping ?
From the man himself (Robert Hirschfeld): http://www.jot.fm/issues/issue_2008_03/article4/
Ha, I remember going through this one a few years back actually. I didn't know any names at that time.
That's basically what we end up doing in the February section. It fixes the information passing problem but then leads to new problems with re-usability. You can't parametrise a function using only global scoping. So then we ended up with both a stack and this global table lookup thing. We tried resolving it with that cursor system so you could pass a cursor to the root if you trusted a function or pass a cursor to some walled-off tree if you want to isolate it.
Another reason we weren't really satisfied with any of the hierarchical models because they required making lots of choices about ordering in the hierarchy that weren't really relevant to the actual problem being solved.
Now that we've moved to relational datalog-like language we just don't have scoping at all. Every table has a unique id and ambiguity between names is resolved at edit-time (eg by choosing from an autocomplete or by dragging a reference directly). We don't have a solid plan for handling reuse but we haven't actually felt the pain yet because doing things collection-at-a-time removes a good chunk of the use cases for reuse.
Another reason we weren't really satisfied with any of the hierarchical models because they required making lots of choices about ordering in the hierarchy that weren't really relevant to the actual problem being solved.
Now that we've moved to relational datalog-like language we just don't have scoping at all. Every table has a unique id and ambiguity between names is resolved at edit-time (eg by choosing from an autocomplete or by dragging a reference directly). We don't have a solid plan for handling reuse but we haven't actually felt the pain yet because doing things collection-at-a-time removes a good chunk of the use cases for reuse.
> You can't parametrise a function using only global scoping. So then we ended up with both a stack and this global table lookup thing.
Exactly. So, just write your functions so they don't use parameters (expect for persistence of the program). In fact, you can bypass functions all together. It also makes it a lot easier define and visually represent behavior.
> Another reason we weren't really satisfied with any of the hierarchical models because they required making lots of choices about ordering in the hierarchy that weren't really relevant to the actual problem being solved.
Hmm. Interesting. Could you give some examples? I would think that the hierarchy would be built out organically as you build out your program. Unless the hierarchy is actually just data that the program manipulates. Then, I could see it as an issue. But if the hierarchy is your program it should just grow organically.
> We don't have a solid plan for handling reuse but we haven't actually felt the pain yet
From a hierarchical standpoint, re-use (I'm guessing code re-use) is really about being able to change context by moving or executing part of your hierarchy (since a program is just a hierarchical composition of objects) in another part of your hierarchy. This is a little more difficult but can be done implementing another scoping mechanism.
Exactly. So, just write your functions so they don't use parameters (expect for persistence of the program). In fact, you can bypass functions all together. It also makes it a lot easier define and visually represent behavior.
> Another reason we weren't really satisfied with any of the hierarchical models because they required making lots of choices about ordering in the hierarchy that weren't really relevant to the actual problem being solved.
Hmm. Interesting. Could you give some examples? I would think that the hierarchy would be built out organically as you build out your program. Unless the hierarchy is actually just data that the program manipulates. Then, I could see it as an issue. But if the hierarchy is your program it should just grow organically.
> We don't have a solid plan for handling reuse but we haven't actually felt the pain yet
From a hierarchical standpoint, re-use (I'm guessing code re-use) is really about being able to change context by moving or executing part of your hierarchy (since a program is just a hierarchical composition of objects) in another part of your hierarchy. This is a little more difficult but can be done implementing another scoping mechanism.
> So, just write your functions so they don't use parameters (expect for persistence of the program). In fact, you can bypass functions all together. It also makes it a lot easier define and visually represent behavior.
I don't really understand what you are proposing. Can you describe how you would write a simple program (say a todo list) without parametrised functions? It works for Eve because its collection/query -oriented rather than value-oriented - we can express operations on arbitrary-length data without parametrisation. Even so, we expect to need it as we start to write bigger programs.
> Hmm. Interesting. Could you give some examples?
A good parallel that I use a lot is with file-systems. Picture your grandmother trying to decide whether http://jokideo.com/wp-content/uploads/2013/05/Funny-cat-and-... belongs in the funny-cat folder or the funny-baby folder.
Similarly, I often end up stuck in functional languages trying to decide whether I should have a map from employers to lists of contractors or from contractors to lists of employers. If I choose the former and I later need to find over-employed contractors the code will be really awkward. If I choose both I have to write extra code to maintain the relationship.
The big innovation of relational databases was the idea that you just directly model the employer-contractor relationship and you can later add contractor->employer or employer->contractor indexes without having to change any of your code. That's the big appeal of http://en.wikipedia.org/wiki/Data_independence .
The more I write code in relational languages, the more I notice how much of my time in other languages is spent trying to figure out where to put stuff and how to get at it later. Huge swathes of advice on design and maintainability are just working around this problem that relational databases (mostly) nailed decades ago.
I don't really understand what you are proposing. Can you describe how you would write a simple program (say a todo list) without parametrised functions? It works for Eve because its collection/query -oriented rather than value-oriented - we can express operations on arbitrary-length data without parametrisation. Even so, we expect to need it as we start to write bigger programs.
> Hmm. Interesting. Could you give some examples?
A good parallel that I use a lot is with file-systems. Picture your grandmother trying to decide whether http://jokideo.com/wp-content/uploads/2013/05/Funny-cat-and-... belongs in the funny-cat folder or the funny-baby folder.
Similarly, I often end up stuck in functional languages trying to decide whether I should have a map from employers to lists of contractors or from contractors to lists of employers. If I choose the former and I later need to find over-employed contractors the code will be really awkward. If I choose both I have to write extra code to maintain the relationship.
The big innovation of relational databases was the idea that you just directly model the employer-contractor relationship and you can later add contractor->employer or employer->contractor indexes without having to change any of your code. That's the big appeal of http://en.wikipedia.org/wiki/Data_independence .
The more I write code in relational languages, the more I notice how much of my time in other languages is spent trying to figure out where to put stuff and how to get at it later. Huge swathes of advice on design and maintainability are just working around this problem that relational databases (mostly) nailed decades ago.
In my favourite homoiconic language, Racket (also in lisps, so it's surprising that OP didn't mention it) you'd typically accomplish it with parameterised dynamic binding. Think global variables, except the lifetime of the new value is limited to a body of code.
A concrete example is a function that writes something to stdout. If you want to output to a file instead, parameterise the function call by setting current-output-port to the handle of your file.
A concrete example is a function that writes something to stdout. If you want to output to a file instead, parameterise the function call by setting current-output-port to the handle of your file.
Dynamic scoping?
Why is hierarchical organization correct?
They are going for a visual language, and visual languages are inherently hierarchal. The goal is to easily represent context of any part of a program in a visual way.
Trying to map source-code that stores a lot of it's context on the stack is really hard to visually represent. Trying to represent the stack in a visual way makes for a icky looking visual language.
However, if you can keep all of the context/state of your program in one area and it is hierarchal... Then, you can build great visual representations of your program because, really, you are displaying a data-structure (your program becomes a data-structure). So, all the cool things you can do visually representing data-structures, you can do with your actual program.
Trying to map source-code that stores a lot of it's context on the stack is really hard to visually represent. Trying to represent the stack in a visual way makes for a icky looking visual language.
However, if you can keep all of the context/state of your program in one area and it is hierarchal... Then, you can build great visual representations of your program because, really, you are displaying a data-structure (your program becomes a data-structure). So, all the cool things you can do visually representing data-structures, you can do with your actual program.
> They are going for a visual language
'Visual language' is a very overloaded term and we are in no way set on any particular ideology. Textual languages and visual layout have different strengths and weaknesses and there are lots of cases where it makes sense to mix both. There is plenty of evidence that both are useful (eg http://www.amazon.com/Small-Matter-Programming-Perspectives-...). My only aversion to text is as an unstructured storage and communication format - it makes much more sense to allow tools to communicate using a shared, versioned and structured database rather than watching for changes in text files.
> visual languages are inherently hierarchal
Excel - https://www.google.com/search?q=excel&client=ubuntu&hs=fRx&c...
Access - https://www.google.com/search?q=access&client=ubuntu&hs=Jnc&...
Labview - https://www.google.com/search?q=labview&client=ubuntu&hs=v5H...
Blender - https://www.google.com/search?q=blender+node+editor&client=u...
PureData - https://www.google.com/search?q=puredata&client=ubuntu&hs=4S...
TouchDevelop - https://www.google.com/search?q=touchdevelop&client=ubuntu&h...
Kodu - https://www.google.com/search?q=kodu&client=ubuntu&hs=N9H&ch...
Pretty much the only hierarchical example that comes to mind is Scratch.
> Trying to map source-code that stores a lot of it's context on the stack is really hard to visually represent.
Yes. But non-local scope and control flow are also pretty hard to represent and understand. That's why we ended up avoiding both.
> However, if you can keep all of the context/state of your program in one area and it is hierarchal...
It doesn't have to be hierarchical. Excel and Access both have amazing (by programming tool standards) visual representations of your program and your data.
> So, all the cool things you can do visually representing data-structures, you can do with your actual program.
Data and code are different though. Eve is homoiconic-ish (code is stored in tables) but the actual visualisations and manipulations applied in 99% of the use cases are very different. We don't spend nearly as much time manipulating code as data compared to reading, writing and understanding code so it makes sense to default to visualisations that optimise for the latter.
'Visual language' is a very overloaded term and we are in no way set on any particular ideology. Textual languages and visual layout have different strengths and weaknesses and there are lots of cases where it makes sense to mix both. There is plenty of evidence that both are useful (eg http://www.amazon.com/Small-Matter-Programming-Perspectives-...). My only aversion to text is as an unstructured storage and communication format - it makes much more sense to allow tools to communicate using a shared, versioned and structured database rather than watching for changes in text files.
> visual languages are inherently hierarchal
Excel - https://www.google.com/search?q=excel&client=ubuntu&hs=fRx&c...
Access - https://www.google.com/search?q=access&client=ubuntu&hs=Jnc&...
Labview - https://www.google.com/search?q=labview&client=ubuntu&hs=v5H...
Blender - https://www.google.com/search?q=blender+node+editor&client=u...
PureData - https://www.google.com/search?q=puredata&client=ubuntu&hs=4S...
TouchDevelop - https://www.google.com/search?q=touchdevelop&client=ubuntu&h...
Kodu - https://www.google.com/search?q=kodu&client=ubuntu&hs=N9H&ch...
Pretty much the only hierarchical example that comes to mind is Scratch.
> Trying to map source-code that stores a lot of it's context on the stack is really hard to visually represent.
Yes. But non-local scope and control flow are also pretty hard to represent and understand. That's why we ended up avoiding both.
> However, if you can keep all of the context/state of your program in one area and it is hierarchal...
It doesn't have to be hierarchical. Excel and Access both have amazing (by programming tool standards) visual representations of your program and your data.
> So, all the cool things you can do visually representing data-structures, you can do with your actual program.
Data and code are different though. Eve is homoiconic-ish (code is stored in tables) but the actual visualisations and manipulations applied in 99% of the use cases are very different. We don't spend nearly as much time manipulating code as data compared to reading, writing and understanding code so it makes sense to default to visualisations that optimise for the latter.
> 'Visual language' is a very overloaded term and
All great points and I agree with them.
> Pretty much the only hierarchical example that comes to mind is Scratch.
I tend to lean towards Sean's post: https://news.ycombinator.com/item?id=8468838.
> non-local scope and control flow
I read you are going for an "Excel" type approach. I thought you were doing this to deal with non-local scope. I hope so because I think it is a good idea.
> are also pretty hard to represent and understand
Control flow is also really hard to represent but I don't think it is because of any aspect of the visual representation or core technology. You can remove coding from programming but you can't remove critical thinking and problem solving from programming: which is basically what programming is.
Control flow is critical thinking and problem solving. That is why when you "presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked."
It is because there are inherent mental models that we use as tools to solve problems and think critically. People need to be given a visual representation of control flow in a way they are familiar with. I'm guessing you guys have done an amazing job (I've seen light table).
Frankly, I have a lot of experience in this space. I've spent ~5 to 6 years studying it and building out tools. I recognize a lot of the pains you are going through because I've been there. I would be more than happy to chat with you about it. I am totally stoked about Eve and would like to help.
> understanding code so it makes sense to default to visualisations that optimise for the latter.
I may be totally reading this wrong but you should not default visualisations to represent a given program. In my opinion, this is a mistake that all VPLish type things have made (as you said, it is a totally overloaded word). You can easily represent a given small part of a program in any domain you like: using the same underlying program. It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
This is one of the great strengths of hierarchical type programming languages: the same code can be represented using multiple visual abstractions tailored to the user.
This is why I totally agree with this statement "The environment should be optimised for the way that people use it, not the way that we think it should be used."
but think you should reconsider this "We should observe users and collect data to discover common actions and workflows." There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
Programmers are only kind of aligned because we all go through this grinder that requires us to fight with compilers and figure out error messages and deal with syntax and so on. So, we all kinda mentally solve problems in the same way (and by kinda, it isn't like close but a lot closer than say an accountant and a painter solve their problems).
I hope this isn't too long winded but I really do have a lot of knowledge in this space.
All great points and I agree with them.
> Pretty much the only hierarchical example that comes to mind is Scratch.
I tend to lean towards Sean's post: https://news.ycombinator.com/item?id=8468838.
> non-local scope and control flow
I read you are going for an "Excel" type approach. I thought you were doing this to deal with non-local scope. I hope so because I think it is a good idea.
> are also pretty hard to represent and understand
Control flow is also really hard to represent but I don't think it is because of any aspect of the visual representation or core technology. You can remove coding from programming but you can't remove critical thinking and problem solving from programming: which is basically what programming is.
Control flow is critical thinking and problem solving. That is why when you "presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked."
It is because there are inherent mental models that we use as tools to solve problems and think critically. People need to be given a visual representation of control flow in a way they are familiar with. I'm guessing you guys have done an amazing job (I've seen light table).
Frankly, I have a lot of experience in this space. I've spent ~5 to 6 years studying it and building out tools. I recognize a lot of the pains you are going through because I've been there. I would be more than happy to chat with you about it. I am totally stoked about Eve and would like to help.
> understanding code so it makes sense to default to visualisations that optimise for the latter.
I may be totally reading this wrong but you should not default visualisations to represent a given program. In my opinion, this is a mistake that all VPLish type things have made (as you said, it is a totally overloaded word). You can easily represent a given small part of a program in any domain you like: using the same underlying program. It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
This is one of the great strengths of hierarchical type programming languages: the same code can be represented using multiple visual abstractions tailored to the user.
This is why I totally agree with this statement "The environment should be optimised for the way that people use it, not the way that we think it should be used."
but think you should reconsider this "We should observe users and collect data to discover common actions and workflows." There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
Programmers are only kind of aligned because we all go through this grinder that requires us to fight with compilers and figure out error messages and deal with syntax and so on. So, we all kinda mentally solve problems in the same way (and by kinda, it isn't like close but a lot closer than say an accountant and a painter solve their problems).
I hope this isn't too long winded but I really do have a lot of knowledge in this space.
> I thought you were doing this to deal with non-local scope.
Non-local data-flow (you can reference any table in a query) but only local control-flow (at the same level as excels 'if' statement) and no scoping (references are absolute and global - they don't change depending on where you are in the program).
> Control flow is critical thinking and problem solving.
Critical thinking and problem solving do not require programming with control flow and they are certainly not limited to programmers (you didn't claim the latter, but it is a common assertion on hn and it needs to die). Here are some of the people we've talked to about their programming needs, both for Eve and in past work:
* a radiologist with phds in both medicine and nuclear physics * an ex- radar engineer who now manages and schedules multi-million dollar aerospace projects * a group of traders at a prestigious prop trading company
These people are certainly capable of critical thinking, abstraction, problem solving etc. All of them build complex programs in Excel and other tools. None of them have got the hang of traditional programming. I've seen research that argues that supplying task-specific primitives and ensuring early success on the learning curve are both vital. I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
That's not to say that non-local control-flow is evil or that it can't be taught, just that it seems to put a big hump in the learning curve and we can usually get away without it.
> It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
When I say visualisation I include all of the above, not just graphical representations. For example, our default table view is a grid of cells and our default rule view is a mixture of graphical meta-data and textual formulae.
> There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
I think you are thinking much higher level than I am. There are lots of common actions that every programmer I know does every day. For example, I run my program in the browser, check the console for exceptions, put a breakpoint on the line where the exception happened and try to cause the exception to happen again so I can observe the local state, often having to click through the breakpoint multiple times to get the actual error case. In any sane environment (eg Common Lisp) that process would be optimised to: check the console for exceptions, click the 'open debugger here' button next to the exception.
Non-local data-flow (you can reference any table in a query) but only local control-flow (at the same level as excels 'if' statement) and no scoping (references are absolute and global - they don't change depending on where you are in the program).
> Control flow is critical thinking and problem solving.
Critical thinking and problem solving do not require programming with control flow and they are certainly not limited to programmers (you didn't claim the latter, but it is a common assertion on hn and it needs to die). Here are some of the people we've talked to about their programming needs, both for Eve and in past work:
* a radiologist with phds in both medicine and nuclear physics * an ex- radar engineer who now manages and schedules multi-million dollar aerospace projects * a group of traders at a prestigious prop trading company
These people are certainly capable of critical thinking, abstraction, problem solving etc. All of them build complex programs in Excel and other tools. None of them have got the hang of traditional programming. I've seen research that argues that supplying task-specific primitives and ensuring early success on the learning curve are both vital. I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
That's not to say that non-local control-flow is evil or that it can't be taught, just that it seems to put a big hump in the learning curve and we can usually get away without it.
> It could look like a mathematical equation to a mathematician, textual code to a programmer or process flow abstractions to someone in logistics.
When I say visualisation I include all of the above, not just graphical representations. For example, our default table view is a grid of cells and our default rule view is a mixture of graphical meta-data and textual formulae.
> There are no common actions or assumptions you can make when going visual because there are so many different ways people approach solving a problem.
I think you are thinking much higher level than I am. There are lots of common actions that every programmer I know does every day. For example, I run my program in the browser, check the console for exceptions, put a breakpoint on the line where the exception happened and try to cause the exception to happen again so I can observe the local state, often having to click through the breakpoint multiple times to get the actual error case. In any sane environment (eg Common Lisp) that process would be optimised to: check the console for exceptions, click the 'open debugger here' button next to the exception.
> Non-local data-flow (you can reference any table in a query) but only local control-flow (at the same level as excels 'if' statement) and no scoping (references are absolute and global - they don't change depending on where you are in the program).
I'm quit surprised by this. In data-flow programming (e.g. FRP), non-local data-flow is a big no no, while non-local non-contiguous control-flow rules the day but is completely encapsulated (programmers only manipulate data flow via wiring, control flow is inferred).
> I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
This is a guess, but I don't really see it. A cook can definitely follow a recipe, or even write one for others to read, which involves lots of control flow. We are totally steeped "in time" and our communications often make references to sequenced time-stepped actions.
Spreadsheets are great in accounting and were invented way before we had computers. Math and logic are also nice, and very specialized for the educated, but they came way after we developed our skills for natural language (which was the original OOP). It is quite difficult to say why programming is hard, and what makes it easy.
I'm quit surprised by this. In data-flow programming (e.g. FRP), non-local data-flow is a big no no, while non-local non-contiguous control-flow rules the day but is completely encapsulated (programmers only manipulate data flow via wiring, control flow is inferred).
> I suspect that the distinction between traditional control-flow-heavy programming languages and static-data-flow end-user languages plays a big part in that learning curve.
This is a guess, but I don't really see it. A cook can definitely follow a recipe, or even write one for others to read, which involves lots of control flow. We are totally steeped "in time" and our communications often make references to sequenced time-stepped actions.
Spreadsheets are great in accounting and were invented way before we had computers. Math and logic are also nice, and very specialized for the educated, but they came way after we developed our skills for natural language (which was the original OOP). It is quite difficult to say why programming is hard, and what makes it easy.
> Critical thinking and problem solving do not require programming with control flow
Ya bad choice of words. Control flow requires critical thinking and problem solving (but critical thinking and problem solving is not always about control flow).
Hmm. Though, I guess if you go way deep, every decision we make is based on two or more options.
> These people are certainly capable of critical thinking, abstraction, problem solving etc.
Ya. But the way they think critically, their internal mental model, might be totally different than how you are I think. So, we all may come to the exact same conclusion. Just through a very different thought process using different abstractions to represent that critical thinking.
Ya bad choice of words. Control flow requires critical thinking and problem solving (but critical thinking and problem solving is not always about control flow).
Hmm. Though, I guess if you go way deep, every decision we make is based on two or more options.
> These people are certainly capable of critical thinking, abstraction, problem solving etc.
Ya. But the way they think critically, their internal mental model, might be totally different than how you are I think. So, we all may come to the exact same conclusion. Just through a very different thought process using different abstractions to represent that critical thinking.
> Pretty much the only hierarchical example that comes to mind is Scratch.
Excel is your first example (and the one that the Eve dev blog keeps referring to while showing examples that aren't very Excel-like visually except insofar as anything which uses a grid as one of its visual elements is "Excel-like"), and Excel's objects are accessed through heirarchical identifiers where the parts of the heirarchy that are local to where the reference is used can be omitted. (Well, sort of; for external files, rather than locality, the path can be omitted if the file is currently open in Excel -- and certain references, notably tables, only work in the same circumstances that allow omitting the path.)
The heirarchy is path -> filename -> defined-name for defined names, and path -> filename -> sheetname -> range for unnamed cells or ranges; each of the steps of this heirarchy are also directly represented in the visual UI you interact with when doing references by point/click/drag rather than textually, as well. So, both in the names and the visual metaphor, Excel is heirarchical.
Excel is your first example (and the one that the Eve dev blog keeps referring to while showing examples that aren't very Excel-like visually except insofar as anything which uses a grid as one of its visual elements is "Excel-like"), and Excel's objects are accessed through heirarchical identifiers where the parts of the heirarchy that are local to where the reference is used can be omitted. (Well, sort of; for external files, rather than locality, the path can be omitted if the file is currently open in Excel -- and certain references, notably tables, only work in the same circumstances that allow omitting the path.)
The heirarchy is path -> filename -> defined-name for defined names, and path -> filename -> sheetname -> range for unnamed cells or ranges; each of the steps of this heirarchy are also directly represented in the visual UI you interact with when doing references by point/click/drag rather than textually, as well. So, both in the names and the visual metaphor, Excel is heirarchical.
True, I hadn't considered that. The data-flow is not hierarchical, which is what distinguishes it in my mind, although apparently not in my commenting :)
Let me back up and state my premises more accurately. Most successful visual languages:
* emphasise global data-flow and only have control-flow locally, if at all (eg 'if' expressions in excel)
* support broad, shallow hierarchies for organising large projects but don't require any hierarchy for small projects
* allow data-flow between arbitrary sources/sinks regardless of data organisation
So control-flow hardly exists, data-flow is not hierarchical and the data model has a shallow hierarchy where the leaves are big flat chunks of data.
I suspect that even in organisation, a single strict hierarchy is not the best model. Non-technical users often struggle with hierarchical file-systems and prefer tags or search. File-systems themselves end up needing to implement multiple parents (eg links in linux).
We haven't gotten to large projects yet in Eve but our solution will likely draw from the evolution from hierarchical file-systems to tags, search and 'smart folders'.
Let me back up and state my premises more accurately. Most successful visual languages:
* emphasise global data-flow and only have control-flow locally, if at all (eg 'if' expressions in excel)
* support broad, shallow hierarchies for organising large projects but don't require any hierarchy for small projects
* allow data-flow between arbitrary sources/sinks regardless of data organisation
So control-flow hardly exists, data-flow is not hierarchical and the data model has a shallow hierarchy where the leaves are big flat chunks of data.
I suspect that even in organisation, a single strict hierarchy is not the best model. Non-technical users often struggle with hierarchical file-systems and prefer tags or search. File-systems themselves end up needing to implement multiple parents (eg links in linux).
We haven't gotten to large projects yet in Eve but our solution will likely draw from the evolution from hierarchical file-systems to tags, search and 'smart folders'.
Most of those languages have some kind of hierarchy. Lab view can define sub-components, TouchDevelop is basically procedural, excel has sub-sheets. Kodu is not hierarchical, but neither is it really a full language and the Brooks' subsumption architecture it is based is hierarchical (check out [1] on how to make Kodu hierarchical).
[1] http://research.microsoft.com/pubs/179364/p61-mcdirmid.pdf
[1] http://research.microsoft.com/pubs/179364/p61-mcdirmid.pdf
Most of them don't use hierarchical data models, which is what we were discussing (I think?). Excel has one level of nesting (sheet.cell). Labview is a graph where components are grouped for reuse/abstraction. Access is relational(ish). TouchDevelop and Scratch are more traditionally hierarchical, with control-flow stacks and directed object graphs.
I certainly wouldn't call Excel or Access 'inherently hierarchical'. Neither do I see the connection between 'visual languages' and 'inherently hierarchical' languages. If anything, the majority of visual languages I can think allow placing arbitrary computation at any point between arbitrary data sources/sinks. Where hierarchy exists it is usually fairly flat and used for organisation or code reuse. Arbitrary data-flow between any level or branch of the hierarchy is also a common feature.
I certainly wouldn't call Excel or Access 'inherently hierarchical'. Neither do I see the connection between 'visual languages' and 'inherently hierarchical' languages. If anything, the majority of visual languages I can think allow placing arbitrary computation at any point between arbitrary data sources/sinks. Where hierarchy exists it is usually fairly flat and used for organisation or code reuse. Arbitrary data-flow between any level or branch of the hierarchy is also a common feature.
> Excel has one level of nesting (sheet.cell).
Excel has one level of nesting within a file, but Excel "programs" are not limited to a single file; and, in fact, Excel has features designed specifically for programs that require multiple files to be open at once to interact.
Its true that each level of nesting in Excel is different in kind so that Excel doesn't support arbitrary nesting of namespaces, and that different bits of Excel functionality support different degrees of nesting.
Excel has one level of nesting within a file, but Excel "programs" are not limited to a single file; and, in fact, Excel has features designed specifically for programs that require multiple files to be open at once to interact.
Its true that each level of nesting in Excel is different in kind so that Excel doesn't support arbitrary nesting of namespaces, and that different bits of Excel functionality support different degrees of nesting.
Ah, I see. LabView supports objects, so its data model can at least be hierarchical there. But even if we ignore OO data models, I think even FRP (as pure data flow as we get) is hierarchical since components (and sub-components) can still encapsulate state. The only thing you really lose are aliases/implicit communication channels. Most visual data-flow languages follow this style.
I would argue that relational is also effectively hierarchical if not intrinsically: even without nested tables you always have IDs tacked on in a column that refers to rows in other tables. And once you start doing that, well...object graph.
I would argue that relational is also effectively hierarchical if not intrinsically: even without nested tables you always have IDs tacked on in a column that refers to rows in other tables. And once you start doing that, well...object graph.
> I think even FRP (as pure data flow as we get) is hierarchical since components (and sub-components) can still encapsulate state.
That's an interesting point. We explicitly avoid encapsulating state in that way, preferring views to interfaces.
> I would argue that relational is also effectively hierarchical ... object graph.
A graph is fine. The difference from a typical OO hierarchy is we get to start wherever we want in the graph and we can easily insert new edges anywhere in the graph, or hyperedges that relate multiple entities. It's not like a filesystem where you have to decide which property of the file is most important and gets to be at the top of the hierarchy.
That's an interesting point. We explicitly avoid encapsulating state in that way, preferring views to interfaces.
> I would argue that relational is also effectively hierarchical ... object graph.
A graph is fine. The difference from a typical OO hierarchy is we get to start wherever we want in the graph and we can easily insert new edges anywhere in the graph, or hyperedges that relate multiple entities. It's not like a filesystem where you have to decide which property of the file is most important and gets to be at the top of the hierarchy.
> A graph is fine. The difference from a typical OO hierarchy is we get to start wherever we want in the graph and we can easily insert new edges anywhere in the graph, or hyperedges that relate multiple entities. It's not like a filesystem where you have to decide which property of the file is most important and gets to be at the top of the hierarchy.
Objects support references that give you pretty much the same thing. The ability to add new kinds of edges is more of a function of your object system. In a dynamic language where members are basically defined as a dictionary, you just add a new key-value pair to the dictionary.
Objects support references that give you pretty much the same thing. The ability to add new kinds of edges is more of a function of your object system. In a dynamic language where members are basically defined as a dictionary, you just add a new key-value pair to the dictionary.
> excel has sub-sheets
It has sheets, but not subsheets (which are a feature of some other spreadsheet programs that you can find lots of questions online about how to do or fake in Excel; that being said, its still heirarchical, both in its structure and the visual metaphor for interacting with it.)
It has sheets, but not subsheets (which are a feature of some other spreadsheet programs that you can find lots of questions online about how to do or fake in Excel; that being said, its still heirarchical, both in its structure and the visual metaphor for interacting with it.)
Are sub-components and sub-sheets an argument for _hierarchy_, or for reuse of symbols? I think it's the latter.
As far as abstractions go, they are hierarchical. Actually, abstraction is all about breaking up and managing complexity into separate parts, and one would do pretty poorly without any hierarchy in their code.
Even Kodu is hierarchical to a limited degree since each bot is programmed separately, but its not unlimited as in the other examples listed (which I would argue, is necessary to be considered hierarchical).
Even Kodu is hierarchical to a limited degree since each bot is programmed separately, but its not unlimited as in the other examples listed (which I would argue, is necessary to be considered hierarchical).
OP's premise wasn't that "hierarchy exists", but that it's "the correct organization of data." Now you are just glibly pointing out that everything can be structured as a hierarchy. That does not make the case.
I'm not pointing out that everything can be structured in a hierarchy, I'm pointing out that everything needs to be structured in a hierarchy at some point to deal with complexity; these are very different observations.
> The bad is not the hierarchical organization of data. That is the correct direction.
This is just a repeat of the xml database vs relational database debate. This fight has been fought so many times before - http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.113....
This is just a repeat of the xml database vs relational database debate. This fight has been fought so many times before - http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.113....
The context of the discussion is the representation of state within an executing program.
Xml database, key-value-store, relational databases, csv files, etc. are all ways of persisting.
These are very different disciplines with very different problems being solved.
Xml database, key-value-store, relational databases, csv files, etc. are all ways of persisting.
These are very different disciplines with very different problems being solved.
That distinction is a historical accident. You can page a program heap to disk and you can run a relational database entirely in-memory (this is one of the common use cases for sqlite). True, databases and programming languages are usually optimised for different use cases but that says more about the particular implementation and the way we think about using them then it does about the data models.
The question in both cases is how should we model, query and update our data. The original tradeoff was that hierarchical databases and network databases were faster but relational databases were more flexible and maintainable (due to the separation of logical and physical data representation). Better implementations (eg http://en.wikipedia.org/wiki/IBM_System_R) narrowed the divide and made relational databases more attractive. Systems like Bloom (http://boom.cs.berkeley.edu/) and LogicBlox (http://www.infoq.com/presentations/wavefront) show that the same tipping point could be reached for programming languages.
The question in both cases is how should we model, query and update our data. The original tradeoff was that hierarchical databases and network databases were faster but relational databases were more flexible and maintainable (due to the separation of logical and physical data representation). Better implementations (eg http://en.wikipedia.org/wiki/IBM_System_R) narrowed the divide and made relational databases more attractive. Systems like Bloom (http://boom.cs.berkeley.edu/) and LogicBlox (http://www.infoq.com/presentations/wavefront) show that the same tipping point could be reached for programming languages.
Ok, I see what you are saying now. I really agree that there is no difference really between programs and databases. It really is "how should we model, query and update our data."
So, ya, the question is "How do you pass information between sub-routines". Databases aren't expected to do much data manipulation between sub-routines. Programs are just wow so many ways to do it.
Thank you for taking the time to have this discussion.
So, ya, the question is "How do you pass information between sub-routines". Databases aren't expected to do much data manipulation between sub-routines. Programs are just wow so many ways to do it.
Thank you for taking the time to have this discussion.
> So, ya, the question is "How do you pass information between sub-routines".
Nitpicking now, but phrasing it that way supposes the existence of sub-routines. I would prefer to phrase as just thinking about control-flow and data-flow. Most programming languages have complex control-flow and constraint data-flow to follow along with control-flow. Most successful end-user languages have arbitrary but static data-flow and little to no control-flow.
> Thank you for taking the time to have this discussion.
If nothing else, I find that these sorts of discussions tend to be really useful to clarify my thinking and catch my mistakes. Civil disagreement is much more interesting than ideological agreement or disagreement :)
Nitpicking now, but phrasing it that way supposes the existence of sub-routines. I would prefer to phrase as just thinking about control-flow and data-flow. Most programming languages have complex control-flow and constraint data-flow to follow along with control-flow. Most successful end-user languages have arbitrary but static data-flow and little to no control-flow.
> Thank you for taking the time to have this discussion.
If nothing else, I find that these sorts of discussions tend to be really useful to clarify my thinking and catch my mistakes. Civil disagreement is much more interesting than ideological agreement or disagreement :)
What's going on with Light Table? Anyone use it? Is it "done"? Python? JS? It was hot for a while and I haven't heard anything in a bit.
https://groups.google.com/forum/#!topic/light-table-discussi...
TLDR: There are three new maintainers (https://github.com/one-more-minute https://github.com/cldwalker https://github.com/joshuafcole). The docs are now community owned. The plugin server will be soon. We will be hosting a weekly hack night in our new office to get community members up to speed.
TLDR: There are three new maintainers (https://github.com/one-more-minute https://github.com/cldwalker https://github.com/joshuafcole). The docs are now community owned. The plugin server will be soon. We will be hosting a weekly hack night in our new office to get community members up to speed.
I'm glad to see you guys picked up the pace big time. Really glad. Props.
This is really good to see!
I just ditched it. It's so close to being good, but there are a ton of small bugs that can add up to being quite frustrating. It's no where near the level of a tool that will be used as much as an editor.
And now it looks like they have found something else more interesting to work on anyway.
And now it looks like they have found something else more interesting to work on anyway.
It's been my main editor pretty much since release, mainly for JavaScript and Python. I mainly use it as a snazzier gedit with some neat features like live markdown preview and built-in browser tab.
I still run PyCharm in a separate workspace for debugging, running programs and searching through code, but for editing I find LightTable is wildly preferable imo.
There are some mostly minor annoyances, like the behaviours file simply switched locations at one point, and for some odd reason I can't get the tab settings to work properly for makefiles and such.
I will probably stick with LightTable for the forseeable future, nothing else quite seems to hit my sweet spot. Admittedly I haven't actually tried sublime text.
I still run PyCharm in a separate workspace for debugging, running programs and searching through code, but for editing I find LightTable is wildly preferable imo.
There are some mostly minor annoyances, like the behaviours file simply switched locations at one point, and for some odd reason I can't get the tab settings to work properly for makefiles and such.
I will probably stick with LightTable for the forseeable future, nothing else quite seems to hit my sweet spot. Admittedly I haven't actually tried sublime text.
It's been opensourced which as far as I can tell, means that the developers wash their hands now and leave to move on to new, easier and more fun things than writing a decent text editor. They delivered half of what they promised 1 year overdue. But I guess if you want something done now you're free to write it yourself ;)
"Free" as in speech, beer, and a massive Kickstarter.
you can check it out on github
https://github.com/LightTable/LightTable
last commit 19 hours ago.
I don't know about python, but Javascript works great. The killer app for me is using the editor as an integrated repl for JS.
Doesn't pretty much any browser have this? If you're doing JS dev, chances are you have a browser open.
True. The appeal for me is directly manipulating the code, and getting feedback. The browser console is laid out in a call response model, and you are unable to edit previous entries. You have to retype them in, or hit up on the keyboard to find the previously entered function. It is nicer to just manipulate my code and evaluate it right in the editor. It isn't a huge difference, but it has been helpful for me.
I tried using it as I'm just learning clojure. I'm okay with learning new things if they're worth my time, but I didn't find the editor worth its resources. On OS X, Light Table uses more memory than Eclipse + Counterclockwise (on Ubuntu). After seeing that, I dropped it and went to Eclipse (save RAM and cognitive load).
Huh. I just opened a brand new download of eclipse and it's at 440mb with no files open. LT is 115 + 50 having been open all week.
Did you have some weird plugins installed? Did you create a bunch of Clojure processes and count those?
EDIT: Ah, there was a memory leak for a while. I bet that's what the problem was. It's definitely fixed in master.
Did you have some weird plugins installed? Did you create a bunch of Clojure processes and count those?
EDIT: Ah, there was a memory leak for a while. I bet that's what the problem was. It's definitely fixed in master.
I'll download LT again. I liked it. Just leaked, apparently.
Nice thought, nice venture. I hope it leads to unexplored territory. Good to see constant tests with potential users, and no fear of making major changes.
Eve, Aurora, Light Table, these names all share the similar property in that they are difficult to google because they are common words. We've already gone through this once with go, can we please get a name that is going to be easy to search for?
If the blog maintainer is about - can you allow user zooming on mobile? The screenshots are very tricky to look at and are important to the context of this blog I think.
In that same vein, the dotted pattern in background makes it subtly harder to read the text
You can click though on the images.
Use this bookmarklet to un-cripple the page (from [1]):
[1] http://apple.stackexchange.com/questions/28391/how-can-i-for...
[2] https://news.ycombinator.com/item?id=8468745
[3] http://motherfuckingwebsite.com/
javascript:document.querySelector('meta[name=viewport]')
.setAttribute('content','
width=device-width,
initial-scale=1.0,
maximum-scale=10.0,
user-scalable=1
');
[1] http://apple.stackexchange.com/questions/28391/how-can-i-for...
[2] https://news.ycombinator.com/item?id=8468745
[3] http://motherfuckingwebsite.com/
What an aggressive response. :/
Thanks for the indirect pointer on how to correct the issue though. If you hard refresh you can zoom now.
Thanks for the indirect pointer on how to correct the issue though. If you hard refresh you can zoom now.
Thank you Chris - I realized you could click through after I posted but I think it's good to zoom in and look at the parts while reading the text.
Well, expectancy violation/mismatch is a reliable stressor down to the mesolimbic-corticostriatal level[1], so maybe that's that.
In any case, thank you for re-enabling it. I'm sure many future readers would thank you too.
[1] See for instance Box 2 in Proulx et al. 2012 (http://www.michaelinzlicht.com/wp/wp-content/uploads/downloa...).
In any case, thank you for re-enabling it. I'm sure many future readers would thank you too.
[1] See for instance Box 2 in Proulx et al. 2012 (http://www.michaelinzlicht.com/wp/wp-content/uploads/downloa...).
God, that's confusing. I thought this was the EVE development blog, i.e. http://community.eveonline.com/news/dev-blogs/. Eve is, frankly, a terrible name for a tool. It sounds like a brand name and gives ZERO indication as to its utility.
At the very least, the post (edit: post title) could give some indication as to what the hell Eve is and why I should care—it looks like a spreadsheet meets a repl.
At the very least, the post (edit: post title) could give some indication as to what the hell Eve is and why I should care—it looks like a spreadsheet meets a repl.
[deleted]
As an Eve On-Line player: same here. Spreadsheet with a REPL... might help crunch market orders.
This is the function-calling train issue and what happens if you try to make a sort of quasi homoiconic language ("Hierarchical organisation of data complects the data itself with the common access paths.") and attempt a push-pull approach to programming (you push the data down into the hierarchy) with traditional data scoping through parameters.
The bad is not the hierarchical organization of data. That is the correct direction.
The bad is trying to use a traditional scoping approach to what is effectively a homoiconic type language. Homoiconic type systems need a different scoping approach.