One potential solution direction, which you can try out via my own incomplete drawing toy [1] is to treat punctuation characters like SP (“ “), COMMA (“,”), and SEMICOLON (“;”) as markers for the product operations of a family of monoids that allow you to specify more and more complicated sequences without requiring the typist to “move the cursor left” to add a matching character.
This way, simple lists can be specified via juxtaposition:
a b c
And then more complex lists
thing 1, thing 2, thing 3
and still more complex lists like
A complex thing; with data, and more data
can be specified in a way that is potentially still human-legible and easily editable.
Combined with ~instant feedback while typing and, ideally, a “brushing” system to allow selection of parts of the textual model via the linked drawing, I am hopeful that this can be solved resiliently, at least for the most common use cases.
(Part of why I am excited about OP’s work here though is that while I have done a fair bit in my own project on drawing a related kind of diagrams, I have myself only begun thinking about how to make the resulting drawings nicely stylable/themeable.)
IMO there are always going to be tensions (as well as differing personal preferences) for surface syntax and authoring environment.
What I think may emerge though is a better shared understanding of “what’s needed” as well as a great (ideally libre) library for implementing those ingredients consistently and interoperably — e.g., a DOM/CSSOM for these graphs as well as maybe an adaptagrams- or cairo-like library for the common layout and drawing capabilities.
(For example, have you noticed how many general-purpose drawing tools now have “align and distribute” capabilities that all work similarly? AIUI, many of those tools are either directly using or are inspired by Kim Marriot, PJ Stuckey, Tim Dwyer, & colleagues’ work in this area?)
This looks like a really nice attempt to improve the comfort and capability of styling drawings of small graphs that I’m excited to explore; thanks for building and sharing it!
Total control is good for small charts and for especially important charts but for complex uses, the number of clicks and drags required to perfectly position connectors really rapidly degrades the editing experience, especially if you’re still working out the contents of what needs to be depicted.
As a result, for either high-speed or larger scale usage, my experience is that text (in combination with constraint-respecting automatic layout options that suit one’s personal aesthetic) really starts to shine.
In my experience, typing dot requires a lot of quoting and brace-matching, often in editors without the best affordances for those operations, with so-so error reporting.
The result is that it’s very challenging to type syntactically correct dot at “line-speed”, e.g., if one is taking notes.
(This is one of several reasons that I think there is definitely still room for improvement in notation for graph-drawing, as I have begun exploring in my own work.)
You might find it helpful to distinguish between visualizing the design of the system being implemented by your software, visualizing protocols being implemented by your software, visualizing the design of your software implementation itself, and visualizing important implementation details at runtime, e.g. for debugging, profiling, and operations.
For visualizing system designs, you should take a look at STAMP, e.g., via “Engineering A Safer World” + the resources at mit.edu/psas + on YouTube.
(Multiple tools, both commercial and libre, exist and are being developed to make these diagrams, although for what it’s worth, I mostly hear about people making them using draw.io, Google Drawings, on physical paper/whiteboards, or occasionally with specialized tooling.
I have also recently published a project in this area, https://github.com/mstone/depict, which I believe is well on its way toward addressing some unmet needs here.)
For visualizing protocols, things like sequence diagrams, data flow diagrams, DRAKON flow charts, value stream maps, and occasional more specialized objects like CPSA “cryptographic protocol shapes” / strand space skeletons are where I start depending on the flavor of what’s needed.
For visualizing the design of implementations themselves, I have not yet seen anything that I feel obliged to recommend; rather, here, I suggest investing in adding illustrations to your existing documentation in whatever way is easiest for you to use to clarify whatever subtleties you need to clarify for your audience.
(Here I tend to look at things like ASCII-art, SQLite’s railroad diagrams (now made with pikchr, AIUI), and sequence diagrams, as mentioned by other commenters, as helpful examples to start with.)
Finally, for implementing debugging/profiling/operational illustrations, there is a such a rich set of examples to turn to — whether from the very specialized (custom process model video rendering pipelines in robotics) to TensorBoard for TensorFlow to general-purpose tools like browser performance debugging suites, flame charts, or Go’s built-in profile graphing tools - that rather than learn any particular such tools, I’d instead suggest trying to get comfortable with the building blocks underlying these systems, which include contemporary GUI/web apps, custom drawing and animation tools like SVG, pretty printers, and Grammar-of-Graphics systems like vega-lite.
(Note: although it may seem superficially extraneous to your question, the reason I also suggest thinking about debugging visualizations in this context is because IMO, to work, they ~necessarily encode a visual model of the design of your implementation since it is the design of the implementation that provides the vocabulary and relationships that have to be understood and navigated in order to successfully debug/optimize/monitor any given running instance of whatever system you are building.)
Doing all sorts of things will introduce you to all sorts of problems; from there, it’s just a matter of habitually looking around to try to see who/what has already worked on whatever problem is currently in front of you.
For example, I first found minion when I was learning about scheduling problems, initially to help a friend who was a Chief Resident on a pediatrics ward and later, to automate the administrative hassle of scheduling incident responders. (I wrote about that a little bit many years ago here if you’re curious: https://mstone.info/posts/scheduling/)
Concurrently with that, while I was working with and supervising security researchers, I spent quite a bit of time implementing various program analyses to answer questions about what certain programs of interest might do, and to find inputs that would make them do interesting things. For this purpose, though, minion is much less immediately relevant than Z3 since the interfaces and research interests of the relevant communities are so different.
Finally, this year, I am focusing on problems that have a more conventionally geometric flavor as opposed to the more discrete search spaces mentioned above. Here, convex optimization is a starting point that cvxpy makes incredibly accessible, especially in combination with Stephen Boyd’s Stanford SEE EE364A lectures (or similar): https://see.stanford.edu/Course/EE364A
After several years of exploring, my current gotos are:
* Minion for IP for scheduling + edge crossing minimization.
* cvxpy (wrapping ECOS, OSQP, and SCS by default) for convex optimization for making nice geometry.
* Z3 and STP for SAT/SMT for program analysis.
All are FLOSS, which is my main criterion in many situations.
Beyond that, I like minion for its focus on only providing efficiently implementable primitives, cvxpy for the awesome lectures and documentation the folks behind it have produced, and z3 + stp for their use in tools I like, such as klee.
If it helps, one connection I see is that recommender systems often involve causal questions like: “how will user behavior change if we change the order these results appear in, or if we change which results appear in the first page of results, etc.”. Additionally, since we can only show one ranked set of answers for each query, counterfactual questions also rapidly arise about what would have happened if we had answered past queries differently.
Martin’s writing and thinking has been hugely inspirational for me too — thanks for highlighting the automerge work here!
Next, regarding your other thoughts, two notes, one on performance and one on “common themes”.
Re performance: while in my initial comment I linked directly to McSherry & friends’ work on differential dataflow mostly because I value it as a principled and effective framework for incrementalizing many kinds of computation I care about, I would be remiss if I failed to also highlight here their complementary “COST” project of using this framework to demonstrate how carefully engineered (single-threaded!) implementations of graph computations in Rust can often considerably outperform more complex but more wasteful pre-existing approaches (at least in the database world).
(Whether the same flavor of result will also appear over time in the build systems or UI worlds seems like an interesting question for future work!)
Finally, re “common themes in incremental systems”: yes! - this and your point about “fundamental principles” are exactly what I was hoping to suggest by linking to the build systems + OT materials!
Lovely post, and an interesting problem area. Three additional perspectives that have been on my mind in this area:
1. Differential dataflow. Like Adapton and Incremental (which Raph cites), I have found DD to be hugely helpful for understanding which computations can be efficiently incrementalized, and how (including in Rust).
2. Operational transforms. Often times, I am working on UI as part of a larger distributed/networked system. In these cases (especially when I already want the UI to display other people and their actions), the division of responsibility between the UI, the network code, and the various internal models has always felt like a missed opportunity for better and more principled integration / compositionality.
3. Build systems. The problem of figuring out how to bring a UI efficiently up to date in response to some parts of the underlying state changing can also be seen as a build system, or cache coherency problem. From that standpoint, where do the paradigms of the Build Systems a la Carte paper fit into Raph’s pictured theory of reactive UI?
Zeynep Ton’s research: https://www.zeynepton.com/book finds that several supermarkets appear to have found ways to overcome low margins to provide good jobs by making several strategic choices differently, thereby suggesting that low margins cannot be the essential problem (but leaving open the possibility that the “race to the bottom” behavior you describe could be contributing).
I am a happy user of differential dataflow (DD), which has been great for simplifying an app that I care about.
The app in question already includes a form of the connected components computation Frank is describing improvements to in this post so I’m excited to see whether these ideas help make things work even better.
For a deeper look from the systems engineering side, check out http://mit.edu/psas, specifically the book-length treatment in “Engineering a Safer World”.
(As background, the author, MIT Prof. Nancy Leveson, summarizes decades of work in the field, offers groundbreaking new theoretical tools that scale up to some of the world's most complex accidents, and has the experience and evidence to back up their relevance e.g. via work on Therac-25, the Columbia Space Shuttle, and Deepwater Horizon to name just a few...)
This way, simple lists can be specified via juxtaposition:
a b c
And then more complex lists
thing 1, thing 2, thing 3
and still more complex lists like
A complex thing; with data, and more data
can be specified in a way that is potentially still human-legible and easily editable.
Combined with ~instant feedback while typing and, ideally, a “brushing” system to allow selection of parts of the textual model via the linked drawing, I am hopeful that this can be solved resiliently, at least for the most common use cases.
(Part of why I am excited about OP’s work here though is that while I have done a fair bit in my own project on drawing a related kind of diagrams, I have myself only begun thinking about how to make the resulting drawings nicely stylable/themeable.)
[1] https://mstone.info/depict/ -> https://github.com/mstone/depict