Programming at the REPL: Data Visualization(clojure.org)
clojure.org
Programming at the REPL: Data Visualization
https://clojure.org/guides/repl/data_visualization_at_the_repl#_graphical_and_web_based_visualizations
3 comments
This is great! For any SWEs that would be more comfortable in a Python REPL, you can do the same thing like this:
import subprocess
import json
def osquery(query):
return json.loads(subprocess.check_output(["osqueryi", "--json", query]))
#>>> osquery("select * from routes where destination = '::1'")
#>>> [{'destination': '::1', 'flags': '2098181', 'gateway': '::1', 'hopcount': '0', 'interface': 'lo0', 'metric': '0', 'mtu': '16384', 'netmask': '128', 'source': '', 'type': 'local'}]
I always mention that a shell is a poor man's REPL, based on REPL based OSes like the ones from Xerox and ETHZ used to offer their OS interaction workflows alongside the development experience.
Shells like Powershell and Fish are the closest to it, due to their integration with programming language and OS APIs.
Shells like Powershell and Fish are the closest to it, due to their integration with programming language and OS APIs.
I think the time is right to revamp a basic shell. Something that generalizes pipes to async DAGs. The idea floats in the air between reactive objects graph in a browser or systemd services, docker composer ..
It would reseat the base os user interaction to something cleaner and leaner IMO.
It would reseat the base os user interaction to something cleaner and leaner IMO.
Completely agree, having an interactive programming interface to the system is superior to using a shell in practically every way.
The difference is that each of those are their own little silo, instead of being part of a single language ecosystem.
clojure.inspector is an interesting namespace, but in practice I haven't found it very useful.
- It is rare that this problem is the limiting factor. In Emacs, which has horrible problems rendering long lines, usually by the time I realise there is a visualisation problem my editor has died.
- There is a thin band between too-big-to-print and too-big-to-visualise where this tool is useful. Almost all practical data seems to fall outside this range.
That said, Clojure already has some good tools for solving this problem. Immutable data structures and good generic primitives that are easy to inspect make it fun to deal with inspecting objects.
- It is rare that this problem is the limiting factor. In Emacs, which has horrible problems rendering long lines, usually by the time I realise there is a visualisation problem my editor has died.
- There is a thin band between too-big-to-print and too-big-to-visualise where this tool is useful. Almost all practical data seems to fall outside this range.
That said, Clojure already has some good tools for solving this problem. Immutable data structures and good generic primitives that are easy to inspect make it fun to deal with inspecting objects.
> - It is rare that the problem is the limiting factor. In Emacs, which has horrible problems rendering long lines, usually by the time I realise there is a visualisation problem my editor has died.
Sounds like the problem is your editor, not the Clojure namespace :)
Maybe try a real editor, such as vim/neovim ;) It gets a bit bogged down, even on my desktop computer, but at least it never crashes.
Sounds like the problem is your editor, not the Clojure namespace :)
Maybe try a real editor, such as vim/neovim ;) It gets a bit bogged down, even on my desktop computer, but at least it never crashes.
There is nothing wrong with the namespace. I just never found situations where it was useful.
And I see I accidentally wrote "the problem" instead of "this problem". I fixed that.
And I see I accidentally wrote "the problem" instead of "this problem". I fixed that.
As a user of Common Lisp but not Clojure, where does Clojure dev environment stand with respect to something like SLIME? What unique features do Clojure dev environments have?
SLIME was used in the early days of Clojure but is long sinced replaced by Cider (also emacs) or other impressive tooling, i.e. add-ons for IntelliJ (like Cursive). I would say there is nothing that SLIME offers that was not replicated or improved upon in Clojure's tooling.
I'm curious what you would count as improved upon?
Similar to SLIME, there is Cider (https://docs.cider.mx/cider/index.html) for Emacs.
There are a lot of debugging and data visualization tools for the repl with features I haven't found in Common Lisp like :
- https://github.com/jpmonettas/flow-storm-debugger (which I'm actively working on)
- https://github.com/djblue/portal
There are a lot of debugging and data visualization tools for the repl with features I haven't found in Common Lisp like :
- https://github.com/jpmonettas/flow-storm-debugger (which I'm actively working on)
- https://github.com/djblue/portal
These look like nice tools. In CL land I think we can find some of their functionality in other tools than Slime, such as SLY, a fork of Slime with more features:
> - Instrument any Clojure form
This would be Stickers: annotate any Lisp form with a sticker, run your code, interactively walk through the recordings.
https://joaotavora.github.io/sly/#Stickers
> - Provide a GUI to explore your values and execution flow
SLY has an improved trace dialog: https://joaotavora.github.io/sly/#Trace-Dialog Also LispWorks' visual stepper is easier to use than SLY or Slime's. There is also an in-progress portable visual stepper for CL: https://zenodo.org/record/3742759
> time travel stepper
ah. Nice job.
> - Instrument any Clojure form
This would be Stickers: annotate any Lisp form with a sticker, run your code, interactively walk through the recordings.
https://joaotavora.github.io/sly/#Stickers
> - Provide a GUI to explore your values and execution flow
SLY has an improved trace dialog: https://joaotavora.github.io/sly/#Trace-Dialog Also LispWorks' visual stepper is easier to use than SLY or Slime's. There is also an in-progress portable visual stepper for CL: https://zenodo.org/record/3742759
> time travel stepper
ah. Nice job.
Nice, thanks for the links.
Yeah I don't know how stickers trace code, but FlowStorm tracing debugger takes advantage of the immutable data structures used everywhere and by default in Clojure. You can then instrument entire code bases, run them, and tracing is just a matter of retaining(leaking) pointers of every intermediate expression so the GC doesn't collect them. I don't think this is possible when working with mutable objects, since then only way would be to somehow serialize the objects involved at every step so you can analyze the data later, which is prohibitively expensive in most situations.
Yeah I don't know how stickers trace code, but FlowStorm tracing debugger takes advantage of the immutable data structures used everywhere and by default in Clojure. You can then instrument entire code bases, run them, and tracing is just a matter of retaining(leaking) pointers of every intermediate expression so the GC doesn't collect them. I don't think this is possible when working with mutable objects, since then only way would be to somehow serialize the objects involved at every step so you can analyze the data later, which is prohibitively expensive in most situations.
A closer experience to the old Common Lisp commercial IDEs, when using cursive.
https://cursive-ide.com/
https://cursive-ide.com/
I’ve only dabbled with CL/SLIME, but Cider seems to me to be a very close analog (and is similarly very useful).
For example, you can start a Babashka nREPL by running bb --nrepl-server and connect an Clojure editor to it. Then you can do a lot of fun stuff from there. For example, if you have osquery installed, then you can start querying your system for all kinds of stuff, get data back in EDN and manipulate it using standard Clojure functions:
I find I often just leave a bb nREPL running with Calva connected to it and some scripts to do common system tasks in it.