Yes, Rash has a variety of limitations. Let me give some more context to these:
>1. no job control
Racket is missing a feature in its rktio library needed to do job control with its process API, which Rash uses. At one point I added one or two other minor features needed for job control, but I ran out of steam and never finished the final one. It's a small feature, even, though now I don't remember much of the context. I hope I wrote enough notes to go back and finish this some day.
>2. multi-line editing is limited
I always intended to write a nice line editor that would do this properly. But, again, I never got around to it. I would still like to, and I will probably take a serious look at your line editor some time.
The design was intended as something to use interactively as well as for scripting. But since I never improved the line editing situation, even I only use it for scripting. After documentation issues, this is the most pressing thing that I would fix.
>3. from what I understand, shell syntax is available only at REPL top level. Once you switch to Lisp syntax with `(`, you can return to shell syntax only with `)`. Thus means you cannot embed shell syntax inside Lisp syntax, i.e. you cannot do `(define j {find -type f | less})`
As mentioned is not correct, you can recursively switch between shell and lisp.
>4. shell commands are Lisp functions, not Lisp objects. Inspecting and redirecting them after they have been created is difficult
This one is a design flaw. I've meant to go back and fix it (eg. just retrofitting a new pipe operator that returns the subprocess pipeline segment as an object rather than its ports or outputs), but, of course, haven't gotten around to it.
>5. Rash is written in Racket, which has larger RAM footprint than schemesh running on vanilla Chez Scheme: at startup, ~160MB vs. ~32MB
Yep.
>6. Racket/Rash support for multi-language at REPL is limited: once you do `#lang racket`, you cannot go back to `#lang rash`
So actually `#lang` is not supported at all in the REPL. It's neither supported in the Racket REPL nor the rash repl. In practice, what `#lang` does is (1) set the reader for a module, and (2) set the base import for the module, IE what symbol definitions are available. With the REPL you have to do this more manually. The repl in Racket is sort of second class in various ways, in part due to the “the top level is hopeless” problems for macros. (Search for that phrase and you can find many issues with repls and macros discussed over the years in the Racket mailing list.) Defining a new `#lang` in Racket includes various pieces about setting up modules specifically, and since the top level repl is not a module, it would need some different support that is currently not there, and would need to be retrofitted for various `#lang`s. But you can start a repl with an arbitrary reader, and use `eval` with arbitrary modules loaded or symbols defined. My intention with a rash line editor would also have been to make some infrastructure for better language-specific repls in racket generally. But, well, obviously I never actually did it. If I do make time for rash repl improvements in the near future, it will just as likely be ways for using it more nicely with emacs rather than actually writing a new line editor... we'll see.
I'm always sad when I think about how I've left Rash to languish. In grad school I was always stressed about publication (which I ultimately did poorly at), which sapped a lot of my desire and energy to actually get into the code and make improvements. Since graduating and going into industry, and with kids, I've rarely felt like I have the time or energy after all of my other responsibilities to spend time on hobby projects. Some day I would like to get back into it, fix its issues, polish it up, document it properly, etc. Alas, maybe some day.
The docs are terrible, and I've always meant to go back and improve them... but it's never quite been a priority and I've never actually summoned the motivation to do it...
But it's always nice to hear when someone uses it and likes it despite that!
I think the parent poster agrees with you, but wrote somewhat ambiguously. I think he meant something like “a movie that was visually very striking, [but] without any plot fundamentals[,] that [therefore] felt like a deep betrayal to the universe.”
For what it's worth, I agree with him. When I saw the movie and even just the promotional materials I thought it was visually striking and had very strong color themes. But wow, it was a train wreck in terms of plot, characters, faithfulness to the series, etc. I could go on for hours.
But to be fair, I also think The Force Awakens was terrible and painted the story into a dumb direction. Instead of “what if the Nazis came back to power in Argentina”, they should have moved the story into a direction more like “the alliance against a common enemy is fractured”, like what actually happened after World War 2, or “there are now many factions of ambitious warlords rising among the widely deployed and still incredibly powerful imperial military”, or some of both. The Mandalorian did the setting much better in that sense.
While I agree that adding fuzzy searching and natural language discoverability are great, there are some other pretty important differences.
I thought a lot about this line:
> you had the best of the terminal paired with the best of graphical interfaces
Maybe there are many parts of a terminal that are “the best” for different people or use cases, but to me the key feature of the terminal is the ability to compose, extend, and abstract. Use a command with certain options a lot? Turn it into an alias or one-liner script. Use a pattern of commands a lot? Turn it into a script. Use a particular command really a lot? Add it as a custom keybinding. And it's pretty easy -- once you've paid the price of learning to use the shell at all, you are ready to start writing basic scripts. Granted, the price of entry to learn the shell is high compared to command palettes. But there is so much there that command palettes are still missing.
I wish I could upvote this 1000 times. Most firewalls are just so much security theater cargo culting. NAT and universal firewalls have done immeasurable, and at this point practically irreversible, damage to network freedom and free software.
Since everybody else is peddling their better shells, I'll drop a plug for mine: https://rash-lang.org
It's a shell embedded in the Racket programming language. If you're familiar with Xonsh, it's similar, but both more powerful and less polished. It allows easy, recursive mixing of shell code (not posix compatible, but with a similar feel) and Racket. Its interactive mode is not polished (I need to write a better line editor), but it works, and it's great for programming.
I haven't worked on it much lately (I need to wrap up other things to finish my PhD), but this weekend I added support for user-programmable substitutions with automatic cleanup. As an example, beyond common substitutions like process substitution (IE <() and >() in bash), I added a demo “closure substitution” form that allows you to send Racket functions to `find -exec`. Importantly, this is something that a user could add, because Rash is super extensible and malleable.
Shell is a great DSL, but there are huge advantages to having an embedded DSL rather than a stand-alone DSL. Rash inherits all the cool features of Racket and can be mixed with other Racket languages (eg. Typed Racket, Honu, etc), has advanced features like first-class delimited continuations and the world's most advanced macro system (which makes Rash possible), and any shell script can import functions from Racket's catalog of third-party packages. Embedded shells like Rash allow a shell script to be copied from interactions like you do with Bash, but then grow past the “rewrite in Python” stage gradually with no rewrite -- just a gradual transition from being more shell code to being more “normal” code.
Though I'm kicking myself for continually not getting around to rewriting all the documentation, which is still very poor.
For several years previously I ran Arch Linux on laptops, Debian on servers, and a mix of the two on various desktops. I'm generally much happier with NixOS. NixOS-style package management is the future, it's clearly better than all its competitors (I put Guix together with Nix as NixOS-style here). NixOS-style system configuration is also much nicer than most OSes, though the quality of configuration design for different components varies a lot, and I think there could be big improvements to the configuration language and APIs in future OSes that try to improve on the model (largely due to NixOS being the first system of its kind). I'm happy that Guix is also exploring this space and making some different decisions, and it's the only other OS that I would seriously consider right now. All in all, I'm very pleased with NixOS.
As to `#lang python` interop with Racket, it's possible, but the farther a language's semantics and object model differ from Racket, the harder and less pleasant it is to build and use in practice.
In other words, you need converters between runtime values between Racket and Python when their semantics differ.
For example, Python strings are instances of Python's string class, with a bunch of methods, and you can mutate the string class at runtime.
They are very different from Racket's strings.
Then you have all the differences in core reduction semantics, how concurrency and parallelism is handled, etc, and there are a lot of differences to smooth over.
There is a project called PyonR (https://github.com/pedropramos/PyonR) that implements Python 2.7 as a Racket #lang, but it hasn't been developed for a while, and I'm not sure how interoperable it actually is.
It certainly isn't to the point where you could pull in your favorite Python package from PyPI to use in Racket, but it does claim that you can use Racket functions in your Python code at least.
The hardest part just using PyPI packages is probably that many of the most important Python packages lean on Python's C FFI, which is a large extra hurdle, though there are many worthwhile packages that are pure Python code.
The most successful Racket #langs are languages that fit a particular domain or style of programming better than basic Racket, but that lean on Racket's core semantics and object types as a building block rather than something to be constantly slightly different from in ultimately unimportant ways.
While working with Racket, these languages may be very different from the core Racket language, such as documentation languages (scribble), logic languages (datalog), shells (rash), database query languages, etc.
If you just wanted to let people write new code for their editor in a language that looks and feels like Python alongside Racket and its #langs, it would be a much easier project to make a language with Pythonic syntax (syntax is shallow), easy constructors for Python-like lists and such (just a few custom datatypes), and a Python-like way of providing generic operators (eg. __magic_method__), etc, but otherwise with semantics closer to Racket's.
Frankly, you would end up with a better language than Python anyway, with powerful features that most Pythonistas haven't dreamt of. (Bias revealed!)
But I do dream of having a `#lang python` implementation good enough to just use Python packages in a pinch when Racket doesn't have an equivalent package.
It's possible, but it's a big enough project and there is little enough manpower in the Racket world that nobody is seriously working on it.
I intend to write at least a small version of this.
There are a couple of existing “Emacs-in-Racket” projects which I may borrow code or ideas from.
The older one is https://github.com/tonyg/rmacs, which runs but is not in a state where it's really usable for anything, and there is a newer https://github.com/soegaard/remacs which I have never gotten to run on a Linux system.
Essentially my plan is to make an Emacs-like editor with a line-editor interface for my embedded-in-Racket shell, Rash (https://rash-lang.org).
Of course, I will also make a TUI interface, like terminal Emacs, and while I probably won't write it myself, I would leave it open to have a GUI interface as well, and generally be embeddable in arbitrary Racket programs that need an editor component.
While I don't have the time or will to really make something that could compete with Emacs as a general-purpose editor, or seriously try to implement a `#lang elisp` to give Emacs compatability, I think building an Emacs-like editor in Racket with an actual function that people will use (better line editing for Racket, Rash, other Racket languages) could give people enough reason to hack on it themselves that it could grow and be useful at least as a line editor.
Additionally, I have some ideas on plugin sandboxing I would like to try out.
In other words, while I would obviously just have an interpreter with which you could run arbitrary code to modify the editor, like Emacs but with Racket code in any #lang, I also may add an API using Racket's sandboxing features so that you can run extensions with limited, explicit privileges.
I want to be able to run extension code written by internet randos without auditing it, but trust that it won't steal my ssh keys and such.
That said, I'm trying to finish my PhD and I haven't even done any direct work on Rash itself for a while.
My advisor says “your editor should be your first side project after graduating”, and I agree with him.
So my Emacs-in-Racket may yet be eternal vaporware, and even if it isn't, advanced features that I want might not make the cut when I actually determine how much time I can commit to it.
We'll see.
At any rate, even if other people do it first and better, I'm likely to make a small version of this some day just because I want to -- I've already written my own terminal emulator in Racket (which works but is slow, and I don't recommend anybody use it), and my own shell in Racket (which is awesome, and I do recommend people use it, at least for scripting, and eventually I will develop it further [contributors welcome!]), so naturally I need to make my own editor as well.
One of my favorite features of Racket[1] is that each file declares its dialect on the top line with `#lang language-name`. Different dialects, such as custom embedded DSLs, can have slight syntax modifications or even their own custom lexer and parser. In the end Racket modules all boil down to a smaller core language and interoperate beautifully.
>You'll regret it if you ever think about playing with BSD, POWER9, or the platform of tomorrow
But the platform of tomorrow is clearly a resurrected Lisp Machine! In all seriousness, though, I'm pretty sure Racket runs on BSD, and it would probably not be that hard to get it working on any Unix. (In fact, portability is one of the main thrusts right now in core Racket development.)
>Your Racket-based shell requires a supported Racket platform, which is honestly a pathetic group: Linux, macOS, and Windows, on x86_64, i686, and ARM
>It's not about you: it's about everyone.
The same could be said about pioneering new architectures. I think you're doing great work on eg. getting RISC-V available on your build platform. But by the time I will realistically buy a RISC-V machine to replace the server I run most of my shell scripts on a lot more software will be ported to it, probably including Racket. I'm under no delusions that the masses will be flocking to my shell, but there are so many shells that are better than Posix for (at least what I perceive to be) most shell scripting. If any of these better shells becomes very widely used I'm sure it will be ported to $platform_of_tomorrow (oops! I mean "$platform_of_tomorrow"!) before it is very widely deployed.
>A comfortable interactive shell experience is a good thing, and has nothing to do with POSIX.
At least we agree there!
As an aside, I think your projects (sway, wlroots, sr.ht) are really cool, and I like your blog, despite our disagreement about things like sticking to Posix shell and C. Cheers!
> it's pretty damn presumptuous to make someone install some random shell on my machine to build your code, imo
Is it more presumptuous to make people install a language you used than to make them install a library you used? Either way people need to get your dependencies if they want to run your code.
> The standard exists to define the environment.
Standards are great, but not all environments need to be standard. Most shell scripts don't need to be ported to dozens of different platforms, and most shell script authors would be much better served by a language with fewer weird gotchas, saner semantics, and the benefit of a lot of language research that's happened in the past few decades. For instance, as I've been porting my personal shell scripts to Rash (my shell in Racket), I've been able to use all the patterns I use in posix shells but with benefits of better readability, easier reasoning, better means of abstraction, etc. It's been a solid win that I would recommend to anybody. Sure, if someone else wants to run my scripts they need to install Rash. But few people (in practice: nobody) are really interested in running my personal shell scripts besides myself. If somebody does want to run them, installing the dependencies is not very hard.
There is a small set of scripts that needs to be portable and be written in a shell-like language. Until standards catch up, posix is perhaps the best thing we have. But most of the time when people write shell scripts they could be writing in any language but choose shell because they want to automate something they do manually or they really want a processes-and-files DSL. Choosing a better shell that fits the processes-and-files domain and allows easy interactive use with better features and less insanity in these situations is a slam dunk.
Some other interesting projects around the strict posix shell are Morbig[1], a more trustworthy static parser for the posix shell, and the work of Michael Greenberg[2], who is trying to formalize the semantics of the posix shell.
But my personal takeaway from this sort of research is that the posix shell is a truly insane language beyond even its readily apparent shortcomings. It's worthwhile to figure out the exact semantics of a language with so much existing code and have a precise interpreter for it, but frankly I think the more important task is to move away from such terrible languages. Bourne-derived shells have some great ideas, but these could be formalized into more sensible languages. In particular, I think the best direction is to embed shell languages inside good general-purpose languages. I'm biased to that end since I've written a shell embedded in Racket. But there is no shortage of better shell languages that benefit from better language design, such as Powershell and Elvish.
Rash only forks to launch processes, and it's all behind Racket's `subprocess` function. So there is no notion of eg. a forked subshell. But the `subprocess` function lets you get the subprocess file descriptors (as a special kind of port) or redirect subprocess IO to the ports of your choice. While pipeline segments that use Racket functions can pass each other arbitrary objects rather than just doing byte-level IO, pipeline segments that are just subprocesses do traditional Unix-style pipeline IO. And you can redirect a pipeline's input/output/error streams to arbitrary ports.
You can run pipelines in the background with Rash, but I haven't implemented job control yet, so I don't have proper handling for eg. auto-suspending a background TUI process and terminal control. (This ends up not being that big of a deal for most use, I find, though I intend to write better job control eventually.) I don't remember what Vim does off the top of my head, but I think running processes blocks Vim. Background pipelines don't block Rash, but if you don't specify to run one in the background the default behavior is to cause the current thread to wait for it to finish.
Hey, I'm the creator of Rash. I just noticed that this was posted here. This thread got no traction and zero comments, but I'll post this for the benefit of... I don't know, people who search through Hacker News history. I think it's worth noting that while Rash isn't totally stable yet, it's getting fairly close, and I've been using Rash as my daily-driver shell for most of a year now. It's a pleasure to script with -- often I start with some commands that I'll want to run, and dump them verbatim in the script, but then write the bulk of the scaffolding around them using normal Racket forms. It was designed with the goal of being convenient for interactive use (IE useful for those who live in the shell day-to-day like me) as well as being a scripting language, and bridging the gap from shell scripting to “serious” programming. Think “scripts to programs”, a phrase of the gradual typing crowd, extended to be “interactions to scripts to programs”.
I'm late to the party, but I'm the author of Rash. Rash's documentation is terrible and out of date, but I've actually been planning to redo/improve all of the documentation within a week or two. I'm hoping that once I do that it will be much more approachable. Rash tries to allow both Bash style things and more normal Racket to be done with ease and mixed, and a lot of the complexity of Rash has to do with getting those two things to work together, and to allow lots of macro extensibility. But I think you'll find that it's quite powerful and useful... once I make it easier to make sense of.
Of course I'm also happy for anyone to make a new shell that they like and want to use themselves. I did it, after all...
Yes, actually I've looked over all the shells on that wiki page. I think most of them haven't taken off because they either their host language has been poor or unpopular, their design or implementation wasn't great, or they haven't really solved the right problem. I think the idea of embedding a shell into a general-purpose language still has a lot of merit. Most of those projects trying to embed a shell DSL into a general-purpose language, scsh included, are basically for programming and not interactive use. Shells that are only interactive or only for programming end up fulfilling less than half of the purpose of shell, in my view, because the interaction and scripting feed back into each other in a virtuous cycle. The only ones on that list aside from Rash that try to be embedded in a general-purpose language while being good at both interactions and programming are Xonsh and eshell. Xonsh is pretty new (I wasn't aware of it until after I had made Rash), and eshell is in Emacs Lisp (which is not a very good programming language or platform to build on for anything except extending the emacs editor).
Rash also tries to be better than the competition by adding object pipelines (much like Powershell, it makes it much more reasonable to make system administration commands in the host language, and have rich interaction and inspection of command results), user-defineable pipeline operators, and generally tighter integration with the host language while still having a light syntax for basic commands.
I would like to be able to have my command language tightly integrated with my programming language, and be able to define my system administration commands (and interactive shell plugins and completion) as functions in the host language (while still being able to naturally use external programs). And I would like my shell scripts to be able to grow and potentially even turn into full-fledged programs, or modules of some larger program. I think there are a lot of benefits to the approach I'm using (which would be too long for a comment here).
That said, I'm not holding my breath for it to catch on widely any more than I'm holding my breath for Racket to take off as a popular programming language (although I frankly wouldn't mind either one). I think a better Posix shell is certainly a noble effort, because whatever better shell does become popular, we certainly need one. And an automatic upgrade path for existing scripts sounds great. So I salute you and wish you good luck with it. Also, as someone looking at shells and their features to copy the best ones, your wiki is a great resource. So thanks.