Beyond Bash: Shell scripting in a typed, OO language(docs.google.com)
docs.google.com
Beyond Bash: Shell scripting in a typed, OO language
https://docs.google.com/presentation/d/11vZzXCfAA0aOFAuHA0nAvAzALGFGCH-dqHxx6XMgbk8/edit#slide=id.g646dd4f7f_0_128
15 comments
I have a large multi-concern shell script for building a web-app. The "complexity" seems to impress my workmates, but I dislike it and avoid extending it if at all possible. Something like this might vastly improve the expressiveness, and scalability.
If you don't mind using python, the sh package [0] can really get you a bash-style repl (dinamically) typed.
.. [0]: http://amoffat.github.io/sh/
.. [0]: http://amoffat.github.io/sh/
Yes, in my mind this is the way to go (the other alternative, probably unviable, being scsh https://github.com/scheme/scsh). I recently made a decision not to write any bash scripts that include a conditional or loop; for all those use cases python+sh is the way to go. The reason: http://mywiki.wooledge.org/BashPitfalls
I can't think of any environment where python would not be available. You can even ship sh.py as a single-file dependency along with your script, so `import cp from sh` will work as expected.
I can't think of any environment where python would not be available. You can even ship sh.py as a single-file dependency along with your script, so `import cp from sh` will work as expected.
Haha! Just reading this thread I got the idea to replace bash with something scheme-like. You have a shell which works like it usually soes, but that easily let's you write any utilities in scheme.
I use scheme to write most of my system stuff (most revently a custom automounter for my raspberry Pi) and thought it would have been a nice project. You could probably go macro galore and make it quite comfortable to do piping and integrate the regular system utilities in the language.
I use scheme to write most of my system stuff (most revently a custom automounter for my raspberry Pi) and thought it would have been a nice project. You could probably go macro galore and make it quite comfortable to do piping and integrate the regular system utilities in the language.
There's also a bash-like Python REPL like Xonsh ⌘ http://xonsh.org/
Still in early development, but very useful.
Still in early development, but very useful.
That's Powershell in the Windows world.
Powershell has some pitfalls of its own (I still much prefer it to Bash despite these):
1. Dynamic scoping
2. Squashing lists of size 0 to nulls and
size 1 to single elements when they come out of pipes.
3. Incomprehensible API for XML handling.
Powershell is exactly what bash should be (and this person wants). Their time would have better spent using and improving the pash project (open source Powershell on Mono): https://github.com/Pash-Project/Pash
Powershell is primarily a repl to a homogeneous .NET-based set of tools, which doesn't describe Unix very well. If you want a type-safe repl that interacts with all the handy tools written for the text-stream-only Unix world you're going to have to do some interpretation.
Li Haoyi has been doing a lot of cool open source Scala work. Ammonite is an example of that. You can find it here https://github.com/lihaoyi/Ammonite
I for one would not to pay the cost of starting the JVM up every time I want to invoke a shell script.
Some people do their shell scripting in a pure, functional, strictly typed language [0] — and I don't judge them!
[0] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
[0] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
That's cool! We should probably join forces. After all, Martin Odersky has stated that moving people onto Haskell was the primary motivation for inventing Scala, which will soon be renamed Hascalator
https://twitter.com/myatlyuka/status/577625111575969792
At the same time, Typesafe is apparently being rename Reactive Banana, which is more evidence of the increased Haskell focus within the community
https://www.reddit.com/r/scala/comments/36ho4u/breaking_type...
EDIT: found a better slide for Hascalator
https://twitter.com/myatlyuka/status/577625111575969792
At the same time, Typesafe is apparently being rename Reactive Banana, which is more evidence of the increased Haskell focus within the community
https://www.reddit.com/r/scala/comments/36ho4u/breaking_type...
EDIT: found a better slide for Hascalator
My friends answered immediately: «Saying that Scala is a gateway drug to Haskell is like saying that poor-quality heroin cut with speed is а gateway drug to LSD».
Shell languages are defined by their terseness.
The author misunderstands how important this is: https://docs.google.com/presentation/d/11vZzXCfAA0aOFAuHA0nA...
The author has created a radically "better" shell language which is twice as verbose as bash. In the space of human-computer interaction where shell languages lie, this is tantamount to total failure.
The worst parts of shell are the flow control structures (if/else...). If only we could improve this without throwing out the entire system. Else we will lose the benefits of the ultra-terse and completely decentralized "language" that is unix shell.
The author misunderstands how important this is: https://docs.google.com/presentation/d/11vZzXCfAA0aOFAuHA0nA...
The author has created a radically "better" shell language which is twice as verbose as bash. In the space of human-computer interaction where shell languages lie, this is tantamount to total failure.
The worst parts of shell are the flow control structures (if/else...). If only we could improve this without throwing out the entire system. Else we will lose the benefits of the ultra-terse and completely decentralized "language" that is unix shell.
Beautiful response to a recurrent misunderstanding.
"Shell languages are defined by their terseness."
It is common to see people put a layer of verbosity on top of the Bourne shell (or system(3)) to make "a new shell".
It is also very common to see people put a layer of abstraction on top of a large, verbose scripting language and claim the result to be a new, "terse" language (with all the power of the shell).
But what I like about the Bourne shell is that it is built from only C. And it does not add too much verbosity. It is, as you say, defined by its terseness.
There is also terseness in the roff-like typesetting languages, assembly languages, FORTH, k/q, etc.
Compared to k/q, sh is verbose.
For me, verbosity means loss of power and loss of time.
I am glad there are terse languages.
They may never again be popular but I think they will always exist.
"The worst parts of the shell are the flow control structures..."
For some reason I dislike if/then/else. Instead I make heavy use return values, ||, test(1) and case/esac.
I always wished Bourne's shell (cf. Joy's C shell) made use of more C operators. There's a file called arith_lex.l in the Almquist sh source but these operators are not used in the sh language.
I would switch to the Plan 9 shell but Bourne sh remains more useful due to its ubiquity.
"Shell languages are defined by their terseness."
It is common to see people put a layer of verbosity on top of the Bourne shell (or system(3)) to make "a new shell".
It is also very common to see people put a layer of abstraction on top of a large, verbose scripting language and claim the result to be a new, "terse" language (with all the power of the shell).
But what I like about the Bourne shell is that it is built from only C. And it does not add too much verbosity. It is, as you say, defined by its terseness.
There is also terseness in the roff-like typesetting languages, assembly languages, FORTH, k/q, etc.
Compared to k/q, sh is verbose.
For me, verbosity means loss of power and loss of time.
I am glad there are terse languages.
They may never again be popular but I think they will always exist.
"The worst parts of the shell are the flow control structures..."
For some reason I dislike if/then/else. Instead I make heavy use return values, ||, test(1) and case/esac.
I always wished Bourne's shell (cf. Joy's C shell) made use of more C operators. There's a file called arith_lex.l in the Almquist sh source but these operators are not used in the sh language.
I would switch to the Plan 9 shell but Bourne sh remains more useful due to its ubiquity.
This is mostly true, and it doesn't look like this is the solution. My belief is any solution needs to be built from the ground up as a language for shell use, not adapting Python/Scala/Haskell to shell use.
However, there is some non-zero amount of verbosity that would be worth trading for better control flow, composability, data structures, etc. As long as you can do simple operations and pipes with the same syntax
Programmers don't live in bash, and deal with the pain points of switching between bash and an actual programming language on a regular basis. I know I'm there. A change that sometimes is more verbose but lets you do more things in shell would still have benefits.
You might not win over the people who are bash experts, but you could still produce a better tool for many people.
However, there is some non-zero amount of verbosity that would be worth trading for better control flow, composability, data structures, etc. As long as you can do simple operations and pipes with the same syntax
| grep foo
cp * path/
rm -rf
Programmers don't live in bash, and deal with the pain points of switching between bash and an actual programming language on a regular basis. I know I'm there. A change that sometimes is more verbose but lets you do more things in shell would still have benefits.
You might not win over the people who are bash experts, but you could still produce a better tool for many people.
The author explicitly states terseness is a goal, and compares alternatives' overhead to bash. However, he recognises there is a limit whilst remaining safe/close to Scala.
I don't agree control structures are a major problem. There are subtle differences between correct and incorrect expressions, and they are not checked at "compile" time.
I don't agree control structures are a major problem. There are subtle differences between correct and incorrect expressions, and they are not checked at "compile" time.
I didn't check it out yet, but I think Tulip (http://www.jneen.net/posts/2015-07-29-tulip-language-updated) has a great potential in this space.
How about an interpreter that executes from an AST, which can also re-generate formatted source in two forms, "revealing" and "terse"?
One should be able to also write a debugger that can switch between these modes in real-time.
One should be able to also write a debugger that can switch between these modes in real-time.
Cool stuff. I really like the idea, but why so many symbols? On slide "2.5 Piping" I find the description on the right to be far more readable than the examples on the left. Also the descriptions on the right are familiar, idiomatic Scala which is also a plus. The stuff on the left is impossible to decipher without some documentation and I don't see how it adds value except from saving a few chars. OTOH it introduces significant load on the developer that has to look them up (again and again) in the manual. If saving these chars are important then one might defend introducing a single new operator for some well thought-out abstraction, but introducing 7 new operators is just to much for anyone to remember. Especially the two last operators seems entirely redundant as the language already has quite compact syntax for function invocation (as does any language).
When I see this unifying of language logic and shell ~logic, as a Lisp head, I can't help but to think of lisp machines:
https://www.youtube.com/watch?v=o4-YnLpLgtk
https://www.youtube.com/watch?v=o4-YnLpLgtk
Wasn't this one of the points of Perl? New languages seem small, but tend to grow.
Path normalization does not work like that. If `/bin` is a symlink referring to `/usr/bin`, then `/bin/..` is the same as `/usr`, not `/` as the author suggests.
Congratulations on making something cool and useful.
But I think it's not Bash vs Ammonite, it's more accurately (Alternatives to Bash) vs (The Ammonite alternative to Bash).
I've never worked in an environment that uses sh for large systems. My use is just my daily conversation with my OS and file system. I say things on the command line. Sometimes I say certain things in sequence often enough that I collect them into personal "jargon" scripts of two to twenty lines of Bash. For anything larger I generally use Python.
My dividing line between Bash and Python (or any similar language) is conversation vs contract. A conversation is relatively informal. Anything larger requires negotiation and agreement by parties (me and the OS) using technically understood formal language (to strain the metaphor).
I think Ammonite is more like the contract than the conversation, but maybe not as much as Python.
But I think it's not Bash vs Ammonite, it's more accurately (Alternatives to Bash) vs (The Ammonite alternative to Bash).
I've never worked in an environment that uses sh for large systems. My use is just my daily conversation with my OS and file system. I say things on the command line. Sometimes I say certain things in sequence often enough that I collect them into personal "jargon" scripts of two to twenty lines of Bash. For anything larger I generally use Python.
My dividing line between Bash and Python (or any similar language) is conversation vs contract. A conversation is relatively informal. Anything larger requires negotiation and agreement by parties (me and the OS) using technically understood formal language (to strain the metaphor).
I think Ammonite is more like the contract than the conversation, but maybe not as much as Python.
https://github.com/valvesoftware/steam-for-linux/issues/3671 is a nice motivating example of why I think this project is useful
Using a lot of shell can really be dangerous. When you are on the ops side you know how to avoid most of this pitfalls by using tools like Ansible, Puppet or Chef.
If anything is going to displace bash at this point it's going to be something that brings substantial benefits to the user. Stricter syntax is good but not of the immediate concern to the user typing his 2x-longer command.
The next evolution of shell is going to be something different. Something that doesn't depend on an antiquated TTY protocol, something that can display images and other constructs more complex that text, and more importantly something that allows the user to accumulate commands instead of typing the same thing over and over again.
Maybe something like ipython/jupyter
The next evolution of shell is going to be something different. Something that doesn't depend on an antiquated TTY protocol, something that can display images and other constructs more complex that text, and more importantly something that allows the user to accumulate commands instead of typing the same thing over and over again.
Maybe something like ipython/jupyter
There's the shell language and the shell interface. The design of those things influence each other, but they're fundamentally separate even though bash/zsh/fish often make changes to both.
The original article is really about languages, and I think that even with the best interface in the world, we want a better language.
The original article is really about languages, and I think that even with the best interface in the world, we want a better language.
Just changing the language is in the diminishing return area and is why it hasn't happened yet. Something else need to change.
Any sort of shell programming is going to be full of side-effects. No matter how strict the language, building the script will still involve trying on the target system, maybe accumulating shell commands in a script, and then have these long full cycles where the script is tested on a fresh system to make sure it's still working. This is happening every time I do any sort of sysadmin-type of stuff. Even in ansible, chef, docker, ...
To escape that something deeper needs to be changed.
Any sort of shell programming is going to be full of side-effects. No matter how strict the language, building the script will still involve trying on the target system, maybe accumulating shell commands in a script, and then have these long full cycles where the script is tested on a fresh system to make sure it's still working. This is happening every time I do any sort of sysadmin-type of stuff. Even in ansible, chef, docker, ...
To escape that something deeper needs to be changed.
Using languages other than shell already brings lots of benefits to the user. That just doesn't matter to the people who are writing large programs in shell. It isn't like those people are basing their decision on rigorous user studies.
PowerShell
Pretty cool... more documentation and examples here: http://lihaoyi.github.io/Ammonite/