ZX – A tool for writing better scripts(github.com)
github.com
ZX – A tool for writing better scripts
https://github.com/google/zx
142 comments
Here's an example: https://gitlab.com/vincenttunru/penny/-/blob/main/scripts/bu...
That used to be a shell script. But this I can way more easily read (even after not touching it for a long time), I get autocompletion, and I can use the APIs I'm already familiar with. It's a pretty neat QoL improvement for me specifically.
That used to be a shell script. But this I can way more easily read (even after not touching it for a long time), I get autocompletion, and I can use the APIs I'm already familiar with. It's a pretty neat QoL improvement for me specifically.
Wouldn't the `await fs.move` run sync? Why not just return the Promise from fs.move() so that Promise.all() can run the multiple moves async?
await Promise.all(
unneededPages.map(async (page) => {
await fs.move(page, `scripts/.cache/${page}`, { overwrite: true });
})
);
Do you mean the `await fs.move` on line 13? Because yes, I do need that to complete before I can run the next command, the build, on line 40.
I could have appended it to the Promises that I'm creating on line 9 to run it in parallel with those moves, but this is not performance-critical code, and moving the two conceptually-unrelated concepts into the same expression wouldn't aid legibility, in my opinion. And since I'm the only one who needs to read the code, that opinion wins by default :)
I could have appended it to the Promises that I'm creating on line 9 to run it in parallel with those moves, but this is not performance-critical code, and moving the two conceptually-unrelated concepts into the same expression wouldn't aid legibility, in my opinion. And since I'm the only one who needs to read the code, that opinion wins by default :)
Above: You have `await Promise.all()`. Remove the `await` in front of `fs.move()`.
You don't need the second `await` for blocking purposes.
Sure, it is your code to do as you like, but if you're going to post it as an example, you might want to correct it first so that others don't make the same mistakes.
You don't need the second `await` for blocking purposes.
Sure, it is your code to do as you like, but if you're going to post it as an example, you might want to correct it first so that others don't make the same mistakes.
Ah! You mean that
Edit: though maybe I'm misunderstanding given that you said it would make them run sync, because as far as I know, the functions would still run in parallel? Essentially, I believe it would roughly be the equivalent of this:
await Promise.all(
unneededPages.map(async (page) => {
await fs.move(page, `scripts/.cache/${page}`, { overwrite: true });
})
);
await Promise.all(
unneededPages.map((page) => {
fs.move(page, `scripts/.cache/${page}`, { overwrite: true });
})
);
Edit: though maybe I'm misunderstanding given that you said it would make them run sync, because as far as I know, the functions would still run in parallel? Essentially, I believe it would roughly be the equivalent of this:
await Promise.all(
unneededPages.map((page) => {
fs.move(page, `scripts/.cache/${page}`, { overwrite: true });
}).then(result => result)
);What’s with all the await-stuff? Couldn’t it be written just straight forward?
That allows other code to run while e.g. files are being moved. It indeed can be somewhat painful ([1] is the classic criticism) and in shell scripts is not as beneficial. I think there are even sync alternatives (i.e. equivalent functions that don't need `await`) for most of them, but again: these are the APIs I'm already familiar with, which is the entire point - and thus the await-stuff comes naturally for me, because I intuitively know those APIs are async.
[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
Yes and a shell script would do it with one &, while this goop is a mountain of async, nesting, futures management, and boilerplate. No thanks.
Sure, but again: I know and recognise this syntax and the APIs. What's boilerplate to me, I hardly see, whereas compact Bash operators are noise to my eyes.
(Though to be honest, I'm also somewhat sceptical that a single & in Bash would be sufficient to move a bunch of files in a particular order, and restore them properly if the process gets terminated somewhere halfway along the process. But what I'm sure about is that I wouldn't have been able to write that in a reasonable amount of time!)
Maybe it should say that somewhere.
> It looks like people don’t get it.
They're probably on the spectrum.
(I'll get my coat...)
They're probably on the spectrum.
(I'll get my coat...)
This looks really cool, actually, and I'm surprised I haven't heard of this before.
My incorrect gut intuition is that JavaScript devs usually don't do very much in depth terminal level command work, but that's of course false on a Bayesian level. There are far more JavaScript developers in the world than there are even Python developers, to say nothing of the long tail of even less popular languages. If 30% of JS devs are heavy terminal users, and 80% of (say) Perl devs, that 30% will still swamp the 80% in terms of absolute numbers.
Hell, that's half the point of Node. Can't believe my gut isn't keeping up with my head here!
My incorrect gut intuition is that JavaScript devs usually don't do very much in depth terminal level command work, but that's of course false on a Bayesian level. There are far more JavaScript developers in the world than there are even Python developers, to say nothing of the long tail of even less popular languages. If 30% of JS devs are heavy terminal users, and 80% of (say) Perl devs, that 30% will still swamp the 80% in terms of absolute numbers.
Hell, that's half the point of Node. Can't believe my gut isn't keeping up with my head here!
> There are far more JavaScript developers in the world than there are even Python developers
I don’t think that’s true. Anecdotally when I was interviewing at a recruiting company (and we let candidates pick any language to be assessed in), Python was by far the most popular choice, chosen by about 60% of candidates or something like that. Java and JavaScript came after, followed by a long tail of languages like C# and go.
I assumed JavaScript would be the most popular too - but that didn't seem to be the case. I have no idea where the demand for all this Python programming is coming from, but it seems Python programmers are everywhere. Maybe it’s just popular in schools.
I don’t think that’s true. Anecdotally when I was interviewing at a recruiting company (and we let candidates pick any language to be assessed in), Python was by far the most popular choice, chosen by about 60% of candidates or something like that. Java and JavaScript came after, followed by a long tail of languages like C# and go.
I assumed JavaScript would be the most popular too - but that didn't seem to be the case. I have no idea where the demand for all this Python programming is coming from, but it seems Python programmers are everywhere. Maybe it’s just popular in schools.
If you need a programming language for you job, but you're not primarily a developer, and two languages would probably be overkill, then your one language is probably python.
That's a lot of people.
That's a lot of people.
That’s exactly how I chose Python.
I'm curious, what was the job that needed some code but not a dedicated developer?
I'm taking computational biology courses on the side, so that's the application that comes to mind for me.
I'm taking computational biology courses on the side, so that's the application that comes to mind for me.
I do digital marketing for clients. There’s a large number of activities that can be made more efficient, quicker, cheaper, or better using code.
For example taking a large number of text documents and turning them into video outlines with illustrations using OpenAI APIs and a quick and dirty Django app to organise everything.
Or connecting up various marketing APIs using Python instead of Zapier - a lot of companies spend thousands of dollars on Zapier each month when a script would literally cost a dollar to run.
For example taking a large number of text documents and turning them into video outlines with illustrations using OpenAI APIs and a quick and dirty Django app to organise everything.
Or connecting up various marketing APIs using Python instead of Zapier - a lot of companies spend thousands of dollars on Zapier each month when a script would literally cost a dollar to run.
Python is just fastest to write. No semicolons, fewer parens, no braces, no const / var / let. List / dict / set comprehensions. Lambdas are clunkier, but `def` is shorter than `function`. Etc.
Perl and Ruby are comparably compact and expressive, but relatively few people know Ruby, end even fewer people know Perl, especially among those younger than 50.
BTW Haskell is also syntactically compact and expressive; I once solved a toy interview problem using it. But even fewer people know it at a practical level, among interviewees and interviewers alike.
Perl and Ruby are comparably compact and expressive, but relatively few people know Ruby, end even fewer people know Perl, especially among those younger than 50.
BTW Haskell is also syntactically compact and expressive; I once solved a toy interview problem using it. But even fewer people know it at a practical level, among interviewees and interviewers alike.
I’m proficient and confident in many languages because I like solving problems with computers. However in a high-stakes and low-tooling situation like a job interview I wouldn’t choose JavaScript any day of the week because of the uncertainty, inconsistency, and footguns it likes to sprinkle everywhere.
There is absolutely a role for JavaScript, it truly does run everywhere which is no small feat. But that’s about all it’s good at.
What I’m getting at is that when you give people a choice they are going to optimize for the scenario, so you have a heavy selection bias in your anecdote.
There is absolutely a role for JavaScript, it truly does run everywhere which is no small feat. But that’s about all it’s good at.
What I’m getting at is that when you give people a choice they are going to optimize for the scenario, so you have a heavy selection bias in your anecdote.
Is it a recruiting company, as it's main business is to recruit candidates ? Or was the core business something else and you were recruiting devs for it ?
I think it's always difficult to get the bigger picture of the dev market through candidates. Many (rightfully) handle their career moves by themselves and will directly apply to companies that fit them, heavily biasing the movement observed from the company or third party recruiters.
I think it's always difficult to get the bigger picture of the dev market through candidates. Many (rightfully) handle their career moves by themselves and will directly apply to companies that fit them, heavily biasing the movement observed from the company or third party recruiters.
Main business is (was) recruiting people for roles all over the place.
I could probably say "you're moving the goalposts here, I'm taking all developers not all working developers" or something, but I think these days you're basically correct. Python was relatively more niche when I picked it up in 2009 or so, but it's really become the lingua franca of our industry.
Maybe a lot of things start in Python, because so many people know it, and no one wants to change languages after a project starts?
Python also has a very wide range of applications so while there are many languages more popular in a particular area, Python is extremely widely used in aggregate. By contrast it is unusual to write numerical code in PHP or web apps in C.
Python also has a very wide range of applications so while there are many languages more popular in a particular area, Python is extremely widely used in aggregate. By contrast it is unusual to write numerical code in PHP or web apps in C.
According to Stack Overflow JavaScript is more popular.
https://survey.stackoverflow.co/2023#section-most-popular-te...
https://survey.stackoverflow.co/2023#section-most-popular-te...
Python seems to be popular for solving leetcode style interview questions, which might be something worth of note.
Here's an anecdote to help your gut. I'm a platform engineer who's favorite language is Typescript. I LIVE in the terminal. The only reason I use vscode is cause it's easier to put him into vscode than it is to put vscode into vim. Otherwise I'd be in the terminal for everything but browsing.
Likewise! I use TypeScript everywhere possible, and zx is a godsend. Unfortunately, I've recently dealt with gobs of Python for ML software. After years of writing TypeScript, it's been a...sobering reminder of many painful coding issues that I'd forgotten even existed. Made me deeply appreciate what TS gives us haha.
I worked at a Python shop for a bit. I spent more time than I wish to admit trying to make all of my Python behave like TypeScript (adding typings, etc.).
Ditto. Python(3.13) types are so crudimentary that it becomes a Sisyphian struggle. Library type support is patchy at best. **kwargs is an abomination. Python is incredibly slow. Async code and anonymous functions are awkward and verbose. Hell, Python is verbose.[1] I could complain endlessly, but I'm mainly just disappointed in how little has changed since I last professionally wrote Python. The biggest benefits were basically a) familiarity, and b) powerful stats/ML libs.
I hadn't seen analogs of `AttributeError`s and `TypeError`s for years, until revisiting Python and remembering that those are still unsolved issues outside of TypeScript. Ugh.
[1] https://news.ycombinator.com/item?id=36407112
I hadn't seen analogs of `AttributeError`s and `TypeError`s for years, until revisiting Python and remembering that those are still unsolved issues outside of TypeScript. Ugh.
[1] https://news.ycombinator.com/item?id=36407112
Python now has type hints which makes it closer to typescript, and easier to code in.
Would a python developer feel the same in reverse?
If they'd go to JavaScript, they'd feel a sense of unfamiliarity similar to what JS developers going to Python would feel.
But the Python equivalent of TypeScript is mypy, and I don't think it's controversial to say (even among Python developers) that it's way behind TypeScript, generally.
But the Python equivalent of TypeScript is mypy, and I don't think it's controversial to say (even among Python developers) that it's way behind TypeScript, generally.
My big use case for this is that it's quite good for glue scripting when you're already in the context of a JS shop. My common use cases are tooling setup, CI scripts, and API response sanity checking.
"[For writing complex scripts] JavaScript is a perfect choice".
Not sold on that. Why should I choose it over Python, PHP or others?
Not sold on that. Why should I choose it over Python, PHP or others?
yea same here. I wrote nodejs programs for years. the focus on the eventloop and everything being asynchronous is great for servers and network IO but does NOT make it the perfect choice for scripts. Its a different use case entirely.
scripts are supposed to be small simple programs that run in sequence and terminate. I shouldn't need to deal with concurrency primitives at all in that particular situation.
scripts are supposed to be small simple programs that run in sequence and terminate. I shouldn't need to deal with concurrency primitives at all in that particular situation.
With modern JavaScript (as in last 5-7 years) it's pretty straightforward to write sequential scripts.
There's more to it than writing a simple sequential python, ruby or perl script. You have to write everything using async/await.
async/await are dead simple. And many Node libs have synchronous versions. It's pretty straightforward if you are familiar with the language. If you aren't, well that isn't really the fault of JS nor any different from bash or python, which can both be just as painful for those unfamiliar.
> scripts are supposed to be small simple programs that run in sequence and terminate. I shouldn't need to deal with concurrency primitives at all in that particular situation.
I suppose bash pipes are a concurrency primitive, but I don't object to them. I think it's more about appropriate primitives.
I suppose bash pipes are a concurrency primitive, but I don't object to them. I think it's more about appropriate primitives.
Modern js is not what it was. Ergonomics of nodejs v20 are not in the same ballpark as node v6. A lot has happened since v12 even.
Js definitely has become a pleasant language to script in. And it's sooooo much faster that python and Ruby.
Js definitely has become a pleasant language to script in. And it's sooooo much faster that python and Ruby.
> And it's sooooo much faster that python and Ruby.
That advantage becomes vanishingly small when the language task is to glue together a bunch of spawn/fork/exec to external processes.
If the glue code somehow needs actual heavy processing, that processing can be factored out into another program dedicated to that, leaving only glue code. Side effect: the heavy processing code immediately becomes composable!
That advantage becomes vanishingly small when the language task is to glue together a bunch of spawn/fork/exec to external processes.
If the glue code somehow needs actual heavy processing, that processing can be factored out into another program dedicated to that, leaving only glue code. Side effect: the heavy processing code immediately becomes composable!
Still the same footguns for the sake of backwards compatibility. Things like:
Yes, it's fast but so is Lua, Dart or even Racket.
typeof '' == 'string'
false
Yes, it's fast but so is Lua, Dart or even Racket.
wha?
Welcome to Node.js v19.9.0.
Type ".help" for more information.
> typeof '' == 'string'
true
Can I qoute my comment verbatim back at you?
Seriously. Just don't use ==. Use ===. Et voila you're using a modern pleasant scripting language.
Modern guides on js won't ever mention ==. As a noob today you won't ever learn it. In a big project there will be a linter to stop you
Seriously. Just don't use ==. Use ===. Et voila you're using a modern pleasant scripting language.
Modern guides on js won't ever mention ==. As a noob today you won't ever learn it. In a big project there will be a linter to stop you
Especially because these languages are only one package/install away and not two. I don‘t really get for which audience is targeted here. Usage in JS projects maybe, but then why not write it as npm tasks. ..
I‘m playing around with dotnet-scripts [1] at the moment (C# shop mainly) and this has the same issue imho. The reason why I looked into it was because we have developers not accustomed to bash etc. I still find it silly and would rather use ruby so…
[1] https://github.com/dotnet-script/dotnet-script
I‘m playing around with dotnet-scripts [1] at the moment (C# shop mainly) and this has the same issue imho. The reason why I looked into it was because we have developers not accustomed to bash etc. I still find it silly and would rather use ruby so…
[1] https://github.com/dotnet-script/dotnet-script
> why not write it as npm tasks.
Do you mean writing the entire script in a single line in a string in a JSON file (package.json)? Of do you mean that string just calling out to a non-inline script? Because the use case for this is very much that latter script - it will often still be started via `npm run`.
Do you mean writing the entire script in a single line in a string in a JSON file (package.json)? Of do you mean that string just calling out to a non-inline script? Because the use case for this is very much that latter script - it will often still be started via `npm run`.
It's asynchronous, which is pretty neat
A script, by its very nature, is synchronous. You run, you get a result. If you want a service, don't use a script.
That’s a completely arbitrary limitation which should not exist. When my scripts contain independent non-instant subprocesses, I run them in parallel. I’m also using js/ts to write all my scripts, because why wouldn’t I use a proper programming language instead of that “bash” thing that can’t even handle its single datatype well and resorts to all sorts of gibberish to perform trivial operations on it.
This is pedantic to the point of being conversationally hostile.
How? Pipes are not synchronous
[deleted]
> Why should I choose [JavaScript] over [...] PHP [...]?
LOL dude
made my day
LOL dude
made my day
PHP is a surprisingly capable scripting language. Some relatively recent version is installed by default on a lot of Linux distros and its standard library has tons of functions for dealing with strings, files, and the network.
This [pun very much intended]
Where's the pun in that?
I’m assuming it’s this https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... (no pun intended)
`this` in javascript
I really really enjoy using zx.
As a dev it lets me get back to a good scripting language, where I process input, run other programs, emit output. Powerful & succinct.
As a tool maker, it helps me show my work. Many of the common tasks like running an script or calling fetch will by default log what's happening in clear format, pass through the activity. It's fantastic to write these devtools that do a bunch of tasks, but are showing their work at every step. If something goes wrong, the person immediately sees the evidence of what was ran & what the output was. It helps close the iteration loop very effectively.
Zx makes such a huge difference, while being little more than a couple small good ideas to help node.js be more shell-scipty. Good stuff.
I haven't tried Dax, another scripting system atop js, but I see zx glowingly mentioned in it's recent submission. https://news.ycombinator.com/item?id=39315689
As a dev it lets me get back to a good scripting language, where I process input, run other programs, emit output. Powerful & succinct.
As a tool maker, it helps me show my work. Many of the common tasks like running an script or calling fetch will by default log what's happening in clear format, pass through the activity. It's fantastic to write these devtools that do a bunch of tasks, but are showing their work at every step. If something goes wrong, the person immediately sees the evidence of what was ran & what the output was. It helps close the iteration loop very effectively.
Zx makes such a huge difference, while being little more than a couple small good ideas to help node.js be more shell-scipty. Good stuff.
I haven't tried Dax, another scripting system atop js, but I see zx glowingly mentioned in it's recent submission. https://news.ycombinator.com/item?id=39315689
An immediately visible problem with zx is that wrapping commands in $`` makes it harder to copy and paste them back and forth to the shell to test them. Did that slowed you down sometimes or added an occasional bug?
Suspect there is one in any language. Here's python's option:
https://sh.readthedocs.io/en/latest/index.html
https://sh.readthedocs.io/en/latest/index.html
When my shell scripts get too large, I usually switch to Perl. Still best in class at handling text processing with a few shell commands here and there.
I'm doing that with Ruby, and tgrought the years, it became also true for scripts that are fetching data, iterating over JSON objects, etc.
What makes Ruby great for this is you can specify dependencies inline
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'json', require: false
gem 'nap', require: 'rest'
gem 'cocoapods', '~> 0.34.1'
endI do the same, in Python. I do my best to keep it zero-dependency to maximise portability.
No way. If your script requires something more complex than what is difficult to implement or maintain in bash, then it is no longer just a script. Regardless of how small the resulting application may be, you should treat it as an application and not merely a script. Node.js is not typically used in the system layer where Bash scripts are commonly employed. No single administrator would replace Bash with Node.js. For parallel execution and asynchronous operations, tools like ansible exist, along with python, which administrators are already familiar with.
ZX?
Is there a rainbow bar over the bottom right corner?
Do words appear when you press each key?
Is there a rainbow bar over the bottom right corner?
Do words appear when you press each key?
I just feel relieved that the majority here also seems to think this doesn’t make sense
It makes most sense if you're already using js and you need cross platform support in your scripts.
Neither is bash great, nor is JavaScript a prefect choice, and the examples are pretty underwhelming with all the inconvenient escapes and keywords
Maybe the example doesn't do justice but it looks more complicated than simply using "&" and "wait".
Yup, it seems the perfect example to wonder if a language with async by default is appropriate for the use cases of bash scripts.
The example runs 7 commands and has to explicitly await for 4 of them. The other 3 can run in parallel and it awaits for the end of all of them, which is fair.
The typical bash script is sequential and I fear that its zx replacement will be an exercise of writing await, await, await.
I'd go with an await by default language and wrap parallel code in whatever that language offers to wait for the end of child processes.
The example runs 7 commands and has to explicitly await for 4 of them. The other 3 can run in parallel and it awaits for the end of all of them, which is fair.
The typical bash script is sequential and I fear that its zx replacement will be an exercise of writing await, await, await.
I'd go with an await by default language and wrap parallel code in whatever that language offers to wait for the end of child processes.
Please don’t make me install Node.js to run your shell script.
Yeah that wouldn't be great.
But:
This feels to me like it might be a really useful tool to do systems-integration work around an app that is built in Node.js. For example, to integrate with incrond or cron, or get a supervisor task into JS. The smallest sensible "shell-ish" wrapper that gets you access to your codebase in JS and stops the coding effort being divided so much between technologies.
But:
This feels to me like it might be a really useful tool to do systems-integration work around an app that is built in Node.js. For example, to integrate with incrond or cron, or get a supervisor task into JS. The smallest sensible "shell-ish" wrapper that gets you access to your codebase in JS and stops the coding effort being divided so much between technologies.
Sure. But I can easily see this becoming the default way to write shell scripts in the near future in the same way Electron is becoming the default way to write desktop applications.
Just use nix-shell
Very nice. You need three clicks to get tho the documentation, yet they don't explain anything. What does the const line do ?
The best thing however is that it is not supported by Google.
The best thing however is that it is not supported by Google.
> What does the const line do ?
To declare variables?
> The best thing however is that it's not suppported by Google
That's the case with many things like zx.
To declare variables?
> The best thing however is that it's not suppported by Google
That's the case with many things like zx.
If I want to write better shell scripts I usually run shellcheck and adjust accordingly or if I need facilities not provided by the shell I switch to a full fledged programming language. And oh yes, `sh` is present almost on every BSD and Linux box for free so I consider it an important thing to at least be comfortable with.
shellcheck: https://www.shellcheck.net/
shellcheck: https://www.shellcheck.net/
>Write your scripts in a file with an .mjs extension in order to use await at the top level. If you prefer the .js extension, wrap your scripts in something like void async function () {...}().
Shebang scripts shouldn't have a file extension in the first place, so requiring one here is a dealbreaker. And having the file extension be used to switch between two different incompatible source formats is really a terrible design.
Shebang scripts shouldn't have a file extension in the first place, so requiring one here is a dealbreaker. And having the file extension be used to switch between two different incompatible source formats is really a terrible design.
I think this is a Node.js limitation, not a design choice by ZX.
You could perhaps use a different JS runtime that doesn't have these admittedly esoteric file extension rules.
You could perhaps use a different JS runtime that doesn't have these admittedly esoteric file extension rules.
node --input-type=module
> Shebang scripts shouldn't have
Should not have or should not need to have? You can perfectly write a script with a shebang and an extension, stick it in your path, and use it without extension.
Therefore, I fail to see how this is a "dealbreaker". Or is it just because "I don't like it!"?
Should not have or should not need to have? You can perfectly write a script with a shebang and an extension, stick it in your path, and use it without extension.
Therefore, I fail to see how this is a "dealbreaker". Or is it just because "I don't like it!"?
So how am I supposed to put such a script in $PATH? e.g. if I hypothetically want to make a replacement for say 'ls', what am I supposed to do, force people to type 'ls.mjs' rather than 'ls'?
No, that's my point, every single shell I've ever used would resolve "ls.mjs" if you called "ls" and it's in your path...
ln is your friend
> And having the file extension be used to switch between two different incompatible source formats is really a terrible design.
Isn't that what all file extensions do? Distinguish between incompatible formats?
Isn't that what all file extensions do? Distinguish between incompatible formats?
I've very rarely seen, say, image viewers which can't handle a JPEG with a .png extension or vice versa, but they're badly designed. Most handle this from the file magic automatically. Keying on extension is definitely an antipattern IMO.
I don't see people naming C++ files .c++03, .c++11, .c++17. I don't see people naming Perl files .pl-strict-mode.
Yes, it's normal for people to use file extensions, but having programs impose forced usage and semantics to them is a crutch and a sign of bad design.
How am I supposed to put a script written using this tool in $PATH?
This bad design seems to be a thing in the TS/JS ecosystem now. At least some language ecosystems (Deno? Not sure.) seem to be encouraging people to do 'import ".../Foo.ts"'. Which is abysmal: Firstly these are URLs, yet file extensions are against web principles and good URL design. Secondly they couple a consumer of a module to what language the module was written in and expose this as part of an API, which is terrible design.
I don't see people naming C++ files .c++03, .c++11, .c++17. I don't see people naming Perl files .pl-strict-mode.
Yes, it's normal for people to use file extensions, but having programs impose forced usage and semantics to them is a crutch and a sign of bad design.
How am I supposed to put a script written using this tool in $PATH?
This bad design seems to be a thing in the TS/JS ecosystem now. At least some language ecosystems (Deno? Not sure.) seem to be encouraging people to do 'import ".../Foo.ts"'. Which is abysmal: Firstly these are URLs, yet file extensions are against web principles and good URL design. Secondly they couple a consumer of a module to what language the module was written in and expose this as part of an API, which is terrible design.
> How am I supposed to put a script written using this tool in $PATH?
Separate file that invokes sh or bash and calls the script file with zx and the path to the mjs file.
Should do it.
Though I agree with your objection. The requirement for specific extension while also suggesting a shebang, breaks conventions and expectations. From what I can tell from comments made, it's done that way because they prefer it. Which... is a bit silly. But to each their own.
Separate file that invokes sh or bash and calls the script file with zx and the path to the mjs file.
Should do it.
Though I agree with your objection. The requirement for specific extension while also suggesting a shebang, breaks conventions and expectations. From what I can tell from comments made, it's done that way because they prefer it. Which... is a bit silly. But to each their own.
Extensions are predefined mapping to the executing program. There is no real concept of compatibility there. Useful for media files where you e.g. pick your image-viewer-of-choice, less useful when the script file already has a very strong opinion as to the expected executing binary. Compatibility can be hinted at, but really only determined on the application level.
If the script runtime binary then looks at the filename and does something differently based on it, it's likely not correct, and if it is done, it'd better be because of some necessity. The "we prefer it this way" is not a necessity. Plus it breaks conventions and expectations.
Enforcing file name extensions at execution, because you want to help an editor figure out how to lint or highlight the file, is also, arguably, letting a specific use case affect something it shouldn't.
If the script runtime binary then looks at the filename and does something differently based on it, it's likely not correct, and if it is done, it'd better be because of some necessity. The "we prefer it this way" is not a necessity. Plus it breaks conventions and expectations.
Enforcing file name extensions at execution, because you want to help an editor figure out how to lint or highlight the file, is also, arguably, letting a specific use case affect something it shouldn't.
GPT4 makes it so easy to write or translate scripts, I don't care if you write it in js or bash or powershell or batch files or python or ruby or whatever. Whatever is most convenient at the time is best, and it can be very easily translated if something else is preferred later.
How does this compare to the recently released Bun Shell?
Bun Shell implements a cross-platform bash-like shell, using builtins for popular commands like `rm`, `cd`, `which`, `ls` etc which support the same flags on Linux, macOS, and Windows. A lot of why we made it was for `bun install` to work on Windows without needing polyfill packages like `rimraf` or `cross-env` in package.json scripts.
zx uses the system-provided shell, which means it isn't cross-platform
For the commands Bun Shell implements as builtins, they typically run something like 10x - 20x faster than in zx on mac/linux because spawning processes is expensive (even more expensive on Windows)
zx uses the system-provided shell, which means it isn't cross-platform
For the commands Bun Shell implements as builtins, they typically run something like 10x - 20x faster than in zx on mac/linux because spawning processes is expensive (even more expensive on Windows)
Bit verbose for me with all that promise/await stuff. Substitution rules probably simpler tho
heh.. this is the engineering culture at Google. You identify a problem that bothers you but perks your interest to solve, and then dedicate 1 day a week to solving it, and then releasing it as an internal library. then get approval to open source it and do so.
I get the utility of it, and probably having everything in the repo of a project in the same programming language is a benefit, when onboarding new people.
but...
Shell scripting has a long and glorious history, filled with incremental improvement in the tooling that have proven themselves over decades. The shell scripting languages themselves and all the awesome tools that come along with them on UNIX.
You are 100% guaranteed to be reinventing the wheel over and over (and poorly) if you write shell scripts without a good understanding of what is available "Off the shelf"
There are great benefits to had by using am appropriate language for the task at hand.
Shell scripting has a long and glorious history, filled with incremental improvement in the tooling that have proven themselves over decades. The shell scripting languages themselves and all the awesome tools that come along with them on UNIX.
You are 100% guaranteed to be reinventing the wheel over and over (and poorly) if you write shell scripts without a good understanding of what is available "Off the shelf"
There are great benefits to had by using am appropriate language for the task at hand.
[deleted]
This feels like it could be learned in an hour and looks so nice. Wonder if it can be used with inline code to process line by line stdin.
Thanks, I hate it.
* I love the idea, and I'm interested to see more progress in this direction.
* I'm not a huge fan of the syntax - for me, the commands are the star of the show, while the async/await is a glue. This syntax emphasizes the async/await while the important stuff get shuffled into quotes.
* Don't make me install node - and especially not system/user-wide packages - to run a script.
Having said all that, this is a neat idea and a cool hack on top of javascript language features.
* I love the idea, and I'm interested to see more progress in this direction.
* I'm not a huge fan of the syntax - for me, the commands are the star of the show, while the async/await is a glue. This syntax emphasizes the async/await while the important stuff get shuffled into quotes.
* Don't make me install node - and especially not system/user-wide packages - to run a script.
Having said all that, this is a neat idea and a cool hack on top of javascript language features.
Related:
Zx 3.0 - https://news.ycombinator.com/item?id=28195580 - Aug 2021 (163 comments)
JavaScript for Shell Scripting - https://news.ycombinator.com/item?id=27072515 - May 2021 (181 comments)
Zx 3.0 - https://news.ycombinator.com/item?id=28195580 - Aug 2021 (163 comments)
JavaScript for Shell Scripting - https://news.ycombinator.com/item?id=27072515 - May 2021 (181 comments)
And tangential:
Nushell (300-400 comments),
- https://news.ycombinator.com/item?id=20783006
- https://news.ycombinator.com/item?id=27525031
- https://news.ycombinator.com/item?id=33419944
Ask HN: Are alternative (oil, nu, etc.) shells usable as daily drivers? (141 comment)
- https://news.ycombinator.com/item?id=34722208
Nushell (300-400 comments),
- https://news.ycombinator.com/item?id=20783006
- https://news.ycombinator.com/item?id=27525031
- https://news.ycombinator.com/item?id=33419944
Ask HN: Are alternative (oil, nu, etc.) shells usable as daily drivers? (141 comment)
- https://news.ycombinator.com/item?id=34722208
And similar:
The Bun Shell - https://news.ycombinator.com/item?id=39071688 - Jan 2024 (225 comments)
The Bun Shell - https://news.ycombinator.com/item?id=39071688 - Jan 2024 (225 comments)
WTH is this, some training program how to write await for no reason? :)
Related story from today:
Dax – Cross-platform shell for Node.js
https://news.ycombinator.com/item?id=39315689 (12 comments)
Dax – Cross-platform shell for Node.js
https://news.ycombinator.com/item?id=39315689 (12 comments)
I would say this is the missing Part to use npm scripts effectively
Well this doesn't make much sense.
If you rightfully find bash inadequate at some point, just switch to the next most natural tool - Python. Nobody's going to be surprised when they see a Python script.
If you start feeling Python is inadequate too, then:
1) You're wrong, just stick to Python
2) If you're so bent on doing away with familiarity, at least get as much leverage as you can in return and use a "proper" language, like Go / Rust. You get static typing and easy to deploy binaries, which are solid benefits.
Trying to use JS here is the worst of both worlds. Nobody expects to see JS in this context and it doesn't bring much to the table.
If you rightfully find bash inadequate at some point, just switch to the next most natural tool - Python. Nobody's going to be surprised when they see a Python script.
If you start feeling Python is inadequate too, then:
1) You're wrong, just stick to Python
2) If you're so bent on doing away with familiarity, at least get as much leverage as you can in return and use a "proper" language, like Go / Rust. You get static typing and easy to deploy binaries, which are solid benefits.
Trying to use JS here is the worst of both worlds. Nobody expects to see JS in this context and it doesn't bring much to the table.
Yes, but this isn't made for general scripting. This project seems to be aimed at JavaScript projects, for example calling zx from package.json/scripts/start
If I had a JavaScript project, and I wanted to migrate my scripts away from Bash, I wouldn't pick Python. Not only would it be another language, I would also have to configure Python to run correctly, ie add it to my Docker image.
Simply installing another NPM package, rather than include an entire other language is a much better and simpler solution.
If I had a JavaScript project, and I wanted to migrate my scripts away from Bash, I wouldn't pick Python. Not only would it be another language, I would also have to configure Python to run correctly, ie add it to my Docker image.
Simply installing another NPM package, rather than include an entire other language is a much better and simpler solution.
Python's ok for this as long as you need 0 third party dependencies and it's a single file. As soon as you want to split into multiple files or import something that isn't part of the standard library then it becomes a total disaster.
I think Deno is a really good option here - better than ZX. You can use third party libraries easily, it works with IDEs, and it's trivial to install.
I think Deno is a really good option here - better than ZX. You can use third party libraries easily, it works with IDEs, and it's trivial to install.
Python has been banned in many places for this sort of thing in my experience. There is just too much shite and baggage associated with python environments for it to be reliably portable between systems. Anaconda that, pip this, 2.x, 3.x, string support. It's a dumpster fire.
I am not saying JavaScript is any better, but python is actively considered harmful for this use case in many places I have been.
I am not saying JavaScript is any better, but python is actively considered harmful for this use case in many places I have been.
in my case im usign groovy scripts with grape dependencies. script is always selfcontained single file. max portability in JVM world. no need to fiddle with dependencies.
Why should a nodejs dev have to pull in python? That is a heavy solution when you can script well in js already.
Honestly this looks better than Python. I hate all the subprocess.run([...], check=True, universal_newlines=True) and then you have the manage the dependency hell of virtualenvs. At least with node the node_modules is right there.
Google is trolling us with the useless use of cat right there in the first actual line of the script.
https://porkmail.org/era/unix/award
https://porkmail.org/era/unix/award
"Bash is great, but when it comes to writing more complex scripts, many people prefer a more convenient programming language."
https://google.github.io/zx/getting-started
I read this and think "1. Bash is not great. 2. Why assume that I want to write complex scripts."
I like simple scripts. I am embarassed by any complexity. It is the mark of a cluttered mind.
As it happens, there have been and still are others that dislike complexity, though they may be greatly outnumbered.
Right now there is another submission in /active on HN titled, "A 2024 Plea for Lean Software. Why Bloat is Still Software's Biggest Vulnerability"
"I want to end this post with some observations from Niklaus Wirth's 1995 paper:
"To some, complexity equals power. (...) Increasingly, people seem to misinterpret complexity as sophistication, which is baffling-the incomprehensible should cause suspicion rather than admiration."
I've similarly observed that some people prefer complicated systems. As Tony Hoare noted long ago, "[T]here are two methods in software design. One is to make the program so simple, there are obviously no errors. The other is to make it so complicated, there are no obvious errors." If you can't do the first variant, the second way starts looking awfully attractive perhaps."
And scripts are not even systems. The idea of intentionally creating "complex scripts" is indeed baffling.
The HN title could be something like "ZX - A tool for writing complex scripts". Or "ZX - A tool for writing better complex scripts". Instead it's "ZX - A tool for writing better scripts".
https://google.github.io/zx/getting-started
I read this and think "1. Bash is not great. 2. Why assume that I want to write complex scripts."
I like simple scripts. I am embarassed by any complexity. It is the mark of a cluttered mind.
As it happens, there have been and still are others that dislike complexity, though they may be greatly outnumbered.
Right now there is another submission in /active on HN titled, "A 2024 Plea for Lean Software. Why Bloat is Still Software's Biggest Vulnerability"
"I want to end this post with some observations from Niklaus Wirth's 1995 paper:
"To some, complexity equals power. (...) Increasingly, people seem to misinterpret complexity as sophistication, which is baffling-the incomprehensible should cause suspicion rather than admiration."
I've similarly observed that some people prefer complicated systems. As Tony Hoare noted long ago, "[T]here are two methods in software design. One is to make the program so simple, there are obviously no errors. The other is to make it so complicated, there are no obvious errors." If you can't do the first variant, the second way starts looking awfully attractive perhaps."
And scripts are not even systems. The idea of intentionally creating "complex scripts" is indeed baffling.
The HN title could be something like "ZX - A tool for writing complex scripts". Or "ZX - A tool for writing better complex scripts". Instead it's "ZX - A tool for writing better scripts".
> I am embarassed by any complexity. It is the mark of a cluttered mind.
It is the mark of a cluttered world.
Sometimes, there is no simple solution. You have to parse this file, you have to use software to do it, and you can't fix up every little quirk in the file's actual grammar versus the grammar you wish the file had. Tag soup HTML is one example, but there are thousands of others, around the world, doing important things for business and education.
Sometimes you get to parse JSON. Sometimes you have to parse SQL.
It is the mark of a cluttered world.
Sometimes, there is no simple solution. You have to parse this file, you have to use software to do it, and you can't fix up every little quirk in the file's actual grammar versus the grammar you wish the file had. Tag soup HTML is one example, but there are thousands of others, around the world, doing important things for business and education.
Sometimes you get to parse JSON. Sometimes you have to parse SQL.
In my opinion any bash script longer than a one liner is complex and should be rewritten in another language. Too many footguns.
For a moment I thought this would have something to do with the ZX Spectrum. I'm disappointed :(
Just another language Google tries to push, it doesn't really solve anything Python/JS/Bash et all can already do.
Remember Carbon who was supposed to replace C/C++ ?
I may be wrong, but I think it will be the same pattern as Carbon here.
“ Bash is great, but when it comes to writing more complex scripts, many people prefer a more convenient programming language. JavaScript is a perfect choice, ”
I am not sure you can call js “convenient “.
I am not sure you can call js “convenient “.
But...why?
“Ruby - A tool for writing better scripts”
Hello everyone, I think I happen to have some experiences to share:
1. Five years ago, I developed my own version of zx/dax version using TypeScript. I call this project TypeShell.
2. In the past three years, I have written x-cmd, implemented by posix shell.
Please visit x-cmd at https://x-cmd.com. It is now close to the public beta version. It is still under construction, and the demo can tell some stories. It tries to enhance posix shell in various ways. Its installation tar.gz package is only 1.02MB (which can be optimized). Even with sh/awk/curl, it can do a lot of things. These features are called modules. And, through the self-contained pkg manager (do not require priviledge to install), x-cmd can gain capabilities from existing binaries and other script projects, becoming increasingly powerful.
---
The main reason I gave up TypeShell project was that the startup time became a problem. At that time, node 8 was mainstream, and node 10 had just become LTS. I remember that even using node 10, the startup time was poor compared to python/perl.
In addition to startup performance, size matters. I work on different servers and must carry a 20MB node/typescript zip package to run js script. This was acceptable to me, but I couldn't help thinking, what if the solution was only 5mb. Then I tried to rewrite it in golang, and got a version with a smaller startup time and size. Unfortunately, as I understood bash more deeply, I found that basic functions could be easily accomplished through bash and curl. When I was halfway through, I realized that bash was not available in some places. I challenge to rewrite it with posix shell and awk. After years of effort, x-cmd is almost ready now.
---
Friends, I'm not saying that tools like zx are not good. I do like to write some scripts using js/ts. I believe pythoners prefer https://xon.sh/ . Perl is interesting. Fish is friendly.
However, I believe posix-shell has its own advantages. The balance among size, code length, and expressiveness. I think the only possible competitors are tcl and perl, maybe lua. But none of the above is included by default in all server/container environments.
---
I wouldn't advise using posix-shell due to its numerous potential issues. I think people should write scripts using the language and library they are familiar with, and do it for fun. Friends of js/ts use zx/bun-sh/..., pythoners use xon.sh, etc.
Engineers can use x-cmd like `x <your-script>.[py|mjs|lua|...]`. Then x-cmd will automatically help you download the script runtime, scripts, and install dependencies before running.
This is the future I have always wanted to achieve.
1. Five years ago, I developed my own version of zx/dax version using TypeScript. I call this project TypeShell.
2. In the past three years, I have written x-cmd, implemented by posix shell.
Please visit x-cmd at https://x-cmd.com. It is now close to the public beta version. It is still under construction, and the demo can tell some stories. It tries to enhance posix shell in various ways. Its installation tar.gz package is only 1.02MB (which can be optimized). Even with sh/awk/curl, it can do a lot of things. These features are called modules. And, through the self-contained pkg manager (do not require priviledge to install), x-cmd can gain capabilities from existing binaries and other script projects, becoming increasingly powerful.
---
The main reason I gave up TypeShell project was that the startup time became a problem. At that time, node 8 was mainstream, and node 10 had just become LTS. I remember that even using node 10, the startup time was poor compared to python/perl.
In addition to startup performance, size matters. I work on different servers and must carry a 20MB node/typescript zip package to run js script. This was acceptable to me, but I couldn't help thinking, what if the solution was only 5mb. Then I tried to rewrite it in golang, and got a version with a smaller startup time and size. Unfortunately, as I understood bash more deeply, I found that basic functions could be easily accomplished through bash and curl. When I was halfway through, I realized that bash was not available in some places. I challenge to rewrite it with posix shell and awk. After years of effort, x-cmd is almost ready now.
---
Friends, I'm not saying that tools like zx are not good. I do like to write some scripts using js/ts. I believe pythoners prefer https://xon.sh/ . Perl is interesting. Fish is friendly.
However, I believe posix-shell has its own advantages. The balance among size, code length, and expressiveness. I think the only possible competitors are tcl and perl, maybe lua. But none of the above is included by default in all server/container environments.
---
I wouldn't advise using posix-shell due to its numerous potential issues. I think people should write scripts using the language and library they are familiar with, and do it for fun. Friends of js/ts use zx/bun-sh/..., pythoners use xon.sh, etc.
Engineers can use x-cmd like `x <your-script>.[py|mjs|lua|...]`. Then x-cmd will automatically help you download the script runtime, scripts, and install dependencies before running.
This is the future I have always wanted to achieve.
Wow, this is a very interesting idea
For me, setting up the environment and installing dependencies has always been cumbersome and frustrating
I'll be honest, I did not pass the word Javascript.
And I don't think I could be convinced to.
Didn't expect people to be so hurt by the truth. Sorry. I just don't trust JS running on my machines.
Didn't expect people to be so hurt by the truth. Sorry. I just don't trust JS running on my machines.
[deleted]
The first command of the first example is a cat|grep... How can I assume the author is an experienced script writer?
In any case, ZX replaces some of the shell's usual quirks with its own different quirks: you have to manually expand * with glob() and ~ with os.homedir(), you can't build commands as strings without repeated calls to push(), etc.
In any case, ZX replaces some of the shell's usual quirks with its own different quirks: you have to manually expand * with glob() and ~ with os.homedir(), you can't build commands as strings without repeated calls to push(), etc.
> example is a cat|grep... How can I assume the author is an experienced script writer?
Perhaps because they are writing documentation that can be understood be less experienced script writers as well as themselves, using a very common example seen everywhere?
I find "cat full | ..." keeps the left-to-right flow of a command nicely, and is better understood by those less familiar with shell work than prepending "< file " which I've seen cause confusion.⁰
Yes, it is an extra process. Yes this is less efficient, both due to the fork and the extra IPC along the pipe. But even under Windows where spawning a process is massively more costly than other OSs¹, if you are that worried about this bit of inefficiency² then perhaps you have picked the wrong tool for the job in using a shell/script in the first place?³
> with its own different quirks
I agree there. Though I can see that for some, starting from already using JS via node, this might be an acceptable trade-off.
--
[0] I sometimes also use it in part to wind up the cult of demogification, because they seem to both want to be wound up and deserve to be wound up :-)
[1] Complain to MS about that, not me and my script/one-liner.
[2] Are you using it in a tight loop? Maybe if you are doing something over thousands of files, but then the resources used by the something likely dwarf the cost of cat into irrelevance.
[3] A question that suggests the supplementary: are you an experienced <whatever>?
Perhaps because they are writing documentation that can be understood be less experienced script writers as well as themselves, using a very common example seen everywhere?
I find "cat full | ..." keeps the left-to-right flow of a command nicely, and is better understood by those less familiar with shell work than prepending "< file " which I've seen cause confusion.⁰
Yes, it is an extra process. Yes this is less efficient, both due to the fork and the extra IPC along the pipe. But even under Windows where spawning a process is massively more costly than other OSs¹, if you are that worried about this bit of inefficiency² then perhaps you have picked the wrong tool for the job in using a shell/script in the first place?³
> with its own different quirks
I agree there. Though I can see that for some, starting from already using JS via node, this might be an acceptable trade-off.
--
[0] I sometimes also use it in part to wind up the cult of demogification, because they seem to both want to be wound up and deserve to be wound up :-)
[1] Complain to MS about that, not me and my script/one-liner.
[2] Are you using it in a tight loop? Maybe if you are doing something over thousands of files, but then the resources used by the something likely dwarf the cost of cat into irrelevance.
[3] A question that suggests the supplementary: are you an experienced <whatever>?
> I find "cat full | ..." keeps the left-to-right flow of a command nicely
Sorry, I don't buy it.
You don't read files like this:
Sorry, I don't buy it.
You don't read files like this:
cat file | less
cat hello.c | gcc -xc -
For a single command acting on a file, especially with no other parameters, like your less example, you are right and I wouldn't, though I have seen it often. There is practically no "left-to-right" flow to break. Longer examples is where I would use it.
Also not in circumstances where explicitly acting on a file like your GCC example. It is unlikely that GCC will commonly be used as part of a more collect pipeline of commands.
Note that I said "I use it", not "I always use it, without fail, no other option makes sense ever". Different tools, different jobs (this is what the Church of Latter Day Demogifiers usual don't accept in this context).
Also not in circumstances where explicitly acting on a file like your GCC example. It is unlikely that GCC will commonly be used as part of a more collect pipeline of commands.
Note that I said "I use it", not "I always use it, without fail, no other option makes sense ever". Different tools, different jobs (this is what the Church of Latter Day Demogifiers usual don't accept in this context).
This isn’t “better” than your tool of choice. This is a tool for JavaScript developers to write non-trivial scripts in a portable way without getting lost in POSIX hell. Who came up with this stuff?
https://stackoverflow.com/a/13373256
If you can write shell code, Perl, Python, COBOL, suit yourself. This is for JavaScript developers in JavaScript projects.