Show HN: Shite – little hot-reloadin' static site maker from shell · HackerTrans

Show HN: Shite – little hot-reloadin' static site maker from shell(github.com)

141 pointsby adityaathalyevor 4 Jahren59 comments

Show HN: Shite – little hot-reloadin' static site maker from shell

https://github.com/adityaathalye/shite

Or more precisely, my little site maker... It is a personal tool that I thoroughly enjoyed making, and enjoy using to write my website <https://evalapply.org>.

A caveat before any more; nobody was supposed to promote this insanity.

Like, terrible things have been done involving inotify and xdotool. But. It showed up on HN some months ago. That too while it was still, shall we say, fermenting. It got "done" some time thence, and of course one could not let the half-past just be. So here we are, for better or worse.

Thank you HN mods for helping me repost! _\\//

The README explains all, animated GIFs and whatnot. Some assorted highlights:

  - shite's, ah, "business logic" (except templates) is about 300 lines of Bash, written in Functional Programming style [^]. Pipeline all the things!
  - The innards won't surprise Perl/PHP/Shell gentleperson hackers from the last century.
  - The local hot reloading workflow is surprisingly nice, and occasionally hilarious! No JavaScript needed.
  - Full rebuilds are low performance and that's fine :)
  - Pandoc is great.
  - Sometimes sed and regex is exactly the HTML parser you need. *Very* sometimes.
  - stdio buffering can mess you up
  - jq -Rr @html # escapes HTML; what?!

... and all sorts of other stuff noted in the README and inline docs.

[^] because shell ain't a bad place to FP... https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp...

---

P.S.The commit history stops at 3-odd months ago, because I've been using a private fork for day-to-day content drafting, publishing, and layout tweaks. The "business logic" is by and large the same as the public version linked here.

17 comments

andrewmcwattersvor 4 Jahren

Hey Aditya, you can make your command a little more unix-y if you feel so compelled by just renaming `shite.sh' to `shite' and run the shell script as a utility since it's already `chmod +x`'d.

Then you can `shite(1)` anywhere you want after you put the command in your PATH and feel like a proper `git(1)`.

Good `shite(1)` mate. Love it.

adityaathalyevor 4 Jahren

Sidelight: I tend to use functions as Unix tools a lot more than scripts as Unix tools. No need for PATH twiddling, file ownership/permission setting, among other things. I wrote about it here: https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp...

freedombenvor 4 Jahren

I do this too! Bash functions are incredible. They have to be the most awesome under-used function (pun intended) I know of

adityaathalyevor 4 Jahren

Ah a fellow person of culture :D

Maybe you will enjoy my "bash-toolkit" repo: https://github.com/adityaathalye/bash-toolkit which I've dubbed my "Swiss Army Toolkit" of functions-as-cmd-line-tools and useful-to-me patterns.

Which reminds me, I've collected more and should update the repo!

adityaathalyevor 4 Jahren

Thank you for the review! If only you knew of all the glorious plans :,)

Apropos your note, may I direct you to "Unrealised Ambitions" in the README https://github.com/adityaathalye/shite#unrealised-ambitions

appletrottervor 4 Jahren

> Apropos your note, may I direct you to "Unrealised Ambitions" in the README https://github.com/adityaathalye/shite#unrealised-ambitions

It doesn't have to be an unrealized ambition with a simple `mv` command ;)

TechBro8615vor 4 Jahren

I like this as a user but it always throws me off when I see it in a Git repo, where I'd prefer the .sh extension to hint at what's inside the file.

adityaathalyevor 4 Jahren

Indeed, this is why I retained the `.sh`.

If it were designed as a proper userspace tool (it is not, and may never be), I might have dropped the file extension. But I might also have chosen to keep the `.sh`, and create an alias instead, during install. That way I get to have both micro-conveniences.

andrewmcwattersvor 4 Jahren

For reference, I think npm does this too, and IIRC, they just symlink the utility name.

chubotvor 4 Jahren

Nice! I also use inotifywait for hot reload when writing https://www.oilshell.org/

I never heard of the xdotool method ... although it sounds like it has some drawbacks? https://github.com/adityaathalye/shite/blob/master/bin/hotre...

I do it with a crappy Python web server that dynamically inserts some JavaScript. This also has bugs and drawbacks, though I don't think any fundamental ones ... At some point I would like to "make it nice", but it has been working well in practice for years

I think the method is reasonable, and doesn't depend on your local desktop environment as much.

I also have a Makefile for incremental rebuilds, but I would really like to replace that with Ninja. Ninja goes pretty well with shell (although it also has the issue of needing to escape $ as $$)

adityaathalyevor 4 Jahren

Lovely to hear from you, Andy! Thank you for sharing. It's good to know this work is in fine company :)

> although it sounds like it has some drawbacks?

Oh yes. The Xdotool Reload Method emulates keystrokes, so it has... drawbacks... as you put it mildly :)

For my use-case, it literally does F5, or it fills out a url and hits Enter. cf. this `case` statement: https://github.com/adityaathalye/shite/blob/master/bin/hotre...

I wanted to keep /mandatory/ moving parts to a minimum in a hand-rolled setup. So I sort of went hardline about avoiding prerequisites like a server process, and especially a JS client. The xdotool trick worked surprisingly well (for the most part), and so I stuck with it!

> it has been working well in practice for years

This is the holy grail. I really hope I get years-long stability out of it. Based on my experience so far, I feel good about the long-term odds.

> I also have a Makefile for incremental rebuilds, but I would really like to replace that with Ninja.

I was going to write a Makefile at some point. I'll have a look at Ninja. Thanks for the tip!

jrm4vor 4 Jahren

Lol, funny, I dig it.

Right here in real life I just do http://zim-wiki.org + a custom CSS Template + a few shell scripts.

http://jrm4.com

adityaathalyevor 4 Jahren

Thanks :)

Likewise, I'm basically locked into "life in plaintext", as a die-hard orgmode user.

With what I have on disk, a big positive is any "system" of organisation is opt-in, post-hoc, and fungible.

Like, today, shite can be seen as merely one lens into a subset of my org files, to compile that subset into a website. Tomorrow I could bring back hugo.

That choice has no bearing on how any other method of organisation might use those same files (e.g. I can pull up index views using orgmode's own query feature set).

And further, a single blog posts's org file can power live demos in Emacs (org-babel), and gets exported as a blog post, and as a PDF slide deck, like this: https://www.evalapply.org/posts/why-and-how-i-use-org-mode/i...

ramrunner0xffvor 4 Jahren

amazing mind-link. i have also built a similar program (also called shite) only centered around plan9's rc shell and ndb, to manage my shite ;) (my use case is building a gemini+html output page with date/tag navigation from mostly plain text + minimal markdown in the form of tags and links).

https://git.2f30.org/shite/files.html

adityaathalyevor 4 Jahren

plan9! Some day I'll try it out. Such a nicely integrated system. As an emacs user, I cracked up at the manpage for Emacs in plan9. Hilarious!

> https://git.2f30.org/shite/files.html

So I looked at the templating code. Have you considered here documents for templating? Being able to declare complete HTML fragments is a big advantage for debugging parts, and keeping overall HTML valid. I author HTML fragments in an HTML-mode buffer and get linting etc. to make sure my markup is OK. Then I can just copy-paste it back into the heredoc.

Another thing I did was to use functions as wrappers for each HTML fragment, so that I could define guarantees in the scope of the HTML (any variable / field that must be present or must be of a certain type).

ramrunner0xffvor 4 Jahren

I haven't really, because the only place where i needed valid html generated mostly by the program is on the navbar , the taglist etc. the main docu is kinda plaintext on a div when rendering html. But i will go back and investigate your idea since it would be elegant and rc supports normal here documents. for example

  ed $3 <<EOF
  g/$1/s//$2/g
  w
  EOF

changes all occurrences of $1 to $2 in file $3. :)

adityaathalyevor 4 Jahren

Yeah, I noticed that `ed` invocation in the rc manual. I'm tempted to find use-cases for it. That way I can lay claim to having used the one true editor :)

In general, I like making programs out of declarative constructs and/or literal data, because it's so much easier to do "local reasoning" with those things, as compared to imperative procedures, or heaven forefend, live objects.

jamal-kumarvor 4 Jahren

This is really impressive, I tried making my own SSG in shell scripts after looking at all these bloated ways to do it and realizing "this is something I could literally do in ten minutes of bash". It works for my use case but you went WAY further with the concept, good stuff

adityaathalyevor 4 Jahren

Thank you! It most assuredly was not "ten minutes of Bash".

You see, there comes a time in every self-respecting Yak-shaving programmer’s life when one is driven to render large-scale follicle trimmage unto a sizeable Yak. In my case, ‘twas and ‘tis the deceptively diminutive but in fact mighty Site-Maker-Yak that renders World Wide Web Sites publishable unto the Internet.

I wager, engaging oneself in the glorious enterprise of Yak-shaving is at least as therapeutic for the trimmer as it must be to the trimmee. I wholeheartedly recommend it :)

jamal-kumarvor 4 Jahren

Yeah the live reload is especially impressive, I was literally just slapping content in between a header and footer and getting away with easy clean results hahaha

masukomivor 4 Jahren

This is amazing and i love it.

As much as I've been enjoying hugo, I've been craving something that's much closer to this.

adityaathalyevor 4 Jahren

Haha thank you.

I confess, I take childlike delight in every hotreload on save. Had it not worked well enough, I'd have stuck with hugo at the cost of a jankier authoring + build pipeline (org --ox-hugo--> markdown --hugo--> html).

082349872349872vor 4 Jahren

One thing I enjoy about shell pipelines is that intermediate processes may terminate them.

eg https://www.99-bottles-of-beer.net/language-bourne-shell-108...

adityaathalyevor 4 Jahren

In fact, I rely on that sort of early termination / circuit breaking as a feature!

That is a big reason why I use functions. They help me define sane fallbacks, and/or enforce API contracts. Those, in turn, are points where I can enforce "fail and die" behaviour.

Design notes: https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp...

Example from shite: Hard stop if mandatory front matter for posts is absent: https://github.com/adityaathalye/shite/blob/master/bin/templ...

edit: clarity

082349872349872vor 4 Jahren

nice ... and the Heredocs are a nice way to template!

adityaathalyevor 4 Jahren

Right?!

It was a bit of a lightbulb moment. I wanted a way to just write HTML templates. What really sealed the deal was the ability to punch content from stdin anywhere within a heredoc's scope using `$(cat -)`: https://github.com/adityaathalye/shite#templating-system

rcarrvor 4 Jahren

This guy also made a bash ssg, no hot reloading though as far as I’m aware:

https://github.com/karlb/karl.berlin

adityaathalyevor 4 Jahren

Ooh, I like what they're doing. Thanks for the pointer!

rltvor 4 Jahren

Nitpick: Generally “hot reloading” refers to reloading a single module or component in the system, while retaining the rest of the page state.

But maybe this counts because it’s a static site.

dborehamvor 4 Jahren

Y'all know that's a word in Scottish English, right?

christophilusvor 4 Jahren

> Before you get too exshited, may I warn you that the MIT license means I don't have to give a shite if this little shite maker fails to make your shite work.

I don’t think they knew that.

satvikpendemvor 4 Jahren

I'm pretty sure that's the joke.

n8henrievor 4 Jahren

What else would it be referring to? Glancing at the top of the readme I don't see eg an acronym spelled out.

jwdunnevor 4 Jahren

Northern English dialect too…

raffraffraffvor 4 Jahren

And Irish

walthamstowvor 4 Jahren

London too

adityaathalyevor 4 Jahren

A Bit of Fry and Laurie, Concerning Language https://www.youtube.com/watch?v=3MWpHQQ-wQg

mrlonglongvor 4 Jahren

And southern too. All shite.

alexchantavyvor 4 Jahren

I really like the dreams and desires section and how it explains the philosophy and how you view the project

adityaathalyevor 4 Jahren

Thank you :)

I started writing a "Goals" and "Anti-Goals" section, as I almost always do for sizeable projects [^]. But then I thought two things, "that's boring", and "waitaminute... when people say 'goals', what do they really want to achieve or fulfill? Dreams and desires.". And so it came to be.

[^] e.g. https://github.com/adityaathalye/clojure-by-example#workshop...

alexchantavyvor 4 Jahren

Yeah, and part of defining what tool is is defining what it is not :)

andrewflnrvor 4 Jahren

I was curious how the hot-reloading works on the browser side. It looks like it starts its own Firefox window, keeps a reference to it, and somehow directly sends an F5 keypress to it? I don't quite follow how that part works.

adityaathalyevor 4 Jahren

xdotool emulates user actions under the X Window System (e.g. typing, mouse around, click etc.).

I'm using it to send keypresses to the browser, as you rightly observe.

So if I want to just reload a page, the browser gets F5.

To GOTO some page, it gets a stream of keystrokes for the URL characters and then Enter.

It's really that simple-minded, and it works!

This case statement covers my usage: https://github.com/adityaathalye/shite/blob/master/bin/hotre...

andrewflnrvor 4 Jahren

Thanks for the explanation!

mawvor 4 Jahren

Serious(-ish) question: does "And last but not least, I hereby decree that all texsht herein be read in Sean Connery voish." mean that this technically isn't FOSS?

m_muellervor 4 Jahren

*FOSSH

dickholesaladvor 4 Jahren

The name is just… thank you.

datpiffvor 4 Jahren

A+ name, no notes

adityaathalyevor 4 Jahren

Heh, thank you. It almost wrote itself.

I was trying to cook up a clever tagline, and fell upon "static sites from shell" and my brain looped about and found the tongue twister, sort of like "She sells sea shells on the sea shore", and well all esh shounds shtarted shounding like thish.

These days I like that name more because it (hopefully) alerts the Dear Reader about the quality of software they are about to visit, should they take it upon themselves to do so.

[deleted]vor 4 Jahren

bashNoIGGERobvor 4 Jahren

yakshaving_jgtvor 4 Jahren

I'm assuming just about everyone is aware of the meaning of the word.

https://www.urbandictionary.com/define.php?term=Shite

Kind of depressing that people find this amusing.

adityaathalyevor 4 Jahren

This remark prompted me to go rewatch (make sure you have headphones on)...

this bit of fry and laurie https://www.youtube.com/watch?v=cVe3fTL1cPM

this bit by atkinson and cleese https://www.youtube.com/watch?v=OGFz9gt0-Fc

and this bit by atkinson alone https://www.youtube.com/watch?v=R7OxTxAvvLw

made my day :)

mrlonglongvor 4 Jahren

Oh do one mate quit griping and get with it!

nmeofthestatevor 4 Jahren

I suggest a SFW rename to "connery"

talideonvor 4 Jahren

You might want to look that up in French, but replace the 'y' with 'ie'...

badcppdevvor 4 Jahren

It's sad because it's not hard to pick a name for a project that isn't a swear word.

082349872349872vor 4 Jahren

I once had a gig someplace where the tradition was to name Hamming Distance 1 away from swear words. Best of both worlds?

pragmatic8vor 4 Jahren

Could you please expound on why you believe that to be depressing?

yakshaving_jgtvor 4 Jahren

Sure. It's not the coarse language that I object to — it's the lazy, juvenile humour. It's similar to penisJS[0] — a software analogue to children defacing the chairs in their school classrooms with drawings of genitalia.

I think it's a pretty bad look for the industry.

Not to say there shouldn't be humour — there should be more humour. Personally, my most popular software projects are purely comedic, and they're also a little more coarse than this.

To put it another way: I'd prefer the software humour to be a little more Sean Lock and a little less Beavis and Butt-Head.

[0]: https://github.com/edankwan/penis.js/

adityaathalyevor 4 Jahren

I hear you. Thank you for sharing your thinking. All I will say is, from where I stand, it is all working as intended; the code and the phraseology :)