I've been building and running my software factory for 8 months now. Granted, there's no automated pulling down tasks and pushing PR right now (soon!). But after specifying what I want, it mostly goes to shipping on its own. On occasion, it does raise issues that I have to make a decision on. After doing systems evals on review, I've stopped looking at code during review for 4 months now.
I do spend a lot of time up front specifying what I want. My prompts aren't one-liners, but rather an interview process where we work through all the open questions and ambiguity.
I haven't hit the wall that the OP talked about (when agents just can't seem to make the right changes, and it's impossible for me to go in and change things manually). I used a lot of guardrails such as plan reviews, browser-based QA, adversarial reviews, unit tests, linters, typecheckers, post-commit hooks, and formal method traces. I also specified engineering principles that steers the code base to minimize state and side-effects: functional core; imperative shell, make impossible states impossible, use pure functional style, etc.
There are times, when I can feel a part of the code base is messy without looking at it, because the agent will make recurring mistakes in the same part of the code base over time. What I found the agent was doing over time is that it's been layering state variables as requirements were discovered. So what helps is to ask it to refactor all these state variables into a single sum type. And if the state machine for it is complicated, I'll ask it to write a formal model of the state in Quint. Then I'll generate traces that get run as unit tests, and ask it to write the code against that.
So while the code base isn't exactly Brownfield, it's over a year old now. As for the code base, there's a backend and a frontend. I think it helps that I established a clear pattern I wanted. You code are like memes: agents will just copy patterns they see in the code base. When it does have to create a new part of the system, I found Sonnet-level models tend to draw system boundaries in all the wrong places. Opus is better. I don't yet know about Fable.
I think Evan's pointing at something, and everyone's looking at the finger. I don't think this is really about Elm at all, but the preparation for Acadia.
In his talk, Economics of Programming Languages https://www.youtube.com/watch?v=XZ3w_jec1v8, he does reveal a peek at what he's been working on for the past seven years: a query language with functional ideas applied to them.
He points at the boundaries in web apps that typically destroys type information:
- language <-> storage
- storage <-> wire
What if we could preserve the type information across these boundaries? What would that buy us? One answer he had was easier migrations. They'd just be a type diff, rather than hand-written SQL.
So I think the right question is, can we infer what Acadia is going to be like given the changes he's shipping with Elm? What does incremental compilation give query languages?
I think I've always called this "Ontology is hard". It's genuinely useful when it's used as a tool for clarification. It's constraining when it's used as a tool for modeling.
You didn't need to say anything. Plenty of people would have stepped up to defend you. Now, the sarcasm looks defensive. This could be another blog post in a couple months.
If this works with writing, it should also work with code. `git blame` should be enough training data to de-anonymize open source programmers. Maybe that'd be addition information to point out who Satoshi is.
Blow originally did Order of Sinking Star as a quick side project. He thought that by using these pre-existing games as a starting point, he'd get it done quicker. But then he decided to experiment with the combinatorics of these mechanics that the game blew up so much in scope that the original starting point didn't help at all.
It's a puzzle maker's puzzle game. The reason why it's so lauded is because the design is so tight. Kinda like how there's certain buildings the public thinks is ugly, but architects all like it because it tickles that part of the architect brain. It's a game that gives you that ah-ha moment. Kinda that moment where you walk from the forest into a clearing, but for your brain.
The game is hard. I only kinda got the hang of it, and I didn't quite get to that ah-ha moment. You have to be willing to sit with it and think. I think with sokoban games, you can often just almost random walk your way to a solution, because the state space and its transitions is easy enough to wander into. But I didn't find that to be the case with SSR. You have to be able to reason about the state space changes, I think because the state space isn't exactly euclidean, so it's harder to wander into the solution.
Sokoban is a common word within puzzle game fans and devs. That article wasn't written for people that didn't like those kinds of puzzles in the first place.
The bug is in the software in our heads, if anything. We learned a little too much, that we're thinking further ahead than we would have when we first started out. So you need to purposefully shut off that part of your eval, so that you get started on anything at all.
If you design with the LLM, then it can make this easier by prompting it to help you not talk yourself out of things.
I found that gstack's /office_hours to be good about encouraging, while being firm. I've only done one of the modes, but it didn't dismiss my pushback when it was just based on my intuition. It took it as a baseline, and tried to evaluate it by taking it seriously. If that's any indication, the other modes for side projects should be just as supportive.
I think LLMs can make it easier to be more ambitious. Non-techies are blown away by being able to build web pages? I'm blown away that I was able to root my 1st gen Kindle Fire to repurpose it as a remote terminal to ssh into my laptop to talk to claude code. I've been trying to root the thing for years and could never find the right instructions to make it work.
It's easy to talk yourself out of doing things when you know a little too much. Sometimes, it's good to get back into the mode where you knew nothing and do things for their own sake, just to get the engine started again.
I didn't find llms.txt useless at all. I was able to download all the library docs and check it into my repo and point my coding agent to it all the time.
I kept thinking that he'd eventually compare it to writing software by hand, and how we're at the end of one golden age. But he never did. So I wonder what the impetus for the essay was.
Doesn't make it excusable. I get it's hard to uphold principles when the stomach is empty. But it's clear the person in the piece wasn't thinking about much else, though he was also clearly not in the streets and starving.
Longform: https://interjectedfuture.com
Twitter: http://www.twitter.com/iamwil
Previously:
Podcasting: The Technium https://www.youtube.com/channel/UCl_rEKDGBw4myn0uOnPxYsg
https://pulley.com - Cap table management software
https://dirtprotocol.com - Token Curated Registries in Crypto (defunct)
https://helmspoint.com - Deploy Keras ML models (defunct)
https://www.pebble.com - Smartwatches (defunct)
https://cubehero.com - Github for 3D printed models (only blog is left)
http://noteleaf.com - Rapportive for meetings (YC W11)
https://techcrunch.com/2008/10/23/frogmetrics-handheld-surveys-you-might-actually-want-to-fill-out/ (YC S08)
Made VR demos. https://www.youtube.com/watch?v=GQAFp4_P6bM&list=PLPDqpcNm-N2cXTL31pcnBEISlFcCxDb6W
Is it recession? https://isitrecession.com