Thanks. After I wrote it a friend said "I think you just gave people permission to do things that they would've felt bad about otherwise." I think he was right, in a way. On the other hand, not everything is obvious to everyone, and it's been 20 years. Regardless of whether people have read the book, the knowledge of these things as grown since then.
That doesn't sound like the sort of problem you'd use it for. I think it would be used for the ~10% of code you have in some applications that are part of the critical core. UI, not so much.
I think we are going to end up with common design/code specification language that we use for prompting and testing. There's always going to be a need to convey the exact semantics of what we want. If not, for AI then for the humans who have to grapple with what is made.
Chat in English? Sure. But there is a better way. Make it a game to see how little you can specify to get what you want.
I used this single line to generate a 5 line Java unit test a while back.
test: grip o -> assert state.grip o
LLMs have wide "understanding" of various syntaxes and associated semantics. Most LLMs have instruct tuning that helps. Simplifications that are close to code work.
Re precision, yes, we need precision but if you work in small steps, the precision comes in the review.
Make your own private pidgin language in conversation.
Key takeaway, LLMs are abysmal at planning and reasoning. You can give them the rules of planning task and ask them for a result but, in large part, the correctness of their logic (when it occurs) depends upon additional semantic information rather then just the abstract rules. They showed this by mapping nouns to a completely different domain in rule and input description for a task. After those simple substitutions, performance fell apart. Current LLMs are mostly pattern matchers with bounded generalization ability.
I always have trouble with takes like this because they are context-free. There are a wide variety of project types and development scenarios.
My nuanced take is that typing is an economic choice. If the cost of failure (MTTR and criticality) are low enough it is fine to use dynamic typing. In fact, keeping the cost of failure low (if you can) gives you much more benefit than typing provides.
Erlang, a dynamic language used to create outrageously resilient systems, is a great example of that for the domains where it can be used.
I'm not a dynamic typing zealot (I like static typing a lot) but I do think that dynamic typing is unfairly maligned.
The cost argument brings the decision down to earth.
When people become very skilled at programming they have the urge to scratch their own itch, either writing tools to solve software development problems or creating something with a technology that they want to use. They are uninterested in mundane, boring, vertical applications, but that's often where the money is.
The guy in the article did some development tools but some other things too. At the end of the day, imagining a market is no substitute for finding one.
To me, the most amazing thing is that he says that it is a problem that would take the average programmer a week back then, but today it could probably be coded in 5 minutes in AWK or Perl.
I'm wondering whether there is any reason why condensers can't be used for all program optimization. It seems like the definition would allow that:
> The primary means for shifting computation is the condenser. A condenser is a component that transforms a program, yielding a program that is semantically equivalent under a stated set of constraints (e.g., “class X will not be redefined”), but may be smaller, faster, or better suited to a particular
execution environment.
If they can be generated from the source they probably shouldn't be in the source. Maybe it should be an IDE plugin that displays comments for code as you hover over it.
Applied to AI I think it would be something like - ease of development increases the complexity attempted.