This is a well-referenced essay, drawing the on writing of David Parnas [1], Peter Naur [2], and Zach Tellman [3].
As software developers we’re intimately familiar with these ideas. But the industry still treats it as “folk knowledge”, despite decades of academic work and systemization attempts like the original Agile.
We really need more connective work, relating the theoretical ideas to the observed behavior of real-life software projects, and to the subsequent damage and dysfunction. I liked this essay because it scratches that itch for me. But we need this work to go beyond personal blogs/newsletters/dev.to articles. It needs to be recognized & accepted as formal “scientific” knowledge, and to be seen and grokked by industry and corporate leadership.
I’m a violinist (amateur but play regularly). When I have an important note, which is held for a while and needs vibrato, I frequently decide to shift my left hand position so that my middle finger is responsible, rather than the index finger. It feels stronger, easier to nail the intonation (pitch) with precision, and freer to perform the desired type of vibrato. (String players do vibrato by wiggling the left hand finger, which affects the pitch and overtones / oscillation modes of the string.) In fact, I tend to avoid using the index finger on notes that require vibrato.
That preference might be explained here, by the precision/strength combination. I tried holding a hammer as described in the author’s hammer exercise, and there’s similarity, though it requires much more weight-holding. The left hand doesn’t hold the weight of the violin (consider a cello or a guitar with shoulder strap), but a little grip strength is required to securely hold down the string, especially with vibrato.
Overall, fascinating article. I feel quite motivated to read more on hand anatomy and biomechanics.
Agreed. I recall being taught in college physics labs: there is no such thing as “human error”. Instead, think about the causes and mechanisms of each source of error, which helps both quantifying and mitigating them.
Same energy here. “Be more careful” is extraordinarily hand-wavy for a profession that calls itself engineering.
A while back I prototyped (very roughly) an auditory equivalent to “syntax highlighting”, using ambient tones and white noise, rather than discrete beeps/sound effects. [1]
I’m actually revisiting this project right now! I’m reimplementing it in Rust and also exploring different ways to communicate parser state and other contextual information through sound.
I’m glad I saw your comment!! I’ve experienced this exact phenomenon when playing Minecraft while listening to an audiobook or podcast. Returning to that area will immediately remind me of the topic or narrative that I heard. Presumably it’s related to the “memory palace” technique, but otherwise, I can’t make heads or tails of it. It’s immediate, as if the location is a hash key mapping to the information. Or as if they’re stored in literally the same place, and fetching one implies fetching the other.
Similarly to you and the article’s author, this doesn’t happen with whatever thoughts I may think while at a location. But in that situation the brain is engaged in generating those thoughts, and not with the task of learning new information. So I don’t find it surprising that it works differently.
I haven’t thought about it in relation to “consciousness” yet. Will have to chew on this article a bit.
The museum was also a generous community space; I remember attending a Seattle Indies game jam and other events before the pandemic. It was very special to be surrounded by reminders of early-computing exploratory spirit.
*If* such a thing could be done under typical corporate incentives/behavior, then I suspect the “high impact” part would need to be scrapped. Because when something is important to a corporation, it turns its eyes that thing (so to speak), which disrupts the other properties.
Or, “high impact” could be spread over the long term. So, unknown-payoff R&D. It would need to be an “invest and ignore” strategy and require a lot of institutional trust.
Question for ML/AI people: is it still common to use the term “overfitting” for these cases, where a model is overtrained on one thing, to the detriment of another? Or is that term only used for literal curve fitting?
I do think it’s an interesting line of inquiry… but not robust enough.
E.g. this paper would be much more interesting if it measured the threshold at which the LLM starts to become good at X, and linked that threshold to the number and character of training examples of X. Then, maybe, we can begin to think about comparing the LLM to a human.
Alas, it requires access to the training data to do that study, and it requires a vast amount of compute to do it robustly.
Good point - I saw the FLAN anomaly and this didn’t occur to me!
A good follow up question would be: why didn’t the other models do better on the 2nd-order question? Especially BLOOM and davinci-003, which were middling on the 1st-order question.
I agree on your overall criticism of the experimental protocol, though.
Skimming through studies like this, it strikes me that LLM inquiry is in its infancy. I’m not sure that the typical tools & heuristics of quantitative science are powerful enough.
For instance, some questions on this particular study:
- Measurements and other quantities are cited here with anywhere between 2 and 5 significant figures. Is this enough? Can these say anything meaningful about a set of objects which differ by literally billions (if not trillions) of internal parameters?
- One of prompts in second set of experiments replaces the word “person” (from the first experiment) with the word “burglar”. This is a major change, and one that was unnecessary as far as I can tell. I don’t see any discussion of why that change was included. How should experiments control for things like this?
- We know that LLMs can generate fiction. How do we detect the “usage” of the capability and control for that in studies of deception?
A lot of my concerns are similar to those I have with studies in the “soft” sciences. (Psychology, sociology, etc.) However, because an LLM is a “thing” - an artifact that can be measured, copied, tweaked, poked and prodded without ethical concern - we could do more with them, scientifically and quantitatively. And because it’s a “thing”, casual readers might implicitly expect a higher level of certainty when they see these paper titles.
(I don’t give this level of attention to all papers I come across, and I don’t follow this area in general, so maybe I’ve missed relevant research that answers some of these questions.)
There are multiple senses of the word “creativity”, and this post focuses on one of them: divergent thinking. The other sense is that of constructive, goal-oriented creation, which ideas alone cannot achieve. It’s too bad we don’t have have more commonly used terms to make this distinction. I see a lot of comments here focusing on that distinction rather than the post’s central thesis.
I do have a comment on the thesis, which is:
> The purpose of this article is to challenge this assumption [that creativity is binary] and discuss aspects of ideation, i.e. the process of coming up with ideas.
I support/agree with this challenge and all of the article’s ideas. “And yet”, right?? “And yet” some people are perceived to “have something” which others do not.
Honestly, the explanation is rather simple, or at least, simply stated. It’s neurodivergence. I’d further claim that cognitive styles gravitate to certain “attractor points”. (That’s scientific lingo for: certain patterns which fit well within the environment and which reinforce themselves. Like the pattern of wheel-ruts which attract wheels, which makes them stronger. The “environment” in this case is all sorts of things, including both the brain’s biological details, and the body’s physical+social environment.)
The strongest of these attractor points, we give labels: ADHD, various species of autism, etc. And of course the “normal person” attractor - not a point, but a broad area with its little micro-attractors and, sometimes, niche wormholes leading to more divergent areas.
People tend to clump around the strongest attractor points, and sometimes get pulled into other more smaller ones. This easily explains the perception of binary other-ness, especially when you consider that deviation from the norm - in any of the many directions - is, itself, a strong, influential force in this dynamic. To the extent that we try to build society to work well enough for the majority, anyone who deviates will have different and novel experiences of those systems.
But look, people are complicated and dynamic. We sometimes work to push away from these pattern-ruts, and other times we let ourselves be pulled into them.
This article is saying: YES. You can do things that make you ideate more divergently. You can also do the work to explore your own cognitive-behavioral niche, and which pushes your idea output into more novel, “creative” realms. Play is a certain type of work, when you need to push yourself to do it.
The article also addresses this:
> Good ideas do not have to be completely novel
> A hallmark of creativity is the knowledge or intuition of picking ideas that make suitable combinations. [more worthwhile to pursue]
…which brings us back to the other sense of creativity: not just divergence, but convergence; pursuit of a vision or goal or “gut feeling” intuition. I think this is the better, fuller meaning of the word. The author describes interaction between convergence and divergence very well. In the best examples of “creative genius”, both of these forces are at play. (No pun intended but perhaps that’s revealing.) Fluid, progressive creativity is at the edge of these two forces, and a “creative” person steers the ship, aware of both convergent goals and overarching visions that can only be reached by leaving those same goals behind.
The general skill of steering is quite meta-learnable by, probably, nearly everyone with any ounce of cognitive control. It takes time and support. It’s easier in more specific contexts, more well-suited to one’s situation.
For what it’s worth, toddlers absolutely do exhibit this full version of creativity, when you consider that they are pursuing the instinctive, hard-wired goal of learning and adapting to the world.
I’ve had a variety of responses to this list over my programming life (~10 years hobby, ~10 years professional).
When I first encountered them as a hobbyist, they were surprising, as perhaps intended, due to the framing of classic vices as virtues. On some reflection though, it made sense, and shaped my understanding of programming as somehow _inherently different_ than other types of creation.
When I got started with a professional career, they functioned to soften the edge of anxiety. It meant that the community of programmers who came before me - which presumably included Larry Wall - would understand that these patterns in coder behavior were ultimately beneficial and that I would maybe fit in with a corporate programming environment. (Now I know that this isn’t always true; sometimes coworkers, both programmers and non-programmers, don’t always realize these unintuitive points, and in some special cases, those programmer instincts aren’t actually valuable.)
At some point, I disagreed with the framing. As others have pointed out, the patterns can be reframed as classic or functional virtues such as curiosity. Then I backpedaled and realized that the framing is important because these are unintuitive patterns and it makes us re-think habitual incentives that reward, e.g., work that is more productive but not more effective.
How did it hit me now? I realize that it’s also related to power dynamics. Programmers are assets to their employers but they’re also potential disrupters. The traditional “virtuous” framing of potentially-less-effective behavior, like patience, is related to the organizing and taming of a workforce.
It’s also related to the types of problems we encounter. When we work with computers, Larry’s list does usually lead to more effective outcomes. But when working with other humans, who have their own agency and idiosyncrasies, the traditional virtues are better-adapted behavior. This is also more true for the complex technical systems we deal with nowadays. So, as our careers transition from programming to system engineering and/or management, the traditional virtues become more relevant.
Anyhow, this is an evergreen and thought-provoking nugget of wisdom. Thanks to Larry Wall and those who have preserved it.
Wow - can you elaborate on your process to “debug” these different layers? What comprises your personal or team’s feedback loop, given that 6 months is short? (I suspect it’s a lot of sequencing?) Are there good comparisons to be made with getting acquainted with a new codebase/technical system? Are there any particular computational tools involved? Pardon all the questions, this stuff is fascinating!
Like genes, the following collections of information are plans, as other entities (workers, compilers, cells, etc) can reliably use them to produce larger, more complex objects:
- Blueprint
- Instruction manual
- Recipe
- Code
Unlike genes, they’re all human-designed. The top-down forcing function - the “back” in the feedback loop which shapes them - is human artifice.
I’ll leave it as an “exercise to the reader” to consider the differences in how their environment and execution apparatuses affect the resulting objects.
Genes are like blueprints, but obviously not the same. For one, they haven’t passed the county permitting process! And living organisms are like buildings because you can point to the plan behind them. But I’ll be darned if a house has ever had to struggle for survival.