I think about LLMs in software today as attempting to provide a similar role as compilers in terms of translating high-level thoughts into low-level details.
The only problem is that I actually trust compilers.
What do you mean, "no mitigation" and "except for kernel updates"?
Somebody made a mistake. Somebody found the mistake, and told the maintainers (and even offered a draft patch). The maintainers fixed the mistake and gave the person who found it a prize.
Granted, I'm grouchy about the fact that software engineering is just about the only engineering field which is neither licensed nor bonded nor insured... which sure, makes it more accessible, but also means that software engineering is "like a box of chocolates: you never know what you're gonna get".
(Meanwhile, all the other engineering fields say things like "those regulations are written in blood").
But of all the subspecialties to complain about, I'm not picking on security research. At least security research has an industry norm of responsible disclosure, plus filing CVEs into a neat database so we can know which bugs impact what, plus an industry norm of bug bounty programs to give awards to security researchers as a thank-you (instead of leaving things open to cybercriminals who will find the same issue and use it for nefarious purposes).
To me, that sounds like a model that everybody else should strive for.
I suppose we could talk about whether there could have been more testing or linting or something to prevent the mistake from getting out to production in the first place. But 1. this mistake is pretty subtle -- of all the mistakes to happen, this one seems relatively understandable 2. spending more on prevention seems like a great way for businesses to spend less on bug bounty payouts ;)
Who knows... someday, we might get to a point where software is actually expected to be defect-free once it reaches production, to the point where patch updates are a rare surprise.
That's obviously not the case today, but here's hoping.
Well... you're not supposed to roll your own cryptography. Like, don't go thinking that you're going to do better than a NIST contest winner if you need a cryptographic primitive.
That goes for protocols as well.
And security-related "build vs. buy" decisions should always include an element of battle-testing. If you have the option to pick an off-the-shelf TLS library rather than implementing your own, both are gonna be full of bugs but at least the former will have already had CVEs filed and fixed. (That's not, btw, an assumption that one likes to write buggy code, but rather, a choice to operate under the assumption that one can always be surprised. Schneier's Law and all).
That said, there comes a point to model your own problem domain, and make sure that your product includes features and components that allow it to meet your information security goals.
I think software engineering has done itself a bit of a disservice by making security seem "scary" (and therefore, something to avoid at all costs) rather than a necessary and boring component of any computer system.
P.S. If anybody wants resources, I'm a big fan of the way that FIPS-199 -> FIPS-200 -> SP 800-53
breaks down security goals, impact levels, and appropriate control measures. You can pick a goal, assess your impact level, and pick something off-the-shelf pretty quickly.
Regarding emacs-style kbds in firefox -- I use vimium to remap most actions, and occasionally resort to an additional global keybinding helper app (e.g. BetterTouchTool on macOS).
I've said it before, I'll say it again, I'm convinced LLMs will be the thing to convince software engineers to get it together and create a licensing board.
Heh, I wasn't suggesting that AI would actually replace decision-making. Rather, I wonder whether attempts to use AI in this way would result in such publicly-embarassing and catastrophic outcomes that software engineers might decide to organize professional guardrails about it.
I fully agree, this seems like a legal liability issue waiting to happen.
This is the part where you'd normally pull the junior engineer aside and politely give them a stern talking to until they understood what they did wrong.
If anybody has suggestions for how to do this with LLMs (short of maintaining CLAUDE_wall_of_shame.md), please share.
Edit: for the record, yes I do run a linter, and generally try not to impose bikeshedding or soapboxes on my peers. It's just that there are certain patterns that I personally am not going to commit under my own username as the engineer of record.
Edit 2: I saw another comment recommending "Always confirm with me before doing $x" (and then always denying). Seems like it might work.
The only problem is that I actually trust compilers.