By all means use whatever works for you, I’m not even going to try to make an argument on ethics (and honestly I’m not even sure where I stand, given the behavior of American AI companies).
But I just cringe every time I see people acting like any of this is done in good faith.
Open source coming out of China is a state-sponsored criminal enterprise, built only for the benefit of the Chinese regime, one of the worst to exist in human history.
I think if you showed these before/after menus to people from 5 years ago, most would tell you it’s a massive improvement.
But now many of us have been trained to detect slop from miles away so it just comes across as cheap.
I often feel the same way when I glance at a project’s documentation and it’s long and full of markdown styles and emojis: “the author didn’t think about the users at all, they just wanted to get this over with quickly, so now I have to be super suspicious of everything I read here and it might end up being a huge waste of my time”. A few years ago I’d probably think “wow, emojis!”
I think many of us engineers (including myself) are suffering from AI-induced procrastination. Fixing that pesky bug in production isn't fun. Building a harness/workflow that detects and fixes pesky bugs in production on its own is way more fun (especially when your employer pays for tokens), and it feels just as important, if not more! So what if it didn't actually fix that pesky bug in production on its own...
The Chinese ecosystem is largely built on stealing intellectual property from and/or distilling of US closed source models.
I’m not getting into the ethics/comedy aspect of it, but let’s not pretend this isn’t the case. Plenty of evidence, the motive is obvious, and the numbers are in plain sight (valuations, salaries, etc).
As long as this continues I think closed source will continue being a few steps ahead, but the steps will probably get smaller over time.
Having said that, this is already priced in, the market predicts this gap will be large enough for the US companies to profit from (astronomically).
Most of these reusable, automated workflows sound really cool in theory, but in practice either they require a lot of customization (not really reusable) or a lot of babysitting (not really automated).
AI has improved my productivity significantly, I'm not a hater, but we also have to be honest about its (current) limitations. The whole "software factory" thing is just a pipe dream people sell to execs / investors / online courses.
This is very interesting. I never thought of myself as a “consumer” in that sense, but I guess it is 100% correct, I’ve always been a consumer and I have in fact taken things for granted.
I don’t see that changing, to be honest, but it’s interesting that this never even occurred to me. It seems so obvious in hindsight. Quite the blindspot.
> won't stop until it achieves or exhausts your prompt
This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”.
It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons.
More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse.
Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).
We keep seeing this. If you had to point out the fundamental problem, what would it be?
I think it’s the disconnect. Each persona is an expert in their own field but is completely oblivious to other critical areas.
The founder knows how to raise money but doesn’t really understand the customers. The engineer knows the tech but doesn’t really understand what it takes to keep the business afloat. The salesperson knows what customers want but doesn’t really understand what’s possible to make. The investor knows the numbers but doesn’t really understand how poorly the business is run.
I suspect if you look at successful startups you’ll often see a very small (1-3) group of founders who are very close, each can do more than one thing really well, and their combined expertise means that together they have very few blindspots.
I sometimes wonder if at some point we’ll discover that the fundamental laws of the universe are simply far beyond anything we can even remotely comprehend, like explaining a microprocessor to bacteria.
But then what if we end up with ASI that can comprehend and make it useful for us?
The thing I struggle with wrt to taste is that LLMs just don’t get it.
Even if I write down every single thing it did wrong and how I’d do it, and even if I turn those into rules, it will know how to follow these specific rules, but for some reason it can’t seem to generalize beyond that. And the real list of rules seem truly infinite.