On a related note - would it be easier, instead of doing a benchmark sweep across the whole NxN set of start-end pairs for which layers to modify, to instead measure cross-correlation between outputs of all layers? Shouldn't that produce similar results?
I think that "hovering the code" and "broken rhythm" are correlated, but still separate issues. There are parts of my code (e.g. vibe-coded mockups) that I don't look at the code of at all, and I don't care about it. There are others where I check everything in detail (same as if I were to review someones PR), and I think I have a very good grasp of that code.
But the broken rhythm problem persists regardless, and I find that issue to become more and more serious as LLMs are able to work for longer and longer on their own.
It might be that what we're experiencing now is just an uncanny valley, where they're not yet good enough for us managing them to work in similar ways as with other developers, but are good enough to allow us to switch our attention away from them while they work. But that attention span is mostly wasted, as the time between interactions isn't enough to e.g. work on something else, or read a book.
It's a stupid analogy, but currently it's similar to having a bathroom break every couple of minutes, and if this continues, most developers will probably start doomscrolling more and more.
I was wondering recently if there are some productive activities that might fit well into this rhythm, but I haven't found any yet. I guess sourdough baking is one such example, but there's only so much bread you can eat...
There's a whole industry of "illusions" humans fail for: optical, word plays (including large parts of comedy), the Penn & Teller type, etc. Yet no one claims these are indicators that humans lack some critical capability.
Surface of "illusions" for LLMs is very different from our own, and it's very jagged: change a few words in the above prompt and you get very different results. Note that human illusions are very jagged too, especially in the optical and auditory domains.
No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless. It's all about how we organize the workflows around these limitations.
This is very "distant" suggestion if you enjoyed Antimemetics, but The Unconsoled by Kazuo Ishiguro is another one of my favourites, and it too explores this idea of unreliable and inconsistent memories, although from a completely different angle.
First, that's quite a sad view of incentives structures. Second, you can't be serious in thinking that "worker worried they might be fired" puts the person in charge closer to the "materials" and more "hands on" with the project.
I don't see why his involvement, explaining to his team how exactly to build a piece, is any different from a developer explaining to an LLM how to build a certain feature, when it comes to the level of "being hands on".
Obviously I am not comparing his final product with my code, I am simply pointing out how this metaphor is flawed. Having "workers" shape the material according to your plans does not reduce your agency.
This makes no sense to me. There are plenty of artists out there (e.g. El Anatsui), not to mention whole professions such as architects, who do not interact directly with what they are building, and yet can have profound relationship with the final product.
Discovering the right problem to solve is not necessarily coupled to being "hands on" with the "materials you're shaping".
I always send myself messages as bookmarks, notrs, and general "here's an idea I might want to come back to later". It's an awful system when it comes to discoverability (I do it in Messenger so search is... bad), but I still do it as it's the most convinient option at the moment when I need to store that something somewhere.
So I built a simple Telegram bot which automatically stores anything I send as a text embedding into a vector database, and allows me to search over it in that same chat (same process that powers the AI Q&A assistants these days).
If I post a link, it automatically scrapes it and stores text as chunks for better search, extracts text from youtube videos (still wip), turns images into text with the visual models, etc.
One thing I'm unhappy about is not being able to easily edit any notes I search for later, but it's miles ahead of my previous "system". Hopefully I can open source this when I clean it up - if anyone is interested, let me know.
How do you look at hiring "experienced people" vs. "enthusiastic interns" on something like this? More generally, how quickly do you think the team will grow, and what the ratio should be between the "old" and the "young"?
As someone who daily sends messages to myself as a form of notetaking, for everything from todos and shoplists to ideas and bookmarks, I've thought about this a lot. The only conclusions I came to are that I keep using messages because (1) they are convinient and synced across devices, (2) app is already installed and frequently opened so it rarely needs to load up making notetaking faster, and (3) I do not have to think about organization.
I believe the last point is most important, and is where your app is misaligned with my reasons for using messages - you seem to have spearate folders and notes, prompting me to think "where does this go?" before writing, which is something I really want to avoid.
I'm always drawn to these types of initiatives, and your whitepaper looks (at least with my very limited knowledge of the domain) interesting.
What I am always wondering, and maybe you can give some details here, is the following: isn't the fact that regulations are in natural language, with all its ambiguity, a necessary requirement to have the system operate without being fully specified?
In other words, wouldn't any kind of strict DSL force us to think through all the edge cases that might possibly arise, instead of dealing with them when they do arise, which is basically what the judiciary is for? And isn't that a price too high to implement these kinds of systems?
So if the product from OP used Azure OpenAI, it would be okay? You say "companies happily pay up", but the pricing is exactly the same (source: my company is paying for both APIs).
It's been quite clear for some time that, between OAI and MS, they very neatly split their market: OAI handles the early development and direct customers, and MS handles enterprises. It would require OAI to be a much bigger company than it is right now to properly handle enterprises, and MS already has all that infrastructure (legal, support, etc.). Seems like a sensible setup to me, and I don't see the need for enterprises to run open source models themselves (in this context - of course I see the value in all the other respects about lock-in and specialization), especially if they are already on Azure.
That (LLM -> SQL -> Pandas using PandaSQL) was my approach with Datasloth [1] some time ago. It then grew into Qluent [2] which I now work on full time, along the lines of what the sibling comment wrote about managers being able to analyze their data without coding.
What an amazing person. I first learned about Clifford when he did his TED talk, and I was immediately a fan. The quote from the Hayes Hall Bell Tower resonated, and stayed with me, for all the years since. I'm deeply grateful that people like him exist, sharing their experiences and knowledge while still being simply kind and approachable. If you're reading this, thank you!