I would estimate that out of every 200 lines of code that Claude Code produces, I notice at least 1 issue that would cause severe problems in production.
In my opinion these discussions should include MREs (minimal reproducible examples) in the form of prompts to ground the discussion.
For example, take this prompt and put it into Claude Code, can you see the problematic ways it is handling transactions?
---
The invoicing system is being merged into the core system that uses Postgres as its database. The core system has a table for users with columns user_id, username, creation_date . The invoicing data is available in a json file with columns user_id, invoice_id, amount, description.
The data is too big to fit in memory.
Your role is to create a Python program that creates a table for the invoices in Postgres and then inserts the data from the json file. Users will be accessing the system while the invoices are being inserted.
The closer you get to releasing software, the less useful LLMs become. They tend to go into loops of 'Fixed it!' without having fixed anything.
In my opinion, attempting to hold the hand of the LLM via prompts in English for the 'last mile' to production ready code runs into the fundamental problem of ambiguity of natural languages.
From my experience, those developers that believe LLMs are good enough for production are either building systems that are not critical (e.g. 80% is correct enough), or they do not have the experience to be able to detect how LLM generated code would fail in production beyond the 'happy path'.
If you are trying to build something well represented in the training data, you could get a usable prototype.
If you are unfamiliar with the various ways that naive code would fail in production, you could be fooled into thinking generated code is all you need.
If you try to hold the hand of the coding agents to bring code to a point where it is production ready, be prepared for a frustrating cycle of models responding with ‘Fixed it!’ while only having introduced further issues.
Any sufficiently complicated LLM generated program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of an open source project.
Windows' moat was not the operating system code, but that they were able to get distribution via IBM, and then grow an ecosystem of applications that were targeted at Windows, which created a snowball effect for further applications.
Was software ever a moat? Software typically only gave companies a small window of opportunity to turn a fleeting software advantage into a more resilient moat (network effects, switching costs etc.)
In what way is the long term impact of LLMs being underestimated? If anything, it seems that it has been overestimated in the past years and that something other than LLMs will be needed to reach the original scaled LLM hope of AGI.
Unfortunately, it seems investors now think that all paid software will be replaced by AI generated software, somehow open source projects laundered through generative AI models should finally convince enterprise customers to go with free.
EXWM is great, having the same flow to manage X applications as for emacs buffers is a huge benefit. My only concern is if X11 will be maintained sufficiently into the future to keep using it, currently there is no Wayland support in EXWM.
I am talking about correctness, not style, coding isn't just about being able to show activity (code produced), but rather producing a system that is correctly performing the intended task
Do you have a link to some of the code that you have produced using this approach? I am yet to see a public or private repo with non-trivial generated code that is not fundamentally flawed.
We work with LLMs on a daily basis to solve business use cases. From our work, LLMs seem to be nowhere close to being able to independently solve end-to-end business processes, in every use case they need excessive hand holding (output validation, manual review etc.). I often find myself thinking that a use case would be solved faster and cheaper using other ML approaches.
LLMs for replacing work in its entirety seems to be a stretch of the imagination at this point, unless an academic breakthrough that goes beyond the current approach is discovered, which typically has an unknown timeline.
I just don't see how companies like Anthropic/OpenAI are drawing these conclusions given the current state.
Does Sam Altman have any relevant technical experience to make that assessment? Sounds like something someone would say that just lost their key technical team members.
> Now that they are open sourcing even Delta table optimizations
Has Databricks recently open sourced additional Delta table features that were previously only available with a paid license? I can't seem to find a relevant announcement.