Intelligence seems to boil down to an approximation of reality. The only scientific output is prediction. If we want to know what happens next just wait. If we want to predict what will happen next we build a model. Models only model a subset of reality and therefore can only predict a subset of what will happen. Llms are useful because they are trained to predict human knowledge, token by token.
Intelligence has to have a fitness function, predicting best action for optimal outcome.
Unless we let AI come up with its own goal and let it bash its head against reality to achieve that goal then I’m not sure we’ll ever get to a place where we have an intelligence explosion. Even then the only goal we could give that’s general enough for it to require increasing amounts of intelligence is survival.
But there is something going on right now and I believe it’s an efficiency explosion. Where everything you want to know if right at hand and if it’s not fuguring out how to make it right at hand is getting easier and easier.
That is a good point. It is impressive. Llms from two years ago were impressive, llms a year ago were impressive, and from a month ago even more impressive.
Still, getting "something" to compile after a week of work is very different from getting the thing you wanted.
What is being sold, and invested in, is the promise that LLMs can accomplish "large things" unaided.
But they can't, as of yet, they cannot, unless something is happening in one of the SOTA labs that we don't know about.
They can however accomplish small things unaided. However there is an upper bound, at least functionally.
I just wish everyone was on the same page about their abilities and their limitations.
To me they understand conext well (e.g. the task, build a browser doesn't need some huge specification because specifications already exist).
They can write code competently (this is my experience anyway)
They can accomplish small tasks (my experience again, "small" is a really loose definition I know)
They cannot understand context that doesn't exist (they can't magically know what you mean, but they can bring to bear considerable knowledge of pre-existing work and conventions that helps them make good assumptions and the agentic loop prompts them to ask for clarification when needed)
They cannot accomplish large tasks (again my experience)
It seems to me there is something akin to the context window into which a task can fit. They have this compact feature which I suspect is where this limitation lies. Ie a person can't hold an entire browser codebase in their head, but they can create a general top level mapping of the whole thing so they can know where to reach, where areas of improvement are necessary, how things fit together and what has been and what hasn't been implemented. I suspect this compaction doesn't work super well for agents because it is a best effort tacked on feature.
I say all this speculatively, and I am genuinely interested in whether this next level of capability is possible. To me it could go either way.
This is entirely too charitable. Basically all this proves is that the agent could run in a loop for a week or so, did anyone doubt that?
They marketed as if we were really close to having agents that could build a browser on their own. They rightly deserve the blowback.
This is an issue that is very important because of how much money is being thrown at it, and that effects everyone, not just the "stakeholders". At some point if it does become true that you can ask an agent to build a browser and it actually does, that is very significant.
At this point in time I personally can't predict whether that will happen or not, but the consequences of it happening seem pretty drastic.
> I can definitely believe that in 2026 someone at their computer with access to money can send the right emails and make the right bank transfers to get real people to grow corn for you.
I think this is the new turing test. Once it's been passed we will have AGI and all the Sam Altmans of the world will be proven correct. (This isn't a perfect test obviously, but neither was the turing test)
If it fails to pass we will still have what jdthedisciple pointed out
> a non-farmer, is doing professional farmer's work all on his own without prior experience
I am actually curious how many people really believe AGI will happen. Theres alot of talk about it, but when can I ask claude code to build me a browser from scratch and I get a browser from scratch. Or when can I ask claude code to grow corn and claude code grows corn. Never? In 2027? In 2035? In the year 3000?
HN seems rife with strong opinions on this, but does anybody really know?
One definition of analysis is: The process of separating something into its constituent elements.
I think when someone designs a software system, this is the root process, to break a problem into parts that can be manipulated. Humans do this well, and some humans do this surprisingly well. I suspect there is some sort of neurotransmitter reward when parsimony meets function.
Once we can manipulate those parts we tend to reframe the problem as the definition of those parts, the problem ceases to exist and what is left is only the solution.
With coding agents we end up in weird place, one, we have to just give them the problem, or we have to give them the solution. Giving them the solution means that we have to give them more and more details until they arrive at what we want. Giving an agent the problem we never really get the satisfaction of the problem dissolving into the solution.
At some level we have to understand what we want. If we don't we are completely lost.
When the problem changes we need to understand it, orient ourselves to it, find which parts still apply and which need to change and what needs to be added, if we had no part in the solution we are that much further behind in understanding it.
I think this, at an emotional level is what developers are responding to.
Assumptions baked into the article are:
You can keep adding features and Claude will just figure it out, sure, but for whom, and will they understand it.
Performance won't demand you prioritize feature A over feature B.
Security (that you don't understand) will be implemented over feature C, because Claude knows better.
Claude will keep getting more intelligent.
The only assumption I think is right, is that Claude will keep getting better. All the other assumptions require you know WTF you are doing (which we do, but for how long will we know what we are doing).
Maybe one day our knee jerk reactionary outrage will be quelled not by any enlightenment but because we are forced to grow weary of falling prey to phishing attacks.
I'd feel pretty stupid getting worked up about something only to realize that getting worked up about it was used against me.
I'm writing this because for a moment I did get worked up and then had the slow realization it was a phishing attack, slightly before the article got to the point.
Anyways, I think the clickbait is kindof appropriate here because it rather poignantly captures what is going on.
"The energy consumed per text prompt for Gemini Apps has been reduced by 33x over the past 12 months."
My thinking is that if Google can give away LLM usage (which is obviously subsidized) it can't be astronomically expensive, in the realm of what we are paying for ChatGPT. Google has their own TPUs and company culture oriented towards optimizing the energy usage/hardware costs.
I tend to agree with the grandparent on this, LLMs will get cheaper for what we have now level intelligence, and will get more expensive for SOTA models.
I get this take, but given the state of the world (the US anyways), I find it hard to trust anyone with any kind of profit motive. I feel like any information can’t be taken as fact, it can just be rolled into your world view and discarded if useful or not. If you need to make a decision that can’t be backed out of that has real world consequences I think/hope most people are learning to do as much due diligence as reasonable. Llms seem at this moment to be trying to give reliable information. When they’ve been fine tuned to avoid certain topics it’s obvious. This could change but I suspect it will be hard to find tune them too far in a direction without losing capability.
That said, it definitely feels as though keeping a coherent picture of what is actually happening is getting harder, which is scary.
When I tried the Gemini/AI formula it didn’t work very well, gpt-5 mini or nano are cheap and generally do what you want if you are asking something straightforward about a piece of content you give them. You can also give a json schema to make the results more deterministic.
Don't know about excel, but for Google Sheets. You can ask chatgpt to write you a appsscript custom function e.g CALL_OPENAI. Then you can pass in variables into. =CALL_OPEN("Classify this survey response as positive, negative, or off-topic: "&A1)
That makes sense for why they are so much better at writing code than actually following the steps the same code specifies.
Curious, is anyone training in adversarial simulations? In open world simulations?
I think what humans do is align their own survival instinct with a surrogate activities and then rewrite their internal schema to be successful in said activities.
It seems like there is a bunch of research/working implementations that allow efficient fine tuning of models. Additionally there are ways to tune the model to outcomes vs training examples.
Right now the state of the world with LLMs is that they try to predict a script in which they are a happy assistant as guided by their alignment phase.
I'm not sure what happens when they start getting trained in simulations to be goal oriented, ie their token generation is based off not what they think should come next but what should come next in order to accomplish a goal. Not sure how far away that is but it is worrying.
I think this is the wrong take. I don’t agree that people are good or bad, I think actions are, and there are lots of reasons and motivations a person can end up enabling a bad situation, some of those motivations can even at the time be justified.
I do believe Meta is very bad for the world and has way too much power. Anything that can get people to open their eyes to this is important. Dividing those that are trying isn’t helping.
