Awesome. For others, I see that Paras is now running https://turingsdream.co/ "an AI hackhouse running a six-week residency for coders and researchers who're interested in AI. The hackhouse is in Bangalore, India and you can join it either in-person, or in a hybrid fashion."
Right, they seem highly volatile/variable. The thinking is that showing the 99th percentile / range of possibilities would cover that, if it's based on historical data - does that seem right to you or no, and if not why not?
Another fire project idea: show, based on some kind of prediction model that gives 50th, 90th, 99th percentiles (from historical data, perhaps, or perhaps just from wind/fire speeds), how fast a given fire could reach a specific location.
Whenever I've opened watch duty, that's always the question I'm asking. How long might it take to reach [here/there]?
If you dislike a situation you're in and you try and fix it by switching to a new situation, you'll generally bring with you some of the problems that created that prior situation.
If instead, you bit by bit improve the situation until you feel at peace with it, you'll then either no longer want to move to a new situation, or if you do want to move, you'll no longer bring with you the problems of the prior situation.
Applies to job changes, relationships, projects, goals. And, from OP, applies to architecting software projects.
They're only allowed 2-3 guesses per problem. So even though yes it generates many candidates, it can't validate them - it doesn't have tool use or a verifier, it submits the best 2-3 guesses. https://www.lesswrong.com/posts/Rdwui3wHxCeKb7feK/getting-50...
I was confused about this, here's how I understand it now:
Previously when Amazon lost or damaged items in their warehouses, they would reimburse sellers the full sales price. Starting March 2025, Amazon will only reimburse the manufacturing cost of lost or damaged items. Sellers have to either accept Amazon's estimated manufacturing cost or provide documentation of their actual manufacturing costs.
> Oh man :( We tried really hard to neither over- nor underclaim the results in our communication, but, predictably, some people drastically overclaimed them, and then based on that, others concluded that there was nothing to be seen here (see examples in thread). So, let me try again.
> Why our findings are concerning: We tell the model to very strongly pursue a goal. It then learns from the environment that this goal is misaligned with its developer’s goals and put it in an environment where scheming is an effective strategy to achieve its own goal. Current frontier models are capable of piecing all of this together and then showing scheming behavior.
> Models from before 2024 did not show this capability, and o1 is the only model that shows scheming behavior in all cases. Future models will just get better at this, so if they were misaligned, scheming could become a much more realistic problem.
> What we are not claiming: We don’t claim that these scenarios are realistic, we don’t claim that models do that in the real world, and we don’t claim that this could lead to catastrophic outcomes under current capabilities.
> I think the adequate response to these findings is “We should be slightly more concerned.”
> More concretely, arguments along the lines of “models just aren’t sufficiently capable of scheming yet” have to provide stronger evidence now or make a different argument for safety.
As context on Ilya's predictions given in this talk, he predicted these in July 2017:
> Within the next three years, robotics should be completely solved [wrong, unsolved 7 years later], AI should solve a long-standing unproven theorem [wrong, unsolved 7 years later], programming competitions should be won consistently by AIs [wrong, not true 7 years later, seems close though], and there should be convincing chatbots (though no one should pass the Turing test) [correct, GPT-3 was released by then, and I think with a good prompt it was a convincing chatbot]. In as little as four years, each overnight experiment will feasibly use so much compute capacity that there’s an actual chance of waking up to AGI [didn't happen], given the right algorithm — and figuring out the algorithm will actually happen within 2–4 further years of experimenting with this compute in a competitive multiagent simulation [didn't happen].
Being exceptionally smart in one field doesn't make you exceptionally smart at making predictions about that field. Like AI models, human intelligence often doesn't generalize very well.
Opus 3.5 training succeeded. But it's a more profitable decision to use it to train Sonnet 3.5 and serve that instead. Large models are now teachers, not necessarily end products. Too expensive to serve to end users vs what they'll pay, but great for improving smaller models that are cheaper and faster to serve.
Orion (GPT-5) is being used for training data generation and in verifier/reward models. They say it's not economical to serve to end users until Blackwell chips (B200).
Models that can explore reasoning chains get smarter on certain kinds of problems. [My note, not from article: Math, science, law, programming. R&D, law and programming are perhaps the industries that are willing to pay more for higher reliability.]
Scaling with "berry training" - monte carlo tree search generating thousands of different answer trajectories, then uses functional verifiers to get rid of the ones that didn't arrive at the correct answer.
Big focus is on making inference cheaper and faster. [My note: If you want to work in AI, I imagine any research on LLM inference cost and speed will be highly valuable.]
> What resources would you point your friends to who both want to learn about generative AI and assuage their fears that AI will make artists obsolete?
Step 1 - get them to sign up for AI image tools.
* Midjourney is best for quick images
* Playground AI is good if they need to modify images but the quality doesn't need to be perfect
* Leonardo AI (now owned by Canva) is a good full suite
* Photoshop AI feature is best if they already work in photoshop
Then show them how to use these tools! That might require you signing up for these tools first and learning yourself.
Step 2 - For learning about how AI image generators work here's my video list.
But their fears might be valid. A test is perhaps: if their boss spent a few days learning to use AI image generators, would they still need them? For some artists the answer would be no, for many the answer would be yes. It'll change over time as the tools get better, but that's a pretty good proxy. If they're doing things that require more iteration, interacting with users and humans and the physical world, nuanced judgement, in-person work, safer. If they're doing things that are contract based, no iteration, get a request and deliver a result, much less safe.