I find this paragraph to be odd: "Wavelengths as low as 13.5 nanometers can achieve more precise patterns in a single exposure. In fact, extreme ultraviolet lithography can combine three or four photolithography patterning cycles into a single one on a seven-nanometer node. Without EUV, producing five-nanometer nodes might require as many as one hundred different steps."
But a five-nanometer node has a gate pitch of 45nm and a metal pitch of 20nm! Using different forms of the word "nanometer" in the same paragraph is very confusing...
You're right; I should've been more precise. However, we have tools for dealing with this—that's what quality-adjusted life-years are for! I don't contest that surgeries often significantly increase QALYs, and may do so pretty cost-effectively.
Lol, I just care a lot about saving as many lives as I can; the most effective charities I've been able to find good evidence on save one life for $6–8k. If Watsi had a credible claim at being able to save lives 10x cheaper I would redirect my entire donation budget to them!
That said, once again, Watsi is great. I really appreciate all the hard work they've put into making this happen—this is orders of magnitude more impressive and impactful than most projects I've ever seen!
Watsi seems to be doing great work, but the title—"you helped save 33k lives"—reads as misleading to me. I guess "helped" could be doing a lot of heavy lifting here, but I would be incredibly surprised if the counterfactual number of lives saved was more than 3000. (But don't let this dissuade you from donating; concretely improving someone's life is totally a worthwhile goal, and Watsi seems very good at effecting this)
This is a cool setup, but naively it feels like it would require hundreds of thousands of hours of data to train a decent generalizable model that would be useful for consumers. Are there plans to scale this up, or is there reason to believe that tens of thousands of hours are enough?
A) You could have an additional field in the jsonl file which says which rubric to use; then, your reward function could access this via `kwargs["rubric"]` and return a reward based on that example's preferred rubric;
B) currently, pricing on the deployed API is free, but the startup time is a few minutes and it's run on a small GPU node and is therefore not awfully fast. If you would like more production-level inference, email us at [email protected] and we could set you up with something much faster (where we'd charge per token depending on model size)
Having an RL agent that's really good at search across some space sounds very powerful in general; "proofs-as-search" make this an appealing target. Back in the day, when I did more fundamental RL research, we worked on an extension of SoRB [0] where an additional meta-level target was learning improved heuristics to explore the search space faster; would be exciting to figure out what a good setup for doing things like this in LLM-policy-gradient world is these days!
we should publish some; the high-order effect seems to be that LoRAs significantly hurt small model performance vs FFT, with less of an effect for large models. This is maybe because large models have more built-in skills and thus a LoRA suffices to elicit the existing skill, whereas for small models you need to do more actual learning (holding # parameter updates constant). In general I think it's better to get a performant small model with FFT than a performant large model with a large LoRA, which is why we default to FFT, but I agree that we should publish more details here.
Thanks! Our goal is to make rl "just work" with completely automated GPU provisioning/algorithm selection/SFT-warm up, but giving people the ability to switch away from the defaults if they want to.
The way tools currently work in the beta is you add tools via MCP to the configuration, and they get passed in as additional context for the model; the model might then choose to use a tool during inference; the tool is then automatically called and the output is returned as a tool message. If you really want to you could parse the tool output as part of reward calculation, but I expect you'd usually base the reward just on the model's completion. I could give more details if there's a specific tool setup you're envisioning!
Yeah, for better or worse, the way the median startup interfaces with AI these days is through an LLM API, and that's what all the workflows are built around, so that's what we're targeting. Though, depending on what you're trying to do, I wouldn't discount the use of starting with a pretrained model—there was that famous result from 2022 that showed that pretraining a model on _Wikipedia_ made training on Atari games more than twice as efficient [0]; these days, LLMs have huge amounts of priors about the real world that make them great starting points for a surprisingly diverse set of tasks (e.g. see the chemistry example in our video!)
It's for any task that has an "eval", which is often verifiable tasks or ones that can be judged by LLMs (e.g. see [0]). There's also been recent work such as BRPO [1] and similar approaches to make more and more "non-verifiable" tasks have verifiable rewards!
prompt optimization is very cool, and we use it for certain problems! The main goal with this launch is to democratize access to "the real thing"; in many cases, full RL allows you to get the last few percent in reliability for things like complex agentic workflows where prompt optimization doesn't quite get you far enough.
There's also lots of interesting possibilities such as RLing a model on a bunch of environments and then prompt optimizing it on each specific one, which seems way better than, like, training and hot-swapping many LoRAs. In any case, _someone's_ ought to provide a full RL api, and we're here to do that well!
I started listening to it after this very submission became #1 on HN, so it was very meta to listen to it talk about how it might be making stuff up...
But a five-nanometer node has a gate pitch of 45nm and a metal pitch of 20nm! Using different forms of the word "nanometer" in the same paragraph is very confusing...