i mean, to be fair, these are professional researchers.
i'm very inclined to trust them on the various ways that models can subtly go wrong, in long-term scenarios
for example, consider using models to write email -- is it a misalignment problem if the model is just too good at writing marketing emails?? or too good at getting people to pay a spammy company?
another hot use case: biohacking. if a model is used to do really hardcore synthetic chemistry, one might not realize that it's potentially harmful until too late (ie, the human is splitting up a problem so that no guardrails are triggered)
By publishing the poison fountain, you are making it so that researchers will have to invent techniques to "de-poison" data, perhaps contributing to long-term AI advances in intelligent data filtering while training
And secondly, why would you want worse LLMs? Seems less useful that way
I don't know if you're joking, but here are some answers:
"The mind is not a vessel to be filled, but a fire to be kindled." — Plutarch
"Education is not preparation for life; education is life itself." — John Dewey
"The important thing is not to stop questioning. Curiosity has its own reason for existing." — Albert Einstein
In order to think complex thoughts, you need to have building blocks. That's why we can think of relativity today, while nobody on Earth was able to in 1850.
Yeah, one way to think about it is like... protocols restrict things, so that people can expect the same stuff.
With a traditional API, people can build it any way they want, which means you (the client) need API docs.
With MCP, you literally restrict it to 2 things: get the list of tools, and call the tool (using the schema you got above). Thus the key insight is just about: let's add 1 more endpoint that lists the APIs you have, so that robots can find it.
this is the most brilliant piece of writing i've ever seen -- reminds me of Godel, Escher, Bach.
Would love recommendations for other things like this where the style is one of discovery!! (like math textbooks that make you feel like you invented something?)
So if the model has 16 transformer layers to go through on a forward pass, and each layer, it gets to pick between 16 different choices, that's like 16^16 possible expert combinations!
For someone (me) who is new to distributed systems, and what durable execution even is, I learned a lot (both from the blog post & the examples!). thanks!
i'm very inclined to trust them on the various ways that models can subtly go wrong, in long-term scenarios
for example, consider using models to write email -- is it a misalignment problem if the model is just too good at writing marketing emails?? or too good at getting people to pay a spammy company?
another hot use case: biohacking. if a model is used to do really hardcore synthetic chemistry, one might not realize that it's potentially harmful until too late (ie, the human is splitting up a problem so that no guardrails are triggered)