The analogy stems from the notion that neural nets are "grown" rather than "engineered". Chris Olah has an old, but good post with some specific examples: https://colah.github.io/notes/bio-analogies/
Thanks! I expect the book will remain relevant as long as the Transformers architecture does. That’s why we mostly focus on topics we think will stand the test of time, but let’s see how that plays out :)
“QED and the Men Who Made It” [1] might be close to what you’re after for quantum theory at least. Unlike other popular accounts, it gets quite technical and covers a lot of the historical dead ends that people had during the development of quantum field theory.
Indeed we opted for offline methods like Anchored Preference Optimization as we found in the Open R1 project that doing multi-task RL on small models is quite a hassle to get right. With offline methods, you focus much more on dataset curation / generation, but that still provides faster iteration cycles for the model scale we’re dealing with!
> The absolute best way of doing this is these days is likely through a vision based machine learning model, but that is an approach that is very far away from scaling to processing hundreds of gigabytes of PDF files off a single server with no GPU.
I gave the demo a spin and it’s pretty nice! One thing I noticed is that the avatar doesn’t seem to be aware of it’s surroundings- for example, I asked it why it was wearing a cowboy hat and it was adamant that it wasn’t wearing a hat at all :)
> I expect language models to also get crazy good at mathematical theorem proving
Indeed, systems like AlphaProof / AlphaGeometry are already able to win a silver medal at the IMO, and the former relies on Lean for theorem verification [1]. On the open source side, I really like the ideas in LeanDojo [2], which use a form of RAG to assist the LLM with premise selection.
Hello everyone, we just did a speed run with Argilla and KAIST AI to fine-tune the beefy new Mixtral model with some new techniques that came out recently. More details in the model card - enjoy!
The myth is also promoted in Chapter 3 of The Making of the Atomic Bomb by Richard Rhodes:
> Plank had taught at Berlin since 1889. In 1900 he had proposed a revolutionary idea to explain a persistent problem in mechanical physics, the so-called ultraviolet catastrophe
Spoud is a Series A funded startup that is building a data market platform to enable enterprises to easily find, understand and access their data sources. We are looking for engineers and data scientists to help us develop the product and work on a wide range of projects with our customers.
If you like dynamic environments, solving challenging problems and are passionate about event-driven architecture, then you'll love working at Spoud.
We're more interested in a willingness and ability to learn than what you currently know. But in case you're interested, our tech stack is mainly Java, Kafka, Flink, NodeJS, GRPC, and Docker.
It’s a simple harness around Opus, but with tight integration to Hugging Face infra, so the agent can read papers, test code and launch experiments