It’s interesting how Egypt’s efforts to monitor and test for malaria contributed to this accomplishment. It underscores how eradicating many infectious diseases will require a deep understanding not only of the disease itself, but also the cycles of transmission and the complex ecology of different hosts.
Malaria’s complex lifecycle [1] seems like it would be easy to “break” with different interventions, but we’ve seen historically malaria has been difficult to eradicate. Why is this?
For those like myself who design proteins for a living, the open secret is that well before AlphaFold, it was pretty much possible to get a good-enough structure of any particular protein you really cared about (from say 2005) by other means, namely Baker’s Rosetta.
I constantly use AlphaFold structures today [1]. And AlphaFold is fantastic. But it only replaces one small step in solving any real-world problem involving proteins such as designing a safe, therapeutic protein binder to interrupt cancer-associated protein-protein interactions or designing an enzyme to degrade PFAS.
I think the primary achievement is that it gets protein structures in front of a lot more smart eyes, and for a lot more proteins. For “everyone else” who never needed to master computational protein structure prediction workflows before, they now have easy access to the rich, function-determinative structural information they need to understand and solve their problem.
The real tough problem in protein design is how to use these structure predictions to understand and ultimately create proteins we care about.
Agreed, the UX of a magnet timer on the fridge beats using any kind of smart device for the task of setting kitchen timers. Most of them start a simple count up if not programmed for a specific duration, so you can watch the seconds.
Most magnet timers also remember the last duration, so if you use the same timer a lot for the same task (tea, for example), it’s literally a single button press. The same operation on any kind of smart device contains a staggering number of steps, each of which requires cognition and attention.
Magnet timers are also super cheap, so you can get another one if you have two favorite durations. A simple solution meets a simple problem
Agreed on the hopes that these methods lead to novel biocatalysts (but they aren’t quite there yet).
David Baker’s lab has recently published on using their own diffusion model (RFdiffusion) to design novel biocatalysts that perform hydrolysis using a catalytic triad of serine, aspartic acid, and histidine, as well as an oxyanion hole, which is much more complex than the binders designed by AlphaProteo [1].
It gives me hope that we’ll soon be able to design biocatalysts as good as natural ones, but for any problem we care about.
Good point! And a related topic, we call the organism that lives happily in our gut E. coli and we also call the organisms that cause disease the same name. What’s the difference?
It turns out that the Escherichia coli (to spell out its Latin binomial) that cause disease are in some sense “diseased” themselves: the genes that enable them to be pathogenic, or make them pathogenic, I should say, are originally from a phage, a type of virus that infects bacteria [1]. In a manner that is not the same as, but conceptually similar to how HIV inserts its genes into the human’s genome, phages insert their genes (termed the “prophage”) into the bacterial genome.
In addition, most strains of pathogenic Escherichia are also holding on to an entirely separate, small, circular “genome” called a plasmid, also of exogenous origin, that contains additional genes that make them pathogenic.
So in addition to wide genome variation within the “species” (which is not really the same thing for bacteria as for mammals, mind you) of Escherichia coli, many subtypes have additional genetic material from endogenous sources that substantially changes their observed characteristics (phenotype).
The flip side of this is that progress in ML for biology is always going to be _slower_ than progress in ML for natural languages and images [1].
Humans are natural machines capable of sensing and verifying the correctness of a piece of text or an image in milliseconds. So if you have a model that generates text or images, it’s trivial to see if they’re any good. Whereas for biology, the time to validate a model’s output is measured more in weeks. If you generate a new backbone with RFDiffusion, and then generate some protein sequences with LigandMPNN, and then want to see if they fold correctly … that takes a week. Every time. Use ML to solve _that_ problem and you’ll be rich.
TFA mentions the difficulty of performing biological assays at scale, and there are numerous other challenges. Such as the number of different kinds of assays required to get the multimodal data needed to train the latest models like ESM-3 (which is multimodal, in this context meaning primary sequence, secondary structure, tertiary structure, as well as several other tracks). You can’t just scale a fluorescent product plate reader assay to get the data you need. We need sequencing tech, functional assays, protein-protein interaction assays, X-ray crystallography, and a dozen others, all at scale.
What I’d love to see companies like A-Alpha and Gordian and others do is see if they can use the ML to improve the wet lab tech. Make the assays better, faster, cheaper with ML. Like how they use ML to translate the electrical signals of DNA passing through the pore into a sequence in the Nanopore sequencers. So many companies have these sweet assays that are very good. In my opinion, if we want transformative progress in biology, we should spend less time fitting the same data with different models, and spend more time improving and scaling wet lab assays using ML. Can we use ML to make the assay better, make our processes better, to improve the amount and quality of data we generate? The thesis of TFA (and experience) suggests that using the data will be the easy part
Probably worth mentioning that David Baker’s lab released a similar model (predicts protein structure along with bound DNA and ligands), just a couple of months ago, and it is open source [1].
It’s also worth remembering that it was David Baker who originally came up with the idea of extending AlphaFold from predicting just proteins to predicting ligands as well [2].
Unlike AlphaFold 3, which predicts only a small, preselected subset of ligands, RosettaFold All Atom predicts a much wider range of small molecules. While I am certain that neither network is up to the task of designing an enzyme, these are exciting steps.
One of the more exciting aspects of the RosettaFold paper is that they train the model for predicting structures, but then also use the structure predicting model as the denoising model in a diffusion process, enabling them to actually design new functional proteins. Presumably, DeepMind is working on this problem as well.
As cool as this is, the word “microplastics” is a little misleading. There are dozens of types of plastic in common use, each made from a different monomer with a different chemical linkage, of which PET is only one. The engineered protein in TFA will only work on PET and we’ll need to design new proteins for the other types of plastic. (I can help with that.)
The problem with enzymes eating plastic is that enzymes are small Pacman-shaped protein blobs that are maybe 10 nanometers in diameter, whereas things made of plastic like bottles or even microplastics are huge in comparison. How do you get the little Pacman jaws around the bottle to start breaking it down?
The research paper [1] describes the authors’ effective innovation. They make a protein where one end is a pore-forming shape, and the other end is a PET cutting (called a PETase in the jargon of the field). This way, their protein can access nooks and crannies in the macroplastic shapes, allowing tons of copies of this small enzyme to fully degrade a bottle.
Without this, a great deal of physical agitation is required to break down the plastics into small enough chunks that earlier Pacman enzymes could work on, increasing the time and the cost.
I hope we’ll see the idea of linking the enzymatic “scissors” to a protein pore be used to engineer enzymes to degrade other types of plastics in the future, as the general idea of getting the catalytic machinery into physical contact with every bit of the bottle is broadly applicable to all plastics, not just PET (which is great news)
Great article. I think we find time and time again that the best leaders are really “cheerleaders”. It turns out that inspiring people, personally living the values you espouse, and giving your team trust and respect is the key to success.
In my experience, people excel in environments where they are given a high level of trust, autonomy, and clear goals on the timescale of months and years.
Teams on the rack of egotistical middle managers get stretched thin until they break. There aren’t a lot of good examples of “micromanagitis” being cured. So-called leaders taking credit for their team’s work leads to their team not trusting that leadership has their growth in mind. Art Blakey’s real legacy is both the amazing players he mentored and tutored, but also the attitude that if you give people the tools they need, they’ll excel
I think parent has hit on the how and GP has hit on the why.
How LLMs are able to give convincing wrong answers: they “can predict the correct ‘shape’ of an answer” (parent).
Why LLMs are able to give convincing wrong answers is a little more complicated, but basically it’s because the model is tuned by human feedback. The reinforcement learning from human feedback (RLHF) that is used to tune LLM products like ChatGPT is a system based on human ranking. It’s a matter of getting exactly what you ask for.
If you tune a model by having humans rank the outputs, despite your best efforts to instruct the humans to be dispassionate and select which outputs are most convincing/best/most informative, I think what you’ll get is a bias towards answers humans like. Not every human will know every answer, so sometimes they’ll select one that’s wrong but likable. And that’s what’s used to tune the model.
You might be able to improve this with curated training data (maybe something a little more robust than having graders grade each other). I don’t know if it’s entirely fixable though.
The brilliant thing about the parent’s comment about the “shape” of the answer is that it reveals how much humans have (uh, historically, now, I guess) relied on the shape of information to convey its trustworthiness. Expand the notion of “shape” a bit to include the medium. If somebody bothered to take the time to correctly shape an answer, we take that as a sign of trustworthiness, like how you might trust something written in a carefully-typeset book more than this comment.
Surely no one would take the time to write a whole book on a topic they know nothing about. Implies books are trustworthy. Look at all the effort that went in. Proof of effort. When perfectly-shaped answers in exactly the form you expected are presented in a friendly way and commercial context, they certainly read as trustworthy as Campbell’s soup cans. But LLMs can generate books worth of nonsense in exactly the right shapes without effort, so we as readers can no longer use the shape of an answer to hint at its trustworthiness.
So maybe the answer is just to train on books only, because they are the highest quality source of training data. And carefully select and accredit the tuning data, so the model only knows the truth. It’s a data problem, not a model problem
When evaluating this work, it’s important to remember that the functional labels and protein family assignments on each of the 280 million input sequences were originally assigned by an HMM model using human curated sequence groups as part of the pfam project, so the model is predicting a prediction (or perhaps conditioned on a prediction would be more accurate).
Furthermore, the authors must engage a lot of human curation to ensure the sequences they generate are active. First, they pick an easy target. Second, they employ by-hand classical bioinformatics techniques on their predicted sequences after they are generated. For example, they manually align them and select those which contain specific important amino acids at specific positions which are present in 100% of functional proteins of that class, and are required for function. This is all done by a human bioinformatics expert (or automated) before they test the generated sequences. This is the protein equivalent of cherry-picking great examples of, for example, ChatGPT responses and presenting them as if the model only made predictions like that.
One other comment, in protein science, a sequence with 40% identity to another sequence is not “very different” if it is homologous. Since this model is essentially generating homologs from a particular class, it’s no surprise at a pairwise amino acid level, the generated sequences have this degree of similarity. Take proteins in any functional family and compare them. They will have the same overall 3-D structure—called their “fold”—yet have pairwise sequence identities much lower than 30–40%. This “degeneracy”, the notion that there are many diverse sequences that all fold into the same shape, is both a fundamental empirical observation in protein science as well as a grounded physical theory.
Not to be negative. I really enjoyed reading this paper and I think the work is important. Some related work by Meta AI is the ESM series of models [1] trained on the same data (the UniProt dataset [2]).
One thing I wonder is about the vocabulary size of this model. The number of tokens is 26 for the 20 amino acids and some extras, whereas for a LLM like Meta’s LLaMa the vocab size is 32,000. I wonder how that changes training and inference, and how we can adopt the transformer architecture for this scenario.
Nest still sends me a monthly summary email for account that’s maybe nine years stale, despite my repeated attempts to get them to stop. Their customer support says there’s “nothing they can do”.
The Nest smoke alarms at a friend’s house say (as in play over the speakers a voice saying) “Remote access attempt detected” every so often while we’re sitting in the living room. At least twice a year they go off in the middle of the night (without smoke).
So I guess when you’re contacting customers who don’t want it, and your alarms wrongly sound, I would say you have a problem with your false positive rate
And quite a bit longer than that even for the human brain to convolve safely with its surroundings and with other human brains.
One yet further objection to the many excellent already-made points: the deployment of LLMs as clean-slate isolated instances is another qualitative difference. The human brain and its sensory and control systems, and the mind, all coevolved with many other working instances, grounded in physical reality. Among other humans. What we might call “society”. Learning to function in society has got to be the most rigorous training for prompt injection I can think of. I wonder how a LLM’s know-it-all behavior works in a societal context? Are LLMs fun at parties?
Do yourself a favor and skip right through to the Twitter link to another link to this excellent post by Yoav Goldberg [1] on the actual reason that training new models on ChatGPT output in the manner of supervised learning (in contrast to reinforcement learning) will not produce a model as good as ChatGPT
>For this type of interaction, we must use RL training, as supervised training teaches the model to lie. The core issue is that we want to encourage the model to answer based on its internal knowledge, but we don't know what this internal knowledge contains. In supervised training, we present the model with a question and its correct answer, and train the model to replicate the provided answer.
The author says he’s summarizing a talk by John Schulman of OpenAI [2] but I haven’t personally watched the video. In any case, this is an interesting insight.
Say we set up a supervised learning scenario where we ask the model to use its internal knowledge to answer a question and compare its answer to one written by a human. If the two answers essentially say the same thing, but in different words, in the supervised learning case the model is penalized. In the RL case, it’s rewarded. That’s the difference.
If I may suggest a way to train yourself up on this, I’d like to suggest an approach which I learned from the writer Francine Prose in one of her composition courses.
The method is simply this: for half a year, every week, go and find the same news story from three different publishers and read all three versions.
Do this solidly 25 times and you will begin to see the outlines of exactly the set of skills that you need to objectively classify information that’s found on the Internet. This is a simple and highly effective training technique that uses the mass variety of news sites on the Internet for an educational purpose. Once you begin to see and compare the narratives underlying most “news”, they will be impossible to un-see
I’m reminded by a speech by Frederick Douglass [1] that
>This struggle may be a moral one, or it may be a physical one, and it may be both moral and physical, but it must be a struggle. Power concedes nothing without a demand. It never did and it never will. Find out just what any people will quietly submit to and you have found out the exact measure of injustice and wrong which will be imposed upon them, and these will continue till they are resisted with either words or blows, or with both. The limits of tyrants are prescribed by the endurance of those whom they oppress. (Emphasis mine)
My introduction to calculus was “Calculus Made Easy” by Silvanus P. Thompson and I always liked math profs who actively worked to show math for what it is: useful, beautiful but not about the symbols or the jargon. “Any fool can calculate!” I think is what he says in the book.
I did some math in college and when I started knowing how to analyze the behavior of functions (and developing the mental math tools to imagine what they look like without having to actually draw them) that’s when I felt like I was kinda getting it
>if LLM-generated content outpaces human-generated content, the useful data proportion will diminish
I guess I’d ask why the author thinks that training LLMs on their own output will make them worse. Like, if the problem is that LLM-generated content is less useful than human-generated content because it’s “just averaging out inputs” (paraphrase of common argument, not quote from TFA), how does adding more data at the average change the distribution?
>As is now, LLMs regularly hallucinate, generate biased content or fundamentally misinterpret the task even though nothing in the wider world has been adversarial to them.
This really got me thinking about what is meant by “adversarial”. As in, adversarial with whom? The model itself? Its deployers?
If I successfully trick ChatGPT, the system, into telling me some secrets about its inner workings, we can call that an attack on the commercial project as released by OpenAI, but can we call it an attack on the model itself?
All the text used to train LLMs is heavily processed and filtered already. I think it’s more likely that, rather than LLM-made text diluting out the good training data, it will simply add to the corpus. Might add a few cycles to the line-level duplication step
The translation of the Gagarin’s letter in this article, which explains that it’s “translated by Yves Gauthier … author of a magnificent biography in French on Youri Gagarine” reads better than the translation in TFA, in my opinion
It’s a fun time to be interested in AI for proteins. Every new ML model type is inevitably tried out on proteins. As the functional molecules of life, proteins are uniquely important and fundamental to every process in biology. As the targets for every drug, the tools for every cellular job, and the squishy, wiggly, moving and alive parts of living things, proteins presents both exciting possibilities and deep technical challenges for those who design them. A protein can be understood simply as a string of letters about 300 long using the alphabet ACDEFGHIKLMNPQRSTVWY. This turns out to be a great representation for sequence models like transformers. One big public database, UniProt, has 200 million protein sequences you can train your model on.
The very largest plain transformer models trained on protein sequences (analogous to plain text) are about 15B parameters (I am thinking of Meta AI’s ESM-2 [1]). These can do for protein sequences what LLMs do for text (that is, they can “fill in the blank” to design variations, generate new proteins that look like their training data), and tell you how likely it is that a given sequence exists.
Some cool variations of transformers have applications for protein design, like the now-famous SE(3) equivariant transformer used in the structure prediction module of AlphaFold [2], now appearing in the research paper [3] accompanying TFA, as well as variations on the transformer such as the message passing model ProteinMPNN [4], which builds on a neighbor graph-structured transformer [5]
Malaria’s complex lifecycle [1] seems like it would be easy to “break” with different interventions, but we’ve seen historically malaria has been difficult to eradicate. Why is this?
1. https://en.m.wikipedia.org/wiki/Plasmodium#/media/File%3ALif...