One of the interesting things about the lambda calculus is its universality: by itself, it's a complete foundation for computation.
Here's a different old post of mine showing how to build the rest of the programming language, all in a miniscule subset of Python that is the pure lambda calculus:
You can even extract recursion out of the Y combinator or the more primitive U combinator -- out of nothing but lambdas!
So, it's lambdas all the way down.
Another interesting thing about the lambda calculus is that it wasn't intended to be a programming language. When Alonzo Church created it, there were no computers to program.
Alonzo Church was trying to solve problems in the foundations of mathematics.
But, untyped lambda calculus has a "bug" that makes it problematic for mathematics -- the self application that enables recursion is a problem if you're a logician who cares about soundness, but it's fantastic if you're a programmer.
I don't think of functional languages as obfuscating. I think of them as terse and expressive. They let me most directly encode the model in my head as running code.
Wondering the same, and in somewhat different terms.
And as models shrink in size yet go up in intelligence and performance, I'm finding ever more life in older hardware.
When I got my M1 Max in 2021, GPT-3 was about 1.5 years old and it was SOTA.
Yet, that machine is now able to run models that crush with gpt4, and even compete with o1 (SOTA from about 1.5 years ago.)
The idea that I could run something like that locally would have seemed absurd in 2021.
Yet, if somehow I'd had those local models in 2021 on the exact same hardware, I would have had, by far, the most powerful AI on the planet -- and that would have remained true for the next several years.
I'm also noticing that the ever-improving smaller models I can run on this machine are crossing the "good enough" threshold for ever more tasks by the month.
I just don't need a frontier model for every task.
I have an M4 Max 128 GB RAM now, but I still find plenty of tasks to delegate to the M1 Max machine.
I don't know how far this can go in the limit in terms of packing more intelligence into smaller models, but older hardware, if maintained well, seems like it's going to increase the value it can deliver in terms of "intelligence per watt-hour."
Nothing to feel bad about. Thank you for sharing that too.
My son’s life changed my own in profound ways, and even though he died four years ago, he is still changing my life in profound ways. I am always grateful for the reminder and to reconnect with the purpose that his life gave to mine.
That post also reminds me that while he was alive, I did the best I could for him under my abilities, and that’s all any parent can do in the end.
Original author of the guide here. Wonderful to see these little illustrations still making the rounds. I first published them in 2010!
To those in the comments who mentioned you are just starting your own PhD: Good luck to you! And, I hope you, like I once did, find a problem that you can fall in love with for a few years.
To those just finished: Congratulations! Don’t forget to keep pushing!
To those many years out: You have to keep pushing too, but there can be tremendous value in starting all over again by pushing in a different direction. You have no idea what you may find between the tips of two fields.
Same! I use ollama a lot, but when I need to do real engineering with language models, I end up having to go back to llama.cpp because I need grammar-constrained generation to get most models to behave reasonably. They just don't follow instructions well enough without it.
I missed blogging too, so I set a New Year’s resolution in January to write at least two blog posts this year. It had been over seven years since my last!
I still hadn’t written one by November, but now I’ve got three for the year.
One of the interesting things about the lambda calculus is its universality: by itself, it's a complete foundation for computation.
Here's a different old post of mine showing how to build the rest of the programming language, all in a miniscule subset of Python that is the pure lambda calculus:
https://matt.might.net/articles/python-church-y-combinator/
You can even extract recursion out of the Y combinator or the more primitive U combinator -- out of nothing but lambdas!
So, it's lambdas all the way down.
Another interesting thing about the lambda calculus is that it wasn't intended to be a programming language. When Alonzo Church created it, there were no computers to program.
Alonzo Church was trying to solve problems in the foundations of mathematics.
But, untyped lambda calculus has a "bug" that makes it problematic for mathematics -- the self application that enables recursion is a problem if you're a logician who cares about soundness, but it's fantastic if you're a programmer.
I don't think of functional languages as obfuscating. I think of them as terse and expressive. They let me most directly encode the model in my head as running code.