I remember those days. They released smaller versions of the model that were interesting but pretty useless. The wild part is, about 5 years later I was able to train the same model from scratch, using a couple of gpus in my home office, following Kaparthy’s online lectures, for a fraction of the cost. Long Live Open Source!
I agree. I like using Antigravity for some of my frontend work, and I find it does a better job than Claude Code - Opus 4.6. I’ve also found the Gemini Flash models to be good at legal defense research—I use them to help New Yorkers fight parking tickets (https://nyceasyparking.com). That said, the Claude models are still amazing at agentic work.
I’m working on yet another cloud based coding agent https://seniordev.io/ that connects to an existing GitHub repo, spins up a feature branch, commits incremental changes, and opens a PR. You can jump into an embedded VS Code server to review and tweak the code before merging—no local setup needed. Any feedback is greatly appreciated Thanks!
Great work! I love your videos; they've taught me so much. Any plans for a Mixture of Experts (MoE) video? My understanding is that starting from GPT4 most advance models use MoE to some extent. For example, can I take the model from your GPT2 video and just change the feed forward layer to an MoE layer like the one found here (1)? I guess I can just try it myself but I enjoy the expert guidance you provide in your videos. Please don't stop! great content!
The 555 timer is considered one of the most successful chip designs in electronics. It has a simple, well understood, design. The timing on the 555 can be configured to generate a periodic signal using just a few resistors and capacitors. The 555 is often used as a timer in electronic circuits. The Linux scheduler is usually configured to respond to interrupts generated by a cpu. To replace those cpu interrupts by a simple 555 timer is impressive due to the unconventional nature of the setup.
Hi apologies for any confusion. What i meant to say is that OpenWrt can be run on the Raspberry Pi. I did this because I had a Raspberry Pi laying around and did not want to invest in additional hardware just for experimenting with OpenWrt within my internal network. You can also run OpenWrt in a virtual environment like QEMU -- this is useful if you're aiming to connect multiple VMs to OpenWrt, effectively setting up a virtual network with OpenWrt at its core. Please see links below for more info:
It could be that the corpus ChatGPT was trained on is full of ‘confidently wrong’ answers from these ‘experts’. One solution could be to train these LLM on a higher quality corpus from real experts instead of random text from the internet. But would that just bring us back to the days of expert systems?
Wow this looks amazing! I can’t wait to get my copy. I wonder with all the available resources to learn electronics can someone become an electronics engineer without going to college (assuming enough mathematical maturity to understand Electrodynamics, etc)?
Nicely put! I think most "real" engineers dismissal of software engineering is because we can for the most part ignore physics. This somehow seems like cheating and not mathematical enough to be "real" engineering. I've even heard the claim that discrete mathematics, graph theory, and theoretical computer science is not "real" mathematics. It should be clear that a developing complex software system like Unix, Kubernetes, Redis is engineering, but somehow others from different engineering disciplines cannot see that. After bringing so many great inventions and innovations into this world, how can we once and for all make the case that software engineering is real engineering and get the respect that this field deserves?
Also, seems kind of rude as a teacher to erase equations off the board "before students could copy them". Would it hurt so much to ask, has every written this equation?
This is amazing technology. I have one question. How is security being addressed for this technology? Some pretty nasty bugs have recently been found on a particular bluetooth stack, now we are talking about pairing our brains through bluetooth. I would really hate to see what happens when somebody buffer overflows your Neuralink!
TCP/IP Illustrated, It was that first time I realized that all the magic the computer was doing could be understood on a bit by bit level. It launched me into a rewarding career.
Some hobbyists use these die shots to reverse engineer older chips, bypassing the need to decap the chips themselves. The craft is explained nicely in this video https://www.youtube.com/watch?v=aHx-XUA6f9g.
I agree. Keeping the test around provides a safety net (a bug repellent of sorts) for the next person the comes along. The test also serves as a documented history of the bugs of the past. The problem is that test does not guarantee the injection of future bugs of a different type. I don't think any amount of testing will. That's not to say we shouldn't write test, we should just be aware of the limits of testing.