- Diffusion models for video (see https://video-diffusion.github.io/, this paper is from April but I expect to see a lot more published research in this area soon)
- OpenAI Minecraft w/VPT (first model with non-zero success rate at mining diamonds in <20min)
- AlphaCode (from February, reasonably high success rate on solving competitive programming problems)
https://www.deeplearningbook.org/ and http://incompleteideas.net/book/the-book-2nd.html are excellent resources for supervised and reinforcement learning, respectively, and some knowledge of statistics and probability go a long way. But I think by far the most important thing is to just start training models, even very small ones, and developing an intuition for what works and what the failure modes are.
- Get really comfortable with matplotlib or your graphing library of choice. Plot your data in every way you can think of. Plot your models' outputs, find which samples they do best and worst on.
- Play around with different hyperparameters and data augmentation strategies and see how they affect training.
- Try implementing backprop by hand -- understanding the backward pass of the different layers is extremely helpful when debugging. I found Karpathy's CS231n lectures to be a great starting point for this.
- Eventually, you'll want to start reading papers. The seminal papers (alexnet, resnet, attention is all you need, etc) are a good place to start. I found https://www.youtube.com/c/YannicKilcher (especially the early videos) to be a very useful companion resource for this.
- Once you've read some papers and feel comfortable with the format, you'll want to try implementing something. Important tricks are often hidden away in the appendices, read them carefully!
- And above all, remember that machine learning is a dark art -- when your dataloader has a bug in its shuffling logic, or when your tensor shapes get broadcast incorrectly, your code often won't throw an error, your model will just be slightly worse and you'll never notice. Because of this, 90% of being a good ML researcher/engineer is writing tests and knowing how to track down bugs. http://karpathy.github.io/2019/04/25/recipe/ perfectly summarizes my feelings on this.
Huge thanks to the node team for adding this, I’ve been wanting fetch in node for years now! Installing node-fetch for every project was getting kind of old haha
Yeah, the stock driver assists in most cars still need a LOT of work. I drove from San Diego to LA last weekend with a comma, and openpilot handled the entire trip without a single disengagement until I got off the highway. I’d be really impressed to see a stock LKA manage the same (although I’ve heard great things about super cruise!)
Man, I saw that thing moving sideways right after liftoff and I thought it was a gonner! Huge congrats to SpaceX for landing with an offset engine like that on their first try!!!
Shoot you’re right, I read “warning light” and just assumed. Thanks for the correction! On the other hand, it’s even weirder that Boeing didn’t bother fixing it if it was just a software patch...
Yep. The "AOA sensor disagreement" light is built into every cockpit, but it won't turn on unless the airline purchased a specific premium add-on. Boeing claims they initially didn't realize the light was bundled as an add-on, and then once they did realize, they decided it wasn't critical to the airplane's safety and just kept selling it as an add-on, which is... kind of alarming, to put it mildly. https://www.nytimes.com/2019/05/05/business/boeing-737-max-w...
The success of Tesla as a business will have far more impact on our progress against global warming than CA’s regulatory goals will. Maybe some don’t see it that way, but I’m quite certain Musk does, and frankly I agree with him.
I think https://www.alexirpan.com/2019/10/29/openai-rubiks.html is a pretty good take. tldr: unfortunately, even tiny discrepancies between the simulation and reality (like the size of the cube's bezels or a few extra parts in the robotic hand) can significantly degrade your model's performance in the real world, even with tons of domain randomization during training. So, either we need to improve our simulations to randomize a lot more details, or we need to improve our neural nets to generalize more effectively.
Inter-frame jumpy-ness can be smoothed out slightly by the control algorithms, on the assumption of object permanence, as long as the underlying vision NN doesn’t miss things too often.
I’m kind of impressed by how smooth it is, actually. If you watch videos of state-of-the-art object localization NNs, they tend to be EXTREMELY jumpy. These neural nets usually operate on only a single frame at a time, at least in the lower layers, so their predictions tend to jump around a lot from frame to frame (especially when the camera is moving!)
They use both radar and cameras, but primarily cameras. Lines, road markings, stop lights, and street signs can only be detected with vision, but cars+pedestrians can (usually) be verified by both radar and cameras. In fact, they’ve mentioned that they use radar data to help train the camera neural nets to predict distances more accurately. However, radar alone isn’t sufficient for detecting obstacles — it sometimes misses vehicles with a lot of ground clearance, and sometimes registers false positives when driving underneath bridges and overpasses, resulting in phantom braking.
STRONGLY disagree. I’m just a hobbyist, but trying to read Keras models can be a god damn nightmare if the author has to do anything even slightly non-standard. Keras seems to REALLY want you to believe that you can just throw a bunch of layers together and call .fit and everything will just work, but it never seems to be that simple unless you’re training on MNIST or ImageNet.
I agree with this to some extent, but there are real advantages to having all your code in same framework, and PyTorch is significantly easier to iterate on and debug compared to TensorFlow (in my experience). Hopefully PyTorch will start offering better support for tensor processors like google’s TPUs, but from the sound of it, OpenAI is primarily using Azure for their training infrastructure and I don’t think Microsoft currently offers anything except GPUs.
Yeah probably true, I did say “approaches” though! My guess is that short interest will eventually drop below 5% over the next couple years if Tesla continues to perform well and grow their revenue stream, but we’ll see.
The base Model 3 costs $40k in the US, and even without any extra options at all, it’s a great car to drive IMO! (Although it IS a lot more expensive in foreign markets)
- Significantly-improved diffusion models (DALL-E 2, Midjourney, Stable Diffusion, etc)
- Diffusion models for video (see https://video-diffusion.github.io/, this paper is from April but I expect to see a lot more published research in this area soon)
- OpenAI Minecraft w/VPT (first model with non-zero success rate at mining diamonds in <20min)
- AlphaCode (from February, reasonably high success rate on solving competitive programming problems)
- Improved realism and scale for NeRFs (see https://dellaert.github.io/NeRF22/ for some cool examples from this year’s CVPR)
- Better sample efficiency for RL models (see https://arxiv.org/abs/2208.07860 for a recent real-world example)