There is actually a whole lot of research around the "use less data" called data pruning. The goal in a lot of cases there is basically to achieve the same performance with less data. For example [1] received quite some attention in the past.
My website of choice whenever I have to deal with references is dblp [1]. In my opinion more reliable than Google scholar in creating correct BibTeX. Also when searching for a paper you clearly see where it has been published or if it is only on arxiv.
Nassim Taleb (black swan author) does not divide his books into chapters. Instead each book consists of several (sub-)books. In Antifragility he explains that two subbooks from the same book have the same distance to each other than one subbook from book to one from another book. I found that quite interesting.
Yeah, the debris removes itself with time. But there is still some uncertainty around how deorbiting harms the atmosphere [1]. Immediate effects of a single deorbit seem to be quite low but having such deorbiting scenarios at a higher frequency could have some negative effects.
The conventional teaching that I am aware of says that you can scale across three dimensions: data, compute, parameters. But Ilya's formulation suggests that there may be more dimensions along which scaling is possible.
For everyone interested in what makes a classifier put an image in a certain class I can recommend the excellent paper by Robert Geirhos et al. "ImageNet-trained CNNs are biased towards texture" [1]. The paper is a bit older but it's an excellent read.
When the policy is deployed in the real world only the depth camera is used no waypoints etc.. Scan dots and target heading is used in the first Phase of the training to pretrain a policy in simulation. In Phase 2 a policy is trained end-to-end using the pretrained actor network: "First, exteroceptive information is only available in the form of depth images from a front-facing camera instead of scandots. Second, there is no expert to specify waypoints and target directions, these must be inferred from the visible terrain geometry." For policy training in Phase 2 DAgger which is based on Behavior Cloning is used (with the policy from Phase 1 as the expert), they also use some tricks to make sure no actions that are too different from the expert actions are executed during training. In Phase 2 the network learns to extract environment information instead of from the scan dots from the depth camera. Also in Phase 2 they use the pretrained actor network from Phase 1 but the depth embedding must be learned from scratch. This is how I understand it.
Note that it is from 2018. As someone here already mentioned there is a paper that applies the same idea to Vision Transformers published this year [1].
I find it interesting that Notion still uses the Somali .so top level domain. I heard from people not able to access Notion inside their company network because their company blocked the .so domain.