The point is that LLMs can’t backtrack after deciding on a token. So the probability at least one token along a long generation will lead you down the wrong path does indeed increase as the sequence gets longer (especially since we typically sample from these things), whereas humans can plan their outputs in advance, revise/refine, etc.
Nice post! I work on NLP and I think a lot of ideas in this post resonate with what I find exciting about working on the intersection of language + the real world: large text datasets as sources of abundant prior knowledge about the world, structure of language ~ structure of concepts that matter to humans, etc.
I feel like the bottleneck is getting access to paired (language, other modality) data though (if your other modality isn't images). i.e. "bolt on generalization" is an intuitively appealing concept, but then it reduces to the hard problem of "how do I learn to ground language to e.g. my robot action space?" I haven't seen a robotics + language paper that actually grapples with the grounding problem / tries to think about how to scale the data collection process for language-conditioned robotics beyond annotating your own dataset as a proof-of-concept. Unlike language modeling / CLIP-type pretraining, it seems (fundamentally?) more difficult to find natural sources of supervision of (language, action). I'd be curious about your thoughts on this!
> When it comes to combining natural language with robots, the obvious take is to use it as an input-output modality for human-robot interaction. The robot would understand human language inputs and potentially converse with the human. But if you accept that “generalization is language”, then language models have a far bigger role to play than just being the “UX layer for robots”.
You should check out Jacob Andreas's work, if you haven't seen it already - esp. his stuff on learning from latent language (https://arxiv.org/abs/1711.00482).
> My theory is that taste is one quality that separates the academics from the business people. Academia doesn't necessitate a lot of taste. If you have it, great. If you don't have it, no big deal.
This might be true for academics in ancient Greek literature, but certainly isn't true for academics in CS nowadays. If you don't have good taste in research problems that are {important for downstream industry applications, scientifically interesting, tractable}, you won't get anything done, and you won't get published. If anything, the pressure for academics to develop good taste is stronger than for people designing product. You can have a product that provides just one utility that users desperately need and have terrible taste for all the other axes that make a product "good," and do just fine. Academic papers get judged (in peer review / traction after publication) purely against the taste and aesthetics of other people in your community.
Yeah, it's an excellent course. That is typically the unfortunate downside of self-studying courses, but AFAIK the tests aren't that sophisticated - you can mostly test it yourself just by running the test suite over and over again, making sure that it _really_ works 100% of the time and not just 98% of the time (given the nondeterministic nature of distributed systems failures). This is basically what students would do before submitting their labs, and most people got a full score if they ran it enough times :P
The difference between this and most other incubators (at Stanford, Harvard iLab, etc as mentioned below) is that it's very specifically geared towards hard-tech startups that other investors typically shy away from because of the long time to market.