I really don’t understand the people who have a problem with rust. Do you not value the increased memory safety? Now that Microsoft and Google are adopting rust and reporting significant decreases in memory related bugs it’s pretty clear that rust does make a difference.
Thanks, can you explain what you mean by “vouched”? I’ve noticed that my comments have been getting much less engagement recently and sometimes they don’t show up.
Perhaps the biggest issue is the mental framework that people use to approach AI. I've found that there are so many assumptions in people's thinking and these assumptions are strange and/or don't match up with the evidence we have so far.
First of all, you have to ask the question "what is intelligence?". What I've found is most people think intelligence is deeply connected to humanity or that intelligence is synonymous with knowledge. Really, intelligence is the ability to reason, predict, and learn. It's the ability to see patterns in the world, learn and act on those patterns. It doesn't have to be human-like. It doesn't mean emotions, wants, dreams, or desires. It's cold, hard logic and statistics.
Secondly, you have to ask "do I think it's possible for computers to be intelligent?". A lot of people have issues with this as well. The thing is that if you say "no, computers can't be intelligent" you are basically making a religious claim because we have brains and brains are intelligent. We can literally grow intelligence inside a human being during pregnancy. It might be difficult to program intelligence, but saying it's impossible is a bold claim that I don't find very convincing.
Third, you have to ask "if a computer is intelligent then how does it act?". So far the closest thing we have to general intelligence is an LLM model like GPT and even then it's questionable. However, reports indicate that after initial training these models don't have a moral compass. They aren't good or evil, they just do whatever you ask. This makes sense because, after all, they are computers right? Again we have to remember that computers aren't humans. Intelligence also means OPTIMIZATION, so we also have to be careful we don't give the AI the wrong instructions or it might find a solution that is technically correct but doesn't match up with humans wants or desires.
Four, you have to ask "can we control how these models act?" and the answer seems to be kinda but not really. We can shift the statistics in certain ways, like through reinforcement learning, but as many have found out these models still hallucinate, and can be jail broken. Our best attempts to control these models are still very flawed because basically an LLM is a soup of neural circuits and we don't really understand them.
Fifth, you have to ask "ok, if a computer can be intelligent, can it be super intelligent?". Once you've gotten this far, it seems very reasonable that once we understand intelligence we can just scale it up and make AI's super intelligent. Given the previous steps we now have an agent that is smarter than us, can learn and find patterns that we don't understand, and act in ways that appear mysterious to us. Furthermore, even if we had solid techniques to control AIs, it's been shown that as you scale up these models they display emergent behaviors that we can't predict. So this thing is powerful, and we can't understand it until we build it. This is a dangerous combination!
Finally, add in the human element. All along the way you have to worry about stupid or evil humans using these AIs in dangerous ways.
Given all of this, anyone who isn't a bit scared of AI in the future is either ignorant, superstitious, or blinded by some sort of optimism or desire to be build a cool sci-fi future where they have space ships and robots and light-sabers. There are so many things to be worried about here. The biggest point is that intelligence is POWER, it's the ability to shape the world as one sees fit whether that's the AI itself or humans who program it.
I’m curious why you are so confident in your assertion. It seems to me that an advanced statistical model of the world is an essential component of AGI. How do you know that we aren’t a few breakthroughs away from AGI?
Some recent papers have shown significant performance improvements when these models are allowed to respond to their own outputs.
How do you know that putting an LLM in a fancy loop with access to external memory and tools isn’t AGI?
I don't think that's necessarily proof against the Bayesian brain. It seems reasonable that the brain is also using its statistical models to assess the relevance of new evidence. So it's not just "new evidence, I need to update" but more like "new evidence, how likely is this true? I'll update according to the magnitude of the likelihood."
I'm skeptical that RNNs alone will outperform transformers. Perhaps some sort of transformer + rnn combo?
The issue with RNNs is that feedback signals decay over time, so the model will be biased towards more recent words.
Transformers on the other hand don't have this bias. A word 10,000 words ago could be just as important as a word 5 words ago. The tradeoff is that the context window for transformers is a hard cutoff point.