China will accelerate on ML research and optimization regardless of US export control policy. They want to win or at least not lose just as much as the US and will pull every reasonable lever at their disposal to do so. NVIDIA and the US government have no say in this.
Google can't compete with China, neither can Meta. Only two labs in the US can keep chucking billions at the frontier race. Everyone else has a real business to run.
China can keep up because it's cheaper to run a frontier lab there. They also have more researchers and a stronger cultural inclination for this sort of thing. And I guess the business case in China doesn't have to work as well as it does in the US.
Correct. We need open weights, open code and open data. If nobody else can reproduce what someone did there will always be security questions. Even if we can reproduce it there could still be security concerns but it's more realistic to investigate yourself.
The compute constraints never mattered. If China had more compute they'd still end up winning because they have more people and a culture more inclined to math and science.
Even if you find all this amusing, there's no own goal here. Not a policy one anyway.
Science fiction has entertained and inspired millions of people and we should all be grateful for that but it has also distorted what people think space really is.
When you consider the scale of space it becomes pretty understandable why the Milky Way isn't teeming with civilizations sending large amounts of mass all over the galaxy. A realization one comes to despite the facts that it has taken humans a blink of an eye (on a galactic timescale) to go from tools to rockets and the Milky way is billions of years older than the entire history of the Earth.
For better or worse, humans (or any animal) are a lot better at reacting than planning. I'm sure this technology will play out differently than any one of us, or any collection of us, can imagine. The possibility space is enormous.
My main takeaway from LeCun's thesis isn't that you can't build LLMs to do useful things better than the best human, it's that these systems don't learn arbitrary skills efficiently, like humans do. And the question is, why not? 8% on ARC-AGI-3 is amazing for a machine considering how far we've come since digital computers were first built. But it is pretty poor if you're claiming something is well on its way to exhibiting human-like intelligence.
Mythos can do some amazing things (I'm assuming, I've never seen it). A young child can learn to control its body without reading any books on dynamical systems and kinematics. Mythos cannot learn to control a humanoid robot after sucking in every piece of data Anthropic can get their hands on.
2024 is a good cut-off. I use these additional heuristics. Bad cover art. Not from a real publisher or not from an author that hasn't published before 2024.
I hope that real books that just happen to slip through the cracks, that are also genuinely good books will eventually surface as easily findable somehow. Maybe that's wishful thinking, I don't know.
I know one thing. It's grim out there for publishing and real humans who have something good to say but don't have a popular voice.
I didn't mean nobody lives there. I meant if you plucked random people from all over the country and told them to relocate to NYC, odds are they would need a massive income increase to survive. And even then, it would be touch-and-go for most.
I'm from Harlem, and other parts of NYC. I moved out to pursue a career in an industry that is relatively non-existent in the NYC metro area. If I moved back now I could probably afford it on my current salary. But there are no jobs there for what I do. And if there was a bigger tech industry in NYC the costs would likely be even higher.
Religion used to be the biggest supporter of science. Then science surpassed religion in civilization relevancy. Now conservative politics is stepping up to the fight.
> how you guys handle the fact this leaks all your code and is stored forever on servers belonging to God knows who?
I don't.
My company doesn't host any code on GitHub, we have our own Git servers.
Regarding the privacy issues with OpenAI, Anthropic or anyone else, we engineers are just using the tools we've been authorized to use by the company. If there's a security issue, that would need to be worked out between the company and the model providers.
Would I use external model providers through an API for personal projects? For everything I'm doing now, probably. Could I see a day where I'm working on something too sensitive to be willing to give any data to these companies? Possibly.
I don't even look at benchmarks anymore. I just try different models as they're released on our large, proprietary, systems software codebases in real, shipping products or projects that will ship eventually. It's pretty clear which models help me do my job better or faster. I'm fortunate enough to have the token budget to use basically as much as I need, for now.
No need for benchmarks, evals, marketing, system cards or anything like that. I read the web for tips, practices and release announcements. My colleagues and I share our experiences with each other but beyond that, everything else is just noise.
Current: Embedded software, robotics.