The reality is that o1 is a step away from general intelligence and back towards narrow ai. It is great for solving the kinds of math, coding and logic puzzles it has been designed for, but for many kinds of tasks, including chat and creative writing, it is actually worse than 4o. It is good at the specific kinds of reasoning tasks that it was built for, much like alpha-go is great at playing go, but that does not actually mean it is more generally intelligent.
In my younger years, particularly during my schooling, I held a deep resentment towards the educational system. It felt overtly clear to me, as a student, that schools failed to effectively foster learning and growth. However, my perspective has evolved over time. I've come to understand that the issues I observed are not unique to the school system but rather characteristic of large institutions as a whole.
The pervasive failure of these institutions to meet their stated objectives isn't an isolated phenomenon. It's symptomatic of a larger, systemic problem – the widespread presence of perverse and misaligned incentives at all levels within large organizations.
Unless we find a way to counteract this, attempts at reform will merely catalyze further expansion and complexity. The uncomfortable truth is, once an organization surpasses a certain size, it seems to take on a 'life of its own', gradually sacrificing its original mission to prioritize self-preservation and expansion. Who has ever seen an organization like this voluntarily reform itself? I certainly haven't.
It depends on what you mean by "this discussion", but I don't think that follows.
If for example, we were in scenario 2 and it was still the case that a large number of people thought AI doomsday was a serious risk, then that would be a much stronger argument for taking the idea of AI doomsday seriously. If on the other hand we are in scenario 1, where there is a long history of people falling prey to apocalypticism, then that means any new doomsday claims are also more likely to be a result of apocalypticism.
I agree that is is likely that humans will go extinct eventually, but I am talking specifically about AI doomsday in this discussion.
>It was clear that nukes were a risk before they were used; that is why there was a race to create them.
Yes, because there were other kinds of bombs before then that could already kill many people, just at a smaller scale. There was a lot of evidence that bombs could kill people, so the idea that a more powerful bomb could kill even more people was pretty well justified.
>if AGI is to become a thing, what does the moment look like where we can see it is coming and still have time to respond?
I think this implicitly assumes that if AGI comes into existence we will have to have some kind of response in order to prevent it killing everyone, which is exactly the point I am saying in my original argument isn't justified.
Personally I believe that GPT-4, and even GPT-3, are non-superintelligent AGI already, and as far as I know they haven't killed anyone at all.
> So far, 100% of those which are as intelligent as humans have dominated the planet.
This is a much more subjective claim than whether or not the world has ended. By count and biomass there are far more insects and bacteria than there are humans. It's a false equivalence, and you are trying to make my argument look wrong by comparing it to an incorrect argument that is superficially similar.
1) Throughout history many people have predicted the world would soon end, and the world did not in fact end.
2) Throughout history no one predicted the world would soon end, and the world did not in fact end.
The fact that the real world is aligned with scenario 1 is more an indication that there exists a pervasive human cognitive bias to think that the world is going to end, which occasionally manifests itself in the right circumstances (apocalypticism).
Well, for example I believe that nukes represent an existential risk, because they have already been used to kill thousands of people in a short period of time. What you are saying doesn't really counter my point at all though, it is another vague theoretical argument.
Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations throughout history who have made similar vague theoretical arguments that the world would end and they were all wrong.
Sure, but don't they have a vast network of contact tracing? Presumably if a bunch of cases showed up that weren't connected to any other cases because of the lockdowns then that would be a really good indication of where the animal reservoir was located. But they haven't found the animal reservoir yet either, even with all of the lockdowns.
Well its been years, why haven't they found the reservoir yet then? They would obviously want to since it would prove that it wasn't a lab leak, yet as far as I know they haven't claimed to have found it yet.
IMO, it seems obvious from the behavior of China's government that they know it is a lab leak. If it wasn't a lab leak, then presumably there is an animal reservoir of the virus somewhere in China, but as far as I know they haven't claimed to have found it. But if there is an animal reservoir of the virus in China, then how could the Chinese government ever expect a lockdown to work? A lockdown on travel would only really prevent the virus spreading from people bringing it into the country but obviously wild animals would still be spreading it. Yet the Chinese government claimed that their lockdowns did work. How is that at all compatible with the virus being from wild animals and not being a lab leak? It doesn't make any sense.
There is a lot of negativity towards the idea of AI in this thread, and I feel like someone has to say it: it is quite likely that in the near future computers will be better than almost all humans at almost all cognitive tasks.
If you have a task or are trying to accomplish something, and the way you do it is by moving a mouse around or typing on a keyboard then it is very likely that an AI will be able to do that task. Doing so is a more or less straightforward extension of existing techniques in AI. All that is necessary is to record you performing the task and then an AI will be able to imitate your behavior. GPT3 can already do this for text, and doing it instead with trajectories of screen, mouse and keyboard is not fundamentally different.
So yes, it is true that there is a lot of hype right now, but I suspect it is a small fraction of what we will see in the near future. I also expect there will be an enormous backlash at some point.
I don't get why everyone assumes that Havana syndrome is fake. I mean it could be fake but it doesn't seem implausible that it is real either. I seems like everyone jumps to the conclusion it isn't real though.
It's a cybernetic feedback system. Dalle is used to create new images, the images that people find most interesting and noteworthy get shared online, and reincorporated into the training data, but now filtered through human desire.
Sure, maybe it is possible if the residents are okay with the CCP taking over, but on the margin everyone being armed makes them taking action much more costly. Guerrillas were able win against the US in Vietnam, and Mao used guerrilla warfare himself and won against a better equiped military. https://en.wikipedia.org/wiki/On_Guerrilla_Warfare
Yes, I think if literally every single person in Hong Kong had a gun that would be enough to stop the CCP. The entire active military of China is about 2 million people, less than a third of the approx. 7.5 million people in HK. Maybe the CCP could bomb the city into dust, but taking it over intact would be a practical impossiblity.
"Political power grows out of the barrel of a gun." The thing that could have saved HK was legalization and widespread private gun ownership. Yes, there would have been more gun crime, but the victims of all of the random gun crime in the world is a tiny fraction compared to the victims of communism.
Thanks for the recommendation. I am a big fan of Weber, not familiar with Deming but his work sounds very relevant. In general I tend to agree that beyond a certain size organization these problems seem unavoidable. I've read Systemantics/The Systems Bible and it seems to come to a similar conclusion.
When I was younger I was aware of the idea of perverse and misaligned incentives, but I never would have expected the extent to which they pervade practically every human institution.