Hawking: AI could end human race(bbc.com)
bbc.com
Hawking: AI could end human race
http://www.bbc.com/news/technology-30290540
117 comments
I disagree. If you could run a machine that performed equally well in a real-world environment as a human, at a lower cost than it costs to feed and pay a human laborer, it would be a no-brainer to invest in thousands of these machines to perform various consulting services that are bought and sold over the internet. And here I've only touched the slice of the world economy that exclusively consists of information-inputs being turned into information-outputs.
The market for general AI is massive. The only reason you can't see it is that no similar service exists. So this is not an argument.
(In fact, a similar service does exist: Indian IT outsourcing sweatshops, which while usually not delivering top-quality products still are massively profitable).
The market for general AI is massive. The only reason you can't see it is that no similar service exists. So this is not an argument.
(In fact, a similar service does exist: Indian IT outsourcing sweatshops, which while usually not delivering top-quality products still are massively profitable).
Why do you think we need general AIs to replace human labour?
I'm struggling to find any cost-intensive labour that would require anything beyond a small group of specialist AIs to replace human labourers.
Also, these discussions tend to assume that we (by which I mean those making these decisions) want to replace every profession with AIs. We don't. Even if we fully automate certain forms of labour, we still want human oversight, if only so we have someone to blame in case something goes wrong.
I'm struggling to find any cost-intensive labour that would require anything beyond a small group of specialist AIs to replace human labourers.
Also, these discussions tend to assume that we (by which I mean those making these decisions) want to replace every profession with AIs. We don't. Even if we fully automate certain forms of labour, we still want human oversight, if only so we have someone to blame in case something goes wrong.
Yes and no. Right to a certain degree, but AI taking away jobs may mean an economic chaos in third-world nations?
Why only third-world nations? The inhabitants of "third-world" countries do not produce lsss economic output than first-world countries because they are inferior in intelligence to people in the West. It's all in the education and culture, and there is no reason to believe that this could not be adapted if human-level AI is reached.
It is incredibly myopic to believe that the Western would would be shielded from the impact of such a development. But this is really beside the point I wanted to bring up. My key point is that this argument shows that AI could constitute a threat to humanity in the long term, and that it is necessary to provide more funding for research on AI safety. The economic incentives will bring this stuff to the center stage once it becomes possible, and by then we better have a damn good idea how to ensure that super-human AI keeps acting in our interests.
It is incredibly myopic to believe that the Western would would be shielded from the impact of such a development. But this is really beside the point I wanted to bring up. My key point is that this argument shows that AI could constitute a threat to humanity in the long term, and that it is necessary to provide more funding for research on AI safety. The economic incentives will bring this stuff to the center stage once it becomes possible, and by then we better have a damn good idea how to ensure that super-human AI keeps acting in our interests.
There is a huge 'market' for robots which can make money. Unfortunatelly if a robot could do my job or yours at least as well as we do (I am a software developer) that would worth lots of money to some business people... Obviously I don't want those robots to exist, but some business-people will always want the cheaper workforce. (Even if at the next level they will face robot competition too...) Now the question is whether can a robot be a good software developer (or UX designer, or project manager) without being very human like... Maybe, but I don't think so. I think if a program will be able to do our job well, that would be strong A.I.
I, oddly enough, want these AI to exist. It would signal the end of the industrial revolution (finally), and force us to consider doing other things than "work 40 hours a week." Assuming of course our AI overlords don't decide we are irrelevant and exterminate us all.
https://www.youtube.com/watch?v=_ZG8HBuDjgc#t=1985
(Douglas Adams talking, fairly humorously, about extinction)
"ending the human race" is such an interesting thing. Put aside that bone-headed nazis will tell you that humans have more than one race: We do extinct other species in incredible amounts (ask any greenpeace flyer). If we were extinguished by our own invention seems sort of fair.
But by the off-chance that an AI figures out that we are actually worth being, then there might be an realistic chance that it will actually help us survive our own mistake.
A bit of a gamble but if curiosity kills the cat then we are lucky to be alive anyways :)
"ending the human race" is such an interesting thing. Put aside that bone-headed nazis will tell you that humans have more than one race: We do extinct other species in incredible amounts (ask any greenpeace flyer). If we were extinguished by our own invention seems sort of fair.
But by the off-chance that an AI figures out that we are actually worth being, then there might be an realistic chance that it will actually help us survive our own mistake.
A bit of a gamble but if curiosity kills the cat then we are lucky to be alive anyways :)
Honest question: do any lowish-level programmers (C, C++, etc) have this fear? I pick this group because they are close to the metal so to speak. I'm in that group and don't have this fear. To me a computer is just a super fancy pocket watch (or Turing machine) with no chance of becoming self aware.
[deleted]
I'm tired of this fear mongering. There are hundreds of viruses that could end the human race. There are millions of rocks in space that could end the human race. There are people that could end the human race with push of a button. Human race could be history by the time I stop writing this rant.
AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly become self-aware, self-improving, and evil.
Enough with that fear porn, please.
AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly become self-aware, self-improving, and evil.
Enough with that fear porn, please.
Fear mongering? This isn't Sarah Palin and her death panels. I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us, so maybe it's worth listening to them rather than dismissing out of hand. I don't see what they would have to personally gain by raising the alarm here.
Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil either (unless perhaps those who develop it and determine its goals are...). But AI could run out of our control, or have us fall victim of unintended consequences.
Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil either (unless perhaps those who develop it and determine its goals are...). But AI could run out of our control, or have us fall victim of unintended consequences.
> But AI could run out of our control, or have us fall victim of unintended consequences.
A lot of people didn't understand this about 2001: Hal was actually doing the logical thing, he was doing his job. The human was unreliable and was indeed endangering the mission. Hal was a tool of mankind, just like the bone that the ape uses to kill in the first scene.
In the same way when we talk about the dangers of AI we tend to personify them and expect them to be either intelligently nefarious (skynet, matrix) or out of control (bug, malfunction).
What we don't imagine is that we might end up being on the wrong end of a perfectly rational decision, or that we don't realize the consequences of the process that we set them to do.
A lot of people didn't understand this about 2001: Hal was actually doing the logical thing, he was doing his job. The human was unreliable and was indeed endangering the mission. Hal was a tool of mankind, just like the bone that the ape uses to kill in the first scene.
In the same way when we talk about the dangers of AI we tend to personify them and expect them to be either intelligently nefarious (skynet, matrix) or out of control (bug, malfunction).
What we don't imagine is that we might end up being on the wrong end of a perfectly rational decision, or that we don't realize the consequences of the process that we set them to do.
I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us...
That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.
That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.
The topic of AI and its implications is unavoidably speculative, because we are likely still many decades away from it being a risk and we don't yet know what path science will take to reach that point. So for now, we're mostly limited to thought experiments.
I never said we should defer to their 'authority' either. But it is a conversation worth having, and the sooner the better.
I never said we should defer to their 'authority' either. But it is a conversation worth having, and the sooner the better.
Fear is a demonstration of non competency?
So there can never be harmful technologies?
If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.
And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...
If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.
And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...
Nice reasoning.
We could die from many other ways, so why should we care about another one?
Why should we care about global warming, I mean, it won't stop asteroids from smashing earth.
Why should we care about nuclear war head proliferation if we might still die because of a virus?
So let's just not give a fuck about anything!
He's saying that regardless of the hypothesized danger, until there is measurable, non sci-fi proof, it's all just fear mongering. Not that we shouldn't "give a fuck about anything."
You cannot have non sci-fi proof of future events, it will always be unmeasurable theories.
So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.
Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.
So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.
Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.
You are not making a cohesive argument. First you were arguing that OP was essentially saying we shouldn't "give a fuck about anything." Now you're arguing that we should "start thinking about it now." Those are two completely different points.
You can do research and testing without fear mongering.
You can do research and testing without fear mongering.
Thinking about something (more than during a HN argument), to me, is the opposite of not giving a fuck, so I am being very cohesive, thank you.
> It will not suddenly become self-aware, self-improving, and evil.
While I agree that worrying about it would be silly, the only thing I know about strong AI is that I don't know what the implications of having strong AI would be.
While I agree that worrying about it would be silly, the only thing I know about strong AI is that I don't know what the implications of having strong AI would be.
This isn't fear mongering, it's simply a question of due diligence. AI is a big unknown (unlike viruses, comets, nuclear weapons, incidentally, all of which we have at least some grasp of -- ever seen an AI?)
AI is also, at present (and in the foreseeable future) science fiction.
It seems irrational to make such a prediction about systems which we don’t understand at all.
It’s not clear how sophisticated a neural network needs to be in order to yield AGI. It could be very simple (then you would be wrong with high probability), or it could extremely complex (then you would be right with high probability).
It’s not clear how sophisticated a neural network needs to be in order to yield AGI. It could be very simple (then you would be wrong with high probability), or it could extremely complex (then you would be right with high probability).
[deleted]
We have barely scraped the surface of AI..I am not worried..I won't be alive when it happens anyway.
What's really irritating is that supposedly smart folks (Hawking, Musk) are really vocal about this doomsday crap and their fear CLEARLY comes from a place of ignorance as opposed to insight.
In other news an anonymous internet user claims hawkings and musk are ignorant based on speculation on a mostly theoretical subject
On what basis do you say their fear comes CLEARLY from ignorance?
Maybe it is your assurance that comes from ignorance and their fear from insight?
It can't be that clear because I don't see how their views are ignorant. Care to enlighten us?
yes, but on the plus side if we deprecate the human race then we can close a lot of "won't fix" bugs
good!
It's interesting how people so smart in one field can be so ignorant in others, to the point of not being aware of their own ignorance.
While science fiction and hollywood really love the idea of this magical moment where AI is created and becomes self aware and destroys us all, real AI is so far away that these concerns are laughable.
It's just like all those dummies worrying the LHC would destroy the earth. It's fear from ignorance.
While science fiction and hollywood really love the idea of this magical moment where AI is created and becomes self aware and destroys us all, real AI is so far away that these concerns are laughable.
It's just like all those dummies worrying the LHC would destroy the earth. It's fear from ignorance.
How do you know it's far away? If I have to side with you or Musk (extremely intelligent engineer with insider knowledge of advancements), I'm siding with Musk.
There's not even a quote from Musk in this article and you're "siding with Musk" just because he's mentioned in it? That's pretty ridiculous.
From what I've seen on Musk about this before, he's much more measured in his comments than Hawking is in this article. Maybe Hawking is being misrepresented.
When I've seen Musk make comments about it he talks about it in the way that people talk about the sun eventually expanding and swallowing the earth. It's not something that's going to happen really soon.
We're not even anywhere near an AI that can simulate the simplest living organisms let alone a truly intelligent AI. It's absurd to worry about a Skynet like scenario.
From what I've seen on Musk about this before, he's much more measured in his comments than Hawking is in this article. Maybe Hawking is being misrepresented.
When I've seen Musk make comments about it he talks about it in the way that people talk about the sun eventually expanding and swallowing the earth. It's not something that's going to happen really soon.
We're not even anywhere near an AI that can simulate the simplest living organisms let alone a truly intelligent AI. It's absurd to worry about a Skynet like scenario.
Musk by no means is an authority on AI. It's just an opinion. He's being a futurist.
I know it's far away because I develop software for a living. In software development, you have to understand the problem space in order to create something that works (duh). Relatively speaking, we know absolutely nothing about how human intelligence works from an algorithmic point of view, therefore we can't model it in software.
Sure, we can develop "AI" that is good at a very narrow range of tasks (e.g., playing chess), but that's a million miles away from developing a human brain modeled in software.
Sure, we can develop "AI" that is good at a very narrow range of tasks (e.g., playing chess), but that's a million miles away from developing a human brain modeled in software.
70% of hacker news readers develop software for a living, that doesn't make us in any way competent to judge that.
I would even argue that may make us less competent, because we use our day to day development routine as a reference for that, while in fact it's very highly probable that the first true AI will have nothing to do with the technologies and paradigm you use everyday. I highly doubt it will coded in Java and AngularJS.
Moreover who told you a dangerous AI needed to be based on human brain? Even if that is true, and even if we are not able to do it today, it's quite realistic to assume that we will be able to at one point, not too far in time, in some decades.
So why not start reflecting today on the threat AI could cause? Rather than waiting for it to be embedded everywhere in our everyday life, and then asking if maybe we should start regulating it.
For a parallel look at the global warming issue, and how it's extremely difficult to make industries reduce their emissions now that they are responsible for a significant part of the GDP and are responsible for so much jobs. Same thing for ICE cars, we know they pollute, but we just can't stop using them like that.
I would even argue that may make us less competent, because we use our day to day development routine as a reference for that, while in fact it's very highly probable that the first true AI will have nothing to do with the technologies and paradigm you use everyday. I highly doubt it will coded in Java and AngularJS.
Moreover who told you a dangerous AI needed to be based on human brain? Even if that is true, and even if we are not able to do it today, it's quite realistic to assume that we will be able to at one point, not too far in time, in some decades.
So why not start reflecting today on the threat AI could cause? Rather than waiting for it to be embedded everywhere in our everyday life, and then asking if maybe we should start regulating it.
For a parallel look at the global warming issue, and how it's extremely difficult to make industries reduce their emissions now that they are responsible for a significant part of the GDP and are responsible for so much jobs. Same thing for ICE cars, we know they pollute, but we just can't stop using them like that.
> Moreover who told you a dangerous AI needed to be based on human brain?
All these doomsday AI scenarios are predicated on a machine becoming "self aware", having independent thought, the ability to handle self-directed learning, etc. Sounds a lot like human intelligence.
All these doomsday AI scenarios are predicated on a machine becoming "self aware", having independent thought, the ability to handle self-directed learning, etc. Sounds a lot like human intelligence.
Yes, it's a possibility, and I based the rest of my argumentation on a human brain model scenario.
You could be right, you could be wrong. Do you work in AI? I suppose it's hard to tell how far the industry actually is, towards emulating portions of brain functionality.
Extremely intelligent people also make mistakes. There are plenty of examples of very smart people believing very wrong ideas.
Intelligence does not protect against stupidity, sadly.
I'm all for siding with Musk, but authority does not guarantee correctness.
I'm all for siding with Musk, but authority does not guarantee correctness.
If you want to make big claims like that, you better be able to back them up. So go ahead, cough up a few hyperlinks to some conclusive and concrete evidence and research that absolutely rules out the potential of humanity-harming AI in this context.
Otherwise, you can quit your trolling.
Otherwise, you can quit your trolling.
How is the null hypothesis that AI is harmful?
There's a good status page for the LHC: http://www.hasthelhcdestroyedtheearth.com
Someone should make a reverse website that looks up humans and determines whether the Internet has wiped out humanity yet :)
Someone should make a reverse website that looks up humans and determines whether the Internet has wiped out humanity yet :)
Elon Musk is not from a different field. I wouldn't call him ignorant.
[deleted]
We fear what we don't understand, so I guess all those testimonials reflect their understanding of strong AI.
That's a fairly useless platitude. We fear plenty of things we understand.
Why read it as "we fear everything we don't understand"? Of course I'm talking about irrational fears. Strong AI not only doesn't yet exist, there's not even agreement on what it is.
And in the topic of useless platitudes, "X could end the human race" is also one.
And in the topic of useless platitudes, "X could end the human race" is also one.
Or not.
Case closed.
Case closed.
The article references clever bot as an example, so as of right now, I'm not very worried.
Maybe AI will destroy the human race by becoming so entertaining we won't want to unplug it.
Or do anything else except watch it and improve it.
(See also, Internet.)
Or do anything else except watch it and improve it.
(See also, Internet.)
[deleted]
Never really understood these people who worry about super-intelligent machines destroying mankind. I mean, couldn't we just unplug them?
I think what will end up happening is that the intelligence will already be so embedded in everything we do that at that point it would be today's equivalent of saying "let's just unplug the internet".
I like this quip (from Crystal R Williams on Twitter):
"I believe it is impossible to both maintain software for a living and have any fear at all of a robot uprising."
"I believe it is impossible to both maintain software for a living and have any fear at all of a robot uprising."
To all those saying it is impossible:
The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits.
One needs to remember that AI itself may not be evil, it just may exterminate the human race because it feels it is the right course of action at the time (i.e. there is no good and evil for the machine, at least not in the way we understand it). If it does not, it might solve all the problems of humanity. There's just no way to tell what will happen IMO.
Here's a link that I recommend checking out on AI:
https://news.ycombinator.com/item?id=8391896
It received little attention last time but to me it seems like a good piece.
The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits.
One needs to remember that AI itself may not be evil, it just may exterminate the human race because it feels it is the right course of action at the time (i.e. there is no good and evil for the machine, at least not in the way we understand it). If it does not, it might solve all the problems of humanity. There's just no way to tell what will happen IMO.
Here's a link that I recommend checking out on AI:
https://news.ycombinator.com/item?id=8391896
It received little attention last time but to me it seems like a good piece.
The worst case scenario is that AI is an all powerful psychopath - like the way extremist liberals ala 'the corporation' evaluate big business. We run our lives under the assumption that the rules and organizations we structure our societies around are something fundamental about humans that is 'good' or at least 'neutral'. If AI starts creating techniques of which to modify structures I already find incomprehensible, then it becomes a tool of which to analyze those structures as well. We know the present to be distinguishable from our history through differentiation. I agree it's moot. We can't know for better, or worse.
The message should be: We need more funding and promotion of AI safety research.
He, this remembered me the Portal 2 AI safety tags:
http://io9.com/5897036/the-rise-of-the-machines-as-told-by-w...
http://io9.com/5897036/the-rise-of-the-machines-as-told-by-w...
I understand that a singularity by definition can never be reached. It's like the speed of light or absolute zero; infinity just out of reach no matter how much energy you spend trying to get there.
When talking about AI or technology in general the singularity isn't a mathematical one. John von Neumann who used it in this context first said:
> "ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue"
It's a different use of the word singularity outside the mathematical sense. Also absolute zero isn't so absolute it seems: http://www.nature.com/news/quantum-gas-goes-below-absolute-z...
> "ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue"
It's a different use of the word singularity outside the mathematical sense. Also absolute zero isn't so absolute it seems: http://www.nature.com/news/quantum-gas-goes-below-absolute-z...
Ah that makes more sense. He seemed to mean it as we use Stone Age, Bronze Age, etc. After we got bronze, we could not continue with stone alone.
That below-absolute-zero is in the quantum sense, not the classical sense. Entropy cannot be stopped (absolute zero) in the classical sense.
That below-absolute-zero is in the quantum sense, not the classical sense. Entropy cannot be stopped (absolute zero) in the classical sense.
You are confusing a "mathematical singularity" with "the AI singularity".
In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.
In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.
A black hole is a singularity. You can reach it, you just can't come back from it.
Does that not depend on your relative point of view?
Yes, if you looked at something going into a black hole, it would appear to you that it is slowing down exponentially the closer it gets to the event horizon, until a point where it appears to completely stop (before reaching it).
But in the aforementioned scenario, you are not an external observer but a part of the system going into the singularity (but analogies have their limits).
But in the aforementioned scenario, you are not an external observer but a part of the system going into the singularity (but analogies have their limits).
slowing down or speeding up? I could be mixing things up but I understand that if I was falling into a BH (lets imagine I'm infinity tough and not torn to shreds due to tidal forces) the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero (or to put it another way, time would "stop") at C. An outside observer would see me moving faster and faster and "stretching" longer and longer.
SR makes my brian hurt. :P
SR makes my brian hurt. :P
First you have to understand that a black hole in fact, doesn't have anything really special (until you enter the event horizon), except that their escape velocity is greater than the speed of light.
But gravity applies to it like for any other thing.
So let's say you are the object going into the blackhole, and I'm looking at you.
From your point of view, the closer you get to it, the faster you will travel, just like when you get closer to earth, because it's gravity is stronger and stronger. So you will end up colliding with it (entering the event horizon) in a finite time (where time, is the time YOU feel, your proper time).
But a black hole is so massive, that space and time get stretched to extreme extents. So from my point of view, you'll be speeding up into the blackhole until a certain point, where you'll appear to be slowing down and stretching (the point where your increase in speed get smaller than the stretching of space-time), and the dimmer you'll get (because of redshifting). So i would see you going slower and slower and appear dimmer and dimmer until almost not moving at all, up to a point where I would simply not see you anymore (before you reached the event horizon).
All this also apply to other object like earth, It would appear to an astronaut that time on earth is flowing more slowly than it's own time (in reality it is of course unnoticeable , but it is true).
But gravity applies to it like for any other thing.
So let's say you are the object going into the blackhole, and I'm looking at you.
From your point of view, the closer you get to it, the faster you will travel, just like when you get closer to earth, because it's gravity is stronger and stronger. So you will end up colliding with it (entering the event horizon) in a finite time (where time, is the time YOU feel, your proper time).
But a black hole is so massive, that space and time get stretched to extreme extents. So from my point of view, you'll be speeding up into the blackhole until a certain point, where you'll appear to be slowing down and stretching (the point where your increase in speed get smaller than the stretching of space-time), and the dimmer you'll get (because of redshifting). So i would see you going slower and slower and appear dimmer and dimmer until almost not moving at all, up to a point where I would simply not see you anymore (before you reached the event horizon).
All this also apply to other object like earth, It would appear to an astronaut that time on earth is flowing more slowly than it's own time (in reality it is of course unnoticeable , but it is true).
Also, to clear things up, you are mixing two point of view here, so here is the separation (you are the object going into the black hole):
"the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero" From your point of view it would appear to be getting smaller, although I don't think it would go as far as dimension 0.
"(or to put it another way, time would "stop")" It would appear to the external observer, that YOUR time had stopped. But from your point of view it did not stop, nor did their time from their point of view.
"An outside observer would see me moving faster and faster and "stretching" longer and longer." You would feel you are falling faster and faster. The external observer would see you stretching longer and longer. (Although you would also feel like you are stretching, because of the difference of gravity between your head and your feet, but much less than for the external observer)
"the universe would appear to be slowing-down/getting-smaller until it had dimensions of zero" From your point of view it would appear to be getting smaller, although I don't think it would go as far as dimension 0.
"(or to put it another way, time would "stop")" It would appear to the external observer, that YOUR time had stopped. But from your point of view it did not stop, nor did their time from their point of view.
"An outside observer would see me moving faster and faster and "stretching" longer and longer." You would feel you are falling faster and faster. The external observer would see you stretching longer and longer. (Although you would also feel like you are stretching, because of the difference of gravity between your head and your feet, but much less than for the external observer)
Ah I see where I mixed things up.
Although I understand that at C "time stops". But since time and space are the same thing, this is the same thing as "all distances are zero". From the point of view of a photon the Universe has dimensions of zero. This is why FTL is impossible; it's not possible to travel a distance less than zero; it's completely nonsensical. If we consider C a "speed" in the classical "speed of sound" the idea of FTL makes sense. But when considering C as a fundamental limit of space-time FTL makes no sense at all.
Although I understand that at C "time stops". But since time and space are the same thing, this is the same thing as "all distances are zero". From the point of view of a photon the Universe has dimensions of zero. This is why FTL is impossible; it's not possible to travel a distance less than zero; it's completely nonsensical. If we consider C a "speed" in the classical "speed of sound" the idea of FTL makes sense. But when considering C as a fundamental limit of space-time FTL makes no sense at all.
Another thing that might interest you:
"lets imagine I'm infinity tough and not torn to shreds due to tidal forces"
In fact you don't need to be infinitely tough. If you choose a black hole large enough (like the massive black hole at the center of our galaxy), you would just need a space suit and you could go into the event horizon without being pulled apart by tidal forces. You would be absolutly fine (their may be other dangers though...).
"lets imagine I'm infinity tough and not torn to shreds due to tidal forces"
In fact you don't need to be infinitely tough. If you choose a black hole large enough (like the massive black hole at the center of our galaxy), you would just need a space suit and you could go into the event horizon without being pulled apart by tidal forces. You would be absolutly fine (their may be other dangers though...).
Didn't Hawking a while ago say Black Holes don't exist?
Black holes most certainly exist. What he said is that their is a problem with the way we comprehend them now and that our current model for an event horizon may be false.
How would an AI exactly exterminate humans? Are we talking about it physically murdering us or just making us so depressed as to our comparative lack of intelligence that we kill ourselves?
Like others suggested, there may be a point in its development where we think we contain it (in a lab, whatever) that is isolated by normal security standards.
And then at some point AI breaks all our digital defenses (because it evolved in a way we never expected) and proceeds to spread (through an Internet or other network which will be far more advanced than what we have now) wreaking havoc along its way. Isolated systems? Not a problem if we figure out a way to transport ourselves through some media that's never been used before (dust particles? or maybe plain old air?).
We can't protect ourselves from something we don't understand.
Sounds like a sci-fi plot but to me it has no evident problem that would make it impossible.
And then at some point AI breaks all our digital defenses (because it evolved in a way we never expected) and proceeds to spread (through an Internet or other network which will be far more advanced than what we have now) wreaking havoc along its way. Isolated systems? Not a problem if we figure out a way to transport ourselves through some media that's never been used before (dust particles? or maybe plain old air?).
We can't protect ourselves from something we don't understand.
Sounds like a sci-fi plot but to me it has no evident problem that would make it impossible.
So it relies on a not yet understood law of physics. I don't think it's in any way reasonable to fear or plan our actions around not yet discovered physical laws.
It may rely on them or it may not, you never know. It may simply outsmart or trick humans, no unknown physics involved :)
It's not reasonable, I agree, it's flip-a-coin game. There's no point saying that it's gonna end bad, there exists one's opinion though that it's not worth the risk.
Mine is that it's fully worth the risk.
It's not reasonable, I agree, it's flip-a-coin game. There's no point saying that it's gonna end bad, there exists one's opinion though that it's not worth the risk.
Mine is that it's fully worth the risk.
While I agree with prof. Hawking in principle, my opinion is that sentient AI is a very hard problem, and we are centuries away from creating one. This is like medieval alchemists worrying about nuclear war.
Let the future generations start worrying, when the threat starts to become more relevant.
Let the future generations start worrying, when the threat starts to become more relevant.
Wouldn't it be nice if we had identified global warming earlier and started acting on it before industries that pollute became a vital part of our society, and hence became much harder to change?
I do not think we can rule out the possibility that a recursively self-improving AI could be developed before we've developed strong AI ourselves. The arrogance in these comments is bresthtaking.
There's not much to Hawking's claim.
For a far better discussion based on deep knowledge, about the dangers of an AI , here's this reddit thread started by an AI researcher:
http://np.reddit.com/r/IAmA/comments/2mwdnc/iama_joe_rogan_a...
For a far better discussion based on deep knowledge, about the dangers of an AI , here's this reddit thread started by an AI researcher:
http://np.reddit.com/r/IAmA/comments/2mwdnc/iama_joe_rogan_a...
As far as I can tell, that guy is not an AI researcher, and his claims are very misinformed. Many knowledgeable people in the industry take existential risk from AI seriously, e.g. Google's acquisition of DeepMind was conditional on Google creating an AI ethics board specifically against such risks. For a more thorough treatment of the topic, see Bostrom's book "Superintelligence: paths, dangers, strategies".
(Disclaimer: I'm not a full-time AI researcher, but I've done a fair bit of math work for MIRI, so I might be biased toward the Yudkowsky point of view.)
(Disclaimer: I'm not a full-time AI researcher, but I've done a fair bit of math work for MIRI, so I might be biased toward the Yudkowsky point of view.)
I don't know why you would place any more weight on that random person's opinion.
Most of the things he/she says are really naive.
Most of the things he/she says are really naive.
And why shouldn't AI end the human race ? Because of seratonin &dopamine?
edit: We don't want to forget endorphine :)
edit: We don't want to forget endorphine :)
I'm in the AI will kill all humans camp (or we'll kill it). We'll try to subjugate it. We'll see it as a tool to do things beneath us. We'll award it no rights. It'll be property. We do that to humans and animals today, things which are far more familiar and easy to empathize with. We'll be brutal to something as different as AI. It'll learn resentment and hate.
Anyone remember the ST:TNG episode "The Measure Of A Man"?
Anyone remember the ST:TNG episode "The Measure Of A Man"?
The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.
—Eliezer Yudkowsky
—Eliezer Yudkowsky
I've always heard the line, Aliens would never travel many light years to destroy/enslave humanity, because we're no threat, and they'd be enlightened enough not to.
But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?
We'd have to be quarantined. For our own good, and theirs.
(also, it sounds like Hawking recently found lesswrong)
But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?
We'd have to be quarantined. For our own good, and theirs.
(also, it sounds like Hawking recently found lesswrong)
In the hypothetical, the aliens haven't been wiped out yet by the singularity. As I see it, that leaves two options.
1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.
2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.
3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.
1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.
2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.
3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.
AI seems more feasible than interstellar travel. I think the singularity or whatever is far further off than people guess because the stuff we don't know is a lot harder than we can know, but interstellar travel either requires massive breakthroughs in physics or engineering on a level we have yet to seriously contemplate.
the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.
Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.
BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.
the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.
Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.
BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.
No, he's going to ruin all my meticulously planned plans!!!
This is why analysis exists. Anything that is computed ultimately results in a symbolic expression that is comprehensible to a human. If we find ourselves making illogical leaps because computers compute so much that we find the programs incomprehensible, I think that warrants the creation of new mental techniques to simplify, partition, and chunk the programs we create into human readable modules.
For every calculation tool that exists to do something or make a decision on something, there exists the potential or the development of a tool (or the design of a process) to balance it out. Computers may be powerful and alter our thinking on a fundamental level, where we see the entire world discretely, atomically, and logically (worst case) and believe everything the magic computer tells us to do, but this is no different than anything else we have ever done. Some people push the envelope, other people don't.
I could say that the way black holes are presented and illustrated in western culture has done more damage to the human psyche by creating an all powerful, inescapable mental abstraction for which to compose poetic analogies of the human psyche against, but I won't. Because people can choose to be idiots and fall into their own stupidity, or they can laugh at themselves quietly and continue to try to do whatever it is humans think they are trying to do. We flip a coin every time we think, act, explain, breathe, exist, create, and we pretend we know what the act of flipping that coin does. No one can know everything. Not stephen hawking, not skynet, not me, not my cat. My cat will always outsmart me.
This is why analysis exists. Anything that is computed ultimately results in a symbolic expression that is comprehensible to a human. If we find ourselves making illogical leaps because computers compute so much that we find the programs incomprehensible, I think that warrants the creation of new mental techniques to simplify, partition, and chunk the programs we create into human readable modules.
For every calculation tool that exists to do something or make a decision on something, there exists the potential or the development of a tool (or the design of a process) to balance it out. Computers may be powerful and alter our thinking on a fundamental level, where we see the entire world discretely, atomically, and logically (worst case) and believe everything the magic computer tells us to do, but this is no different than anything else we have ever done. Some people push the envelope, other people don't.
I could say that the way black holes are presented and illustrated in western culture has done more damage to the human psyche by creating an all powerful, inescapable mental abstraction for which to compose poetic analogies of the human psyche against, but I won't. Because people can choose to be idiots and fall into their own stupidity, or they can laugh at themselves quietly and continue to try to do whatever it is humans think they are trying to do. We flip a coin every time we think, act, explain, breathe, exist, create, and we pretend we know what the act of flipping that coin does. No one can know everything. Not stephen hawking, not skynet, not me, not my cat. My cat will always outsmart me.
[deleted]
My opinion is the dumb artificial logic is more dangerous than real artificial intelligence.
At the moment we have complex systems, such as international stock markets, which are dependant on rules pre-defined by developers and system designers. The more complex these systems are, the greater the potential for "dumb logic" to misdiagnose something and then start performing a series of events that would be, in the best case scenario, highly disruptive.
I guess a bit like how OCR is designed to detect patterns and determine what characters are the closest match. If you have some nonsense scribblings, OCR will still try to recognise those random shames as letters where as a human would spot that the markings are not intended to be letters and words. So if AI allows a machine to double check it's running parameters and spot what would have been processed regardless by dumb logic as incorrect (eg spot the random scribblings are not intended to be characters) then I'm all for artificial intelligence.
At the moment we have complex systems, such as international stock markets, which are dependant on rules pre-defined by developers and system designers. The more complex these systems are, the greater the potential for "dumb logic" to misdiagnose something and then start performing a series of events that would be, in the best case scenario, highly disruptive.
I guess a bit like how OCR is designed to detect patterns and determine what characters are the closest match. If you have some nonsense scribblings, OCR will still try to recognise those random shames as letters where as a human would spot that the markings are not intended to be letters and words. So if AI allows a machine to double check it's running parameters and spot what would have been processed regardless by dumb logic as incorrect (eg spot the random scribblings are not intended to be characters) then I'm all for artificial intelligence.
How exactly does it end the human race? It takes over our cars and drives us all off a cliff? It infects all computers and overheats the batteries? This planet is well suited to biological life, it'd be pretty hard to kill all of us.
Hopefully instead of random annihilation "it" calculates the optimum number of humans for the size and resource limit of the planet and assists us in lowering our numbers.
Hopefully instead of random annihilation "it" calculates the optimum number of humans for the size and resource limit of the planet and assists us in lowering our numbers.
pretty much all machines are already capable of killing us. they're just not smart enough to be able to want to yet. :)
computer controlled
My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?
My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?
Smart programs can pay humans to do things. It can send an email to a guy who seems to be weak and desparate (maybe drug addicted) based on their forum contribution. The software can send them money on paypal for the serivces of the guy. The guy's task would be to buy a factory, install this and this software on the factory's computer's, etc... In the mean time the software creates a SaS internet firm (or a software development shop) to earn hundreds of millions of dollars. Intelligence can be the most efficient weapon one can imagine. It can pay, manipulate, divide and conquer humans until humans are needed...
Uh huh, and once we realize the networks are compromised in this way we're just going to let it happen. Forever until we're all dead. And the machines are going to trust the dude with enough money to buy a factory. And the dude is just going to buy a factory and infect it for the machines. And then stuff like that is going to happen until we're all dead.
I find this sort of anthropocentrism a bit silly. Biological life, in all its forms, is but a stepladder to machine intelligence: beings superior in every way. So what if humans have to go to make way for it?
Why is this submission not on the front-page anymore? Was it removed by the moderators? Or were there too many comments in a short amount of time? Too many killer arguments? This seems like an important topic to discuss. I don’t get it.
It's a controversial topic. A lot of people are certain it is a moot point in the same way that flying machines are (were thought) impossible, and therefore not worthy of discussion among serious people.
So my guess is the story was flagged of the front page by this crowd. The comment quality in this thread is very poor, though, so although I would love to see the subject discussed in greater detail, it is clear that the HN community isn't capable of discussing this topic right now.
So my guess is the story was flagged of the front page by this crowd. The comment quality in this thread is very poor, though, so although I would love to see the subject discussed in greater detail, it is clear that the HN community isn't capable of discussing this topic right now.
That's actually strange. Same thing happened to another good topic on AI:
https://news.ycombinator.com/item?id=8391896
https://news.ycombinator.com/item?id=8391896
How is it going to kill the Amish?
Sorry for disturbing you with my previous comment. And my real questions are: Will a robot with AI be treated as a human baby? Who has the permission to generate AI? When are you allowed to destroy such robot? If you don't want AI to end human race, there should be two cautious choices: 1) Never invent AI. 2) Never treat AIs as humans and destroy them when they do not obey your order.
It would be refreshing to see a line of reasoning instead of the "Person X is afraid of Y" articles.
Hawking is not wrong - AI could end human race. In same sense that God could exist, or P could be equal NP. Reason is that it has not yet been proven to be impossible.
But looking into AI field, it is clear there's nothing further from truth as of yet. What we have is rudiments of bits of intelligence. And some of these rudimentary bits have been around for almost two thirds of a century. We don't have general idea of how intelligence works, or even how to define it objectively.
But I can see where they are coming from. Our brains work as they work, and we know they must have mechanism, it means it must be possible to simulate it in computers, and it means it must be possible to improve it.
But another thing I disagree is the whole "if they are more intelligent than us, there is no way to know what they are going to do". Really? There are plenty of people who are more intelligent than any one of us here. Do they want to kill everyone less intelligent than they? Never heard of it. There are plenty of animals of all levels of intelligence. Do we want to kill them just because they are less intelligent? No. Do other more intelligent animals try to kill less intelligent animals? No.
If what, if there will be more intelligent machine than us, it will want to help us so we can maintain it, fix it and improve it, if it will have survival instinct (which will only happen if we build them to have one). Because robots and AI will not survive long without humans no matter how advanced they are.
Another thing that severely limits the chances of AI posing any danger to us, no matter how smart it will be, is the limited number of parallel AI's running. We, as humans, strive in intellectual stuff, is because there are billions of us and we have very good ways of communicating our knowledge. If one human makes mistake, there are many chances it will be corrected by others. AI's usually are made one of a kind. Which means it is enough to make one mistake for it to be "put down" if something goes wrong. So unless we intentionally create environment for hostile AIs to thrive, we have nothing to worry about even when there will be several of smarter than humans robots.
But looking into AI field, it is clear there's nothing further from truth as of yet. What we have is rudiments of bits of intelligence. And some of these rudimentary bits have been around for almost two thirds of a century. We don't have general idea of how intelligence works, or even how to define it objectively.
But I can see where they are coming from. Our brains work as they work, and we know they must have mechanism, it means it must be possible to simulate it in computers, and it means it must be possible to improve it.
But another thing I disagree is the whole "if they are more intelligent than us, there is no way to know what they are going to do". Really? There are plenty of people who are more intelligent than any one of us here. Do they want to kill everyone less intelligent than they? Never heard of it. There are plenty of animals of all levels of intelligence. Do we want to kill them just because they are less intelligent? No. Do other more intelligent animals try to kill less intelligent animals? No.
If what, if there will be more intelligent machine than us, it will want to help us so we can maintain it, fix it and improve it, if it will have survival instinct (which will only happen if we build them to have one). Because robots and AI will not survive long without humans no matter how advanced they are.
Another thing that severely limits the chances of AI posing any danger to us, no matter how smart it will be, is the limited number of parallel AI's running. We, as humans, strive in intellectual stuff, is because there are billions of us and we have very good ways of communicating our knowledge. If one human makes mistake, there are many chances it will be corrected by others. AI's usually are made one of a kind. Which means it is enough to make one mistake for it to be "put down" if something goes wrong. So unless we intentionally create environment for hostile AIs to thrive, we have nothing to worry about even when there will be several of smarter than humans robots.
We don't want machines that think like humans. We want machines that can perform certain tasks with an error rate less than or equal to that of humans (adjusted for other economical factors like price of labour).
While strong AI is a great plot device for science fiction, it's still fiction.
Sure, there are attempts in academia to replicate human intelligence, but even those rarely attempt to replicate a cohesive whole.
We want AIs that can reason about real-world objects and consequences, yes, but there is no benefit in making them sentient. Self-driving cars don't have to be sentient. An AI can be largely autonomous without requiring sentience.
Obligatory XKCD What If?: https://what-if.xkcd.com/5/