Ask YC: AI
The field of artificial intelligence has been always something I wanted to take a peek on, but never really had the chance. Could people here recommend me some good books to start my journey on this field?
11 comments
Here's the standard AI textbook: http://aima.cs.berkeley.edu/
I'm assuming you mean "AI" in the sense of "fancy mathematical tricks to solve domain specific problems", not in the sense of artificial general intelligence.
I'm assuming you mean "AI" in the sense of "fancy mathematical tricks to solve domain specific problems", not in the sense of artificial general intelligence.
Jey I agree, I highly recommend Norvig and Russell's book. I had a one semester course back in college that used this. I actually just recently have been programming some of the exercises up in Ruby just to stay sharp with some different kind of coding than the standard webapp stuff. It has some great problems, and has source code available in many languages (not ruby unfortunately, but I guess makes it more of a challenge).
Norvig also has some great essays and example AI code on his site, http://norvig.com/
Norvig also has some great essays and example AI code on his site, http://norvig.com/
I agree it's a great book but IMO it's important to understand that modern AI is about:
1. Classification and Learning (aka is that apple ripe? or what's the best website for a given search?)
2. Modeling / Machine vision (Where are the rocks around this rover.)
3. Goal Seeking (AKA what's the best path from here to DC.)
With enough resources we can do any of the above fairly well. So using AI is more about understanding how to link the above activities to some useful problem. AKA control a rover when your ping is 15 min or solve a CAPTCHA.
1. Classification and Learning (aka is that apple ripe? or what's the best website for a given search?)
2. Modeling / Machine vision (Where are the rocks around this rover.)
3. Goal Seeking (AKA what's the best path from here to DC.)
With enough resources we can do any of the above fairly well. So using AI is more about understanding how to link the above activities to some useful problem. AKA control a rover when your ping is 15 min or solve a CAPTCHA.
I actually don't find this sort of research all that interesting.
I like out of the box solutions more:
http://video.google.ca/videoplay?docid=-8246463980976635143
I like out of the box solutions more:
http://video.google.ca/videoplay?docid=-8246463980976635143
Second that thanks.
That was expletively amazing. I didn't expect to watch more than the intro but couldn't stop. That's an intersection of CS and HCI cleverness raised to the level of art.
That's brilliant, amichail. Thanks.
Fantastic video, thank you!
Start with this book: http://aima.cs.berkeley.edu/ "Artificial Intelligence: A Modern Approach" by Russell/Norvig
It's one of the best books on AI and it's a "standard" AI textbook for most of the intro AI courses.
It's also great as a reference (I have it on my bookshelf beside me) and you can quickly look up algos and implement them in your fav language.
It's one of the best books on AI and it's a "standard" AI textbook for most of the intro AI courses.
It's also great as a reference (I have it on my bookshelf beside me) and you can quickly look up algos and implement them in your fav language.
Also see Mitchell's machine learning: http://www.cs.cmu.edu/~tom/mlbook.html
Norvig, Mitchell, Thrun have good books
Also, google "Andrew Moore CMU" his homepage is full of useful tutorials
That said, "AI" is too vague. What do you want to be able to do?
Also, google "Andrew Moore CMU" his homepage is full of useful tutorials
That said, "AI" is too vague. What do you want to be able to do?
Maybe "Programming Collective Intelligence" would also be a good start.
Yeah, that's definitely a nice, practical book for machine learning stuff (but not really AI proper). For a practical book on AI itself, Norvig's Paradigms of Artificial Intelligence Programming:
http://norvig.com/paip.html
is a great book, and a nice complement to Norvig and Russell, which is somewhat more theory-oriented.
http://norvig.com/paip.html
is a great book, and a nice complement to Norvig and Russell, which is somewhat more theory-oriented.
"... Yeah, that's definitely a nice, practical book for machine learning stuff (but not really AI proper). For a practical book on AI itself, Norvig's Paradigms of Artificial Intelligence Programming ..."
It is ironic that the man who wrote the book uses different methods (maths) to extract meaning from data at google ~ http://news.ycombinator.com/item?id=90298
Something smells about AI.
It has been well funded, attracted the best minds (Minsky, McCarthy, Newell) at places like MIT, Stanford and resulted in lots of useful languages & tools. [0] From Cyc to Powerset companies have tried to use these techniques trying to assign human like intelligence to machines yet have failed.
Why?
Bruce Stirling reckons it has to do a lot with the language. [1] Assigning intelligence to what is essentially a 21'st century equivalent of knitting mills or steam engines of yesteryear. Maybe thats why Google has taken the approach it has. Instead of creating intelligent software to find the meaning, take an insight into users who have pre-assigned intelligence to documents. And use well known sorting, linking techniques and software engineering along with not-so-well applied math techniques. Thus make google appear to make sense of what we search for.
[0] http://en.wikipedia.org/wiki/Artificial_intelligence
[1] ITConversations, 'Bruce Stirling, The Internet of Things, 58m, 26.4 mb, 2006MAR06' ~ http://www.itconversations.com/shows/detail717.html
It is ironic that the man who wrote the book uses different methods (maths) to extract meaning from data at google ~ http://news.ycombinator.com/item?id=90298
Something smells about AI.
It has been well funded, attracted the best minds (Minsky, McCarthy, Newell) at places like MIT, Stanford and resulted in lots of useful languages & tools. [0] From Cyc to Powerset companies have tried to use these techniques trying to assign human like intelligence to machines yet have failed.
Why?
Bruce Stirling reckons it has to do a lot with the language. [1] Assigning intelligence to what is essentially a 21'st century equivalent of knitting mills or steam engines of yesteryear. Maybe thats why Google has taken the approach it has. Instead of creating intelligent software to find the meaning, take an insight into users who have pre-assigned intelligence to documents. And use well known sorting, linking techniques and software engineering along with not-so-well applied math techniques. Thus make google appear to make sense of what we search for.
[0] http://en.wikipedia.org/wiki/Artificial_intelligence
[1] ITConversations, 'Bruce Stirling, The Internet of Things, 58m, 26.4 mb, 2006MAR06' ~ http://www.itconversations.com/shows/detail717.html
I think that some progress in sensory systems would make AGI easier. Obviously, humans spend most of their time structuring the complex, physical world around them, not a stream of text like most AI approaches dictates.
Our genetic code isn't that huge, but the amount of information contained in our brains is. All that information must be coming for somewhere. I refuse to believe that evolution has managed to create _that_ awesome compression algorithms.
Maybe this is what Google does, in a way. They happen to have a huge corpus of machine-readable data, which is in a sense equivalent to what a sophisticated sensory apparatus would give you. I intended to disagree with you when I started writing this comment, but now I'm not so sure. Human intelligence obviously doesn't spend its time reading petabytes of information on other people's search habits, but maybe Google is actually onto something.
Our genetic code isn't that huge, but the amount of information contained in our brains is. All that information must be coming for somewhere. I refuse to believe that evolution has managed to create _that_ awesome compression algorithms.
Maybe this is what Google does, in a way. They happen to have a huge corpus of machine-readable data, which is in a sense equivalent to what a sophisticated sensory apparatus would give you. I intended to disagree with you when I started writing this comment, but now I'm not so sure. Human intelligence obviously doesn't spend its time reading petabytes of information on other people's search habits, but maybe Google is actually onto something.
I think that what Google is doing is along the same lines of what human brains do. Google is crunching a load of mainly textual data to create an accurate statistical model of context. Humans use a much smaller amount of text but use visual, audio, and touch sensory input to create context. Humans also have the ability to refine their mental model by doing experiments on the world, whereas Google, as an observer, has to just use what people create. This means that Google (or any AI) needs a lot, lot, lot more of data to train its accuracy, which is now possible due to the internet.
Why would Google not be be able to experiment on the world?
What is "AI proper", though? I am fairly certain that when the true AI will be built, it won't be an expert system or based on first order logic.
The classical AI stuff is still interesting, of course, and still has practical applications (ie A* search for games). But I tend to count "the other stuff" (ie Data Mining) as AI, too.
The classical AI stuff is still interesting, of course, and still has practical applications (ie A* search for games). But I tend to count "the other stuff" (ie Data Mining) as AI, too.
I'd agree with machine learning falls within the purview of AI -- I'm just saying if all you want to learn is machine learning, then a book on just that subject would probably be more relevant than AIMA. I just got "Pattern Recognition and Machine Learning" by Bishop, and it's a great book.
The Bishop book looks very good, judging from the table of contents.
Perhaps similar, and available as a free ebook: McKay, Information Theory, Inference Theory and Learning Algorithms, http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html
Perhaps similar, and available as a free ebook: McKay, Information Theory, Inference Theory and Learning Algorithms, http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html
In addition to this book, you're probably going to want a good book covering the history and development of the field.
Keep in mind that AI is a broad field. As such you might find it more helpful to identify which computer intelligence problem(s) you want to solve, then start accessing the AI writings that discuss how to solve your problem. This won't give you a general overview, but it at least would reduce what you'd need to learn about to produce something useful.
Keep in mind that AI is a broad field. As such you might find it more helpful to identify which computer intelligence problem(s) you want to solve, then start accessing the AI writings that discuss how to solve your problem. This won't give you a general overview, but it at least would reduce what you'd need to learn about to produce something useful.
I mean AI as Learning Algorythms and Decision Making programs.
There currently are no good general solutions, so the best tool(s) to use strongly depends on the specific problem you're trying to solve, resources you have available, and how good the answer needs to be.
Reading through the Russell & Norvig book would give you a good overview of the sorts of techniques available.
Reading through the Russell & Norvig book would give you a good overview of the sorts of techniques available.
Unfortunately, too often, that perspective is lost. I know of no general solution to what defines intelligence. So I'm very dubious as to whether an encompassing artificial version is possible.
I see lots of specific solutions to specific problems. That, to me, is the best approximation of intelligence. To the extent that many specific solutions can be approximated and optimized, we'll be very well off. It's easy to denigrate a calculator and spell-checker, but I seem much more intelligent than I would be without them.
I see lots of specific solutions to specific problems. That, to me, is the best approximation of intelligence. To the extent that many specific solutions can be approximated and optimized, we'll be very well off. It's easy to denigrate a calculator and spell-checker, but I seem much more intelligent than I would be without them.
"I know of no general solution to what defines intelligence. So I'm very dubious as to whether an encompassing artificial version is possible."
Logical Fallacy Alert! http://en.wikipedia.org/wiki/Argument_from_ignorance
I don't know how to build a rocket ship nor to do a heart transplant, but I'm fairly confident that these are both feasible.
Logical Fallacy Alert! http://en.wikipedia.org/wiki/Argument_from_ignorance
I don't know how to build a rocket ship nor to do a heart transplant, but I'm fairly confident that these are both feasible.
It's not like people haven't been trying. Indeed, the great aspiration to understand humans is the effort to define our understanding (how's that for a circular argument?).
Moreover, you prove your own point - the one I was agreeing with initially out of everything said in this thread. A rocket ship or heart transplant are specific solutions to specific problems. The analogy to intelligence would be one universal solution to flight or one universal solution to getting oxygen and glucose into cells. It's not that a solution isn't possible. It's just that the hope for THE solution is misguided (see also THE cure for cancer).
I really like the flight analogy because of the length of time people put into finding THE solution. But what we end up with instead are rockets, and prop planes, and jet engines, and airfoils, and gliders, and helicopters, and hoverboards...
Moreover, you prove your own point - the one I was agreeing with initially out of everything said in this thread. A rocket ship or heart transplant are specific solutions to specific problems. The analogy to intelligence would be one universal solution to flight or one universal solution to getting oxygen and glucose into cells. It's not that a solution isn't possible. It's just that the hope for THE solution is misguided (see also THE cure for cancer).
I really like the flight analogy because of the length of time people put into finding THE solution. But what we end up with instead are rockets, and prop planes, and jet engines, and airfoils, and gliders, and helicopters, and hoverboards...
You might want to look at 'cognitive science' as well because that field tries to explain how humans think and make decisions for use as AI construction models.
In that case get "Programming Collective Intelligence" like Tichy said, probably the best place to start.
my TA secretly recommended Norvig's book over Winston's, but we used Winston's "Artificial Intelligence"
For more philosophy, read Minsky's books / papers, and the readings outlined here: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Minsky's paper "Steps toward Artificial Intelligence" reads like a summary of 6.034 at MIT.
Speaking of which: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
For more philosophy, read Minsky's books / papers, and the readings outlined here: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Minsky's paper "Steps toward Artificial Intelligence" reads like a summary of 6.034 at MIT.
Speaking of which: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Having looked through the introductory texts, I am left wanting to better understand current state of various parts of field.
Any suggestions on where to get more of that higher-level commentary?
Any suggestions on where to get more of that higher-level commentary?
I recommend "Beyond AI". It gives a high level history, present, and future of AI. It looks through the lens of a search for artificial general intelligence.
http://www.amazon.com/o/asin/1591025117
http://www.amazon.com/o/asin/1591025117
I'd recommend starting with a definition of intelligence. Once you clearly and unambiguously define it, consider you have 51% of the task done already.
[deleted]
I recommend starting from scratch and ignoring the field. All the field will teach you is very complex ways to NOT have AI.
The problem isn't that the field called "AI" is useless, it's just that they accidentally got away with being called "AI". Without the vast body of research done under the banner of "AI" you wouldn't have OCR, speech recognition, fraud detection, search engines, ...
That said, I also don't think you need to be an expert neural network wrangler and genetic algorithms shaman if you want to study "true AI". You should however have an understanding of the fundamentals behind the techniques used in classical AI so that you can rationally dismiss them, or hell, maybe you'll even find some amazing lead or inspiration in the existing work. After all, the field of AI did start out with ambitions of creating "true AI" (whatever that is).
That said, I also don't think you need to be an expert neural network wrangler and genetic algorithms shaman if you want to study "true AI". You should however have an understanding of the fundamentals behind the techniques used in classical AI so that you can rationally dismiss them, or hell, maybe you'll even find some amazing lead or inspiration in the existing work. After all, the field of AI did start out with ambitions of creating "true AI" (whatever that is).
When someone says "AI", I think they mean AI, not e.g. path-finding for writing a Quake bot.
Of course a real AI would be much better at all those things you mentioned anyway -- like deciding which of my e-mail messages are spam.
Or carrying on a conversation.
P.S. I'm an AI.
Of course a real AI would be much better at all those things you mentioned anyway -- like deciding which of my e-mail messages are spam.
Or carrying on a conversation.
P.S. I'm an AI.
Not in academia. In academia, as far as I can tell, they pretty much exclusively mean narrow AI when talking about "AI". Us lay people might be thinking of HAL and Skynet when we say "AI" though. ;-)
Yes, I fully agree that a "real AI" would be able to do those things better, but my point is that we do have real practical utility from the narrow AI tradition, and their work isn't for nothing.
Yes, I fully agree that a "real AI" would be able to do those things better, but my point is that we do have real practical utility from the narrow AI tradition, and their work isn't for nothing.
"When someone says "AI", I think they mean AI, not e.g. path-finding for writing a Quake bot."
Then you're hopelessly out of touch with what most people mean when they say "AI", and what the person who asked the question wants to learn about ("Learning Algorythms and Decision Making programs").
Then you're hopelessly out of touch with what most people mean when they say "AI", and what the person who asked the question wants to learn about ("Learning Algorythms and Decision Making programs").
> Then you're hopelessly out of touch with what most people mean when they say "AI"
Hopeless? Doc, are you telling me there's no cure?!?
In fact, when most people talk about AI, they mean it in the sense of, "do you think computers will one day be as intelligent as humans?". Cf. the moviefilm "AI". If that kind of AI is what you wish to pursue, then the so-called field of AI isn't going to get you there.
I'm reclaiming the term AI for actual AI from the minority that's misusing it and we're having a Pride march on the 23rd. Wheel out your servers and "represent".
Hopeless? Doc, are you telling me there's no cure?!?
In fact, when most people talk about AI, they mean it in the sense of, "do you think computers will one day be as intelligent as humans?". Cf. the moviefilm "AI". If that kind of AI is what you wish to pursue, then the so-called field of AI isn't going to get you there.
I'm reclaiming the term AI for actual AI from the minority that's misusing it and we're having a Pride march on the 23rd. Wheel out your servers and "represent".
Well, that "minority" is essentially the same community that invented the term in the first place. It's not like what you're observing is some ghastly secret: the AI community hasn't been seriously trying to construct strong AI for 25+ years. "AI" is a bad term, and doesn't accurately describe much of the work that is done in the field -- fair enough. Considering that trying to build strong AI is a pipedream anyway (IMHO), I don't think that's such a disaster. The person asking the question was plainly talking about weak AI techniques anyway.
> the AI community hasn't been seriously trying to construct strong AI for 25+ years.
Now that's an excuse if I ever heard one. "Uhh... we weren't seriously trying... uhh..."
That's also why I'm not dating Natalie Portman!
> Considering that trying to build strong AI is a pipedream anyway (IMHO)
That's hilarious. Heavier-than-air flying machines, a man on the Moon, sending pictures through invisible AIRWAVES?!? Pipe dreams!
Now that's an excuse if I ever heard one. "Uhh... we weren't seriously trying... uhh..."
That's also why I'm not dating Natalie Portman!
> Considering that trying to build strong AI is a pipedream anyway (IMHO)
That's hilarious. Heavier-than-air flying machines, a man on the Moon, sending pictures through invisible AIRWAVES?!? Pipe dreams!
"I'm reclaiming the term AI for actual AI ..."
Those people have mostly switched to using the term "Artificial General Intelligence" when describing "actual AI".
Those people have mostly switched to using the term "Artificial General Intelligence" when describing "actual AI".
"Artificial General Intelligence"
Never heard that term before.
And to quote Office Space: "Why should I change my name, he is the one who sucks". So personally I won't adapt "Artificial General Intelligence" into my vocabulary.
Never heard that term before.
And to quote Office Space: "Why should I change my name, he is the one who sucks". So personally I won't adapt "Artificial General Intelligence" into my vocabulary.
Some punk-ass bitches from MIT?
The rest of the world just says AI. It's like a million to one, we win. Don't make me have to call M.C. Hawking.
The rest of the world just says AI. It's like a million to one, we win. Don't make me have to call M.C. Hawking.
[deleted]
Those lessons can be useful, though. You might make the same mistakes.
I have taught AI many times now and I must agree.
You don't know how many times I have heard a student claim that they have made some type of major advance. But on inspection it usually comes down to something that requires exhaustive search.
I think that there is definitely a place for new ideas in the field, but I don't think they will come from complete novices. I think they will come from those who have CRITICALLY read the basic literature. And probably from those who have read more than just the basics too.
My own pet peeve on the field now is the over emphasis on asymptotic complexity. Sure its important, but its not exactly an AI result. The scruffies need to make a come back and start coding! See my own meager efforts (http://www.anaphoric.com)
You don't know how many times I have heard a student claim that they have made some type of major advance. But on inspection it usually comes down to something that requires exhaustive search.
I think that there is definitely a place for new ideas in the field, but I don't think they will come from complete novices. I think they will come from those who have CRITICALLY read the basic literature. And probably from those who have read more than just the basics too.
My own pet peeve on the field now is the over emphasis on asymptotic complexity. Sure its important, but its not exactly an AI result. The scruffies need to make a come back and start coding! See my own meager efforts (http://www.anaphoric.com)
Amazing how people with all this knowledge still haven't gotten anywhere close to AI.
It might even be something simple, overlooked, done a little differently.
It might even be something simple, overlooked, done a little differently.
Let us know when you find it.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
Intelligence is about finding algorithms for successful survival within a given environment. It's some sort of a meta-algorithm, although I have no idea if it's simple or not, but we can be sure it does exist.
> Let us know when you find it.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
Cf. Evolution.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
Cf. Evolution.
Evolutions Are Stupid (But Work Anyway): http://www.overcomingbias.com/2007/11/evolutions-are-.html
I do not want to wait five billion years.
AKA exhaustive search...
Sorry for repeating myself (comment lost in this huge thread), but I think it's the definition of intelligence we should look for first. Once we have a clear and unambiguous definition of intelligence, only and only after that we may be able to think about implementation. And I'm sure implementation would be straightforward once we know what we are trying to achieve.
It's actually amazing if not ridiculous that researchers always start from the other end of a cigar: they raise ideas, one after another, but they don't even try to explain what problem exactly they are solving.
It's actually amazing if not ridiculous that researchers always start from the other end of a cigar: they raise ideas, one after another, but they don't even try to explain what problem exactly they are solving.
[deleted]
If we all took that attitude, we'd never get anything done.
The process of discovery is hands-on, learning by doing. You don't know all the rules at the outset, or all the definitions, or even know what you don't know. You find out as you go along. Thus it has been for every great invention from sex to agriculture to post-it notes to landing a man on the Moon and back to on-demand porn.
The real requirement is that we have some way to test it; that we can do, e.g. by conversation. To cop an example from quantum mechanics (which I know nothing about), we don't have to understand why it is like that in order to make predictions with it.
We don't need a clear and unambiguous definition of intelligence in order to tell that other human beings are intelligent. Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
This was quite impressive, and on the right track, seemingly: http://hci.stanford.edu/~winograd/shrdlu/
The process of discovery is hands-on, learning by doing. You don't know all the rules at the outset, or all the definitions, or even know what you don't know. You find out as you go along. Thus it has been for every great invention from sex to agriculture to post-it notes to landing a man on the Moon and back to on-demand porn.
The real requirement is that we have some way to test it; that we can do, e.g. by conversation. To cop an example from quantum mechanics (which I know nothing about), we don't have to understand why it is like that in order to make predictions with it.
We don't need a clear and unambiguous definition of intelligence in order to tell that other human beings are intelligent. Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
This was quite impressive, and on the right track, seemingly: http://hci.stanford.edu/~winograd/shrdlu/
We don't need a clear and unambiguous definition of intelligence in order to tell that other human beings are intelligent.
The problem here is that not every human being is intelligent and yes, we are actually trying to define intelligence through IQ tests, for example.
Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
So when you need accounting software you say: "write me something that will be as clever as my accountant". Is that the way you formulate tasks for software engineers?
The problem here is that not every human being is intelligent and yes, we are actually trying to define intelligence through IQ tests, for example.
Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
So when you need accounting software you say: "write me something that will be as clever as my accountant". Is that the way you formulate tasks for software engineers?
> we are actually trying to define intelligence through IQ tests, for example
Fail. IQ tests only measure how well you do on IQ tests.
> The problem here is that not every human being is intelligent
Unless they're in a vegetative state, they are more intelligent than any artificial system, so far.
> Is that the way you formulate tasks for software engineers?
The only way to make a specification so precise that it does exactly what you want is to implement it, and then the code itself becomes the specification.
Fail. IQ tests only measure how well you do on IQ tests.
> The problem here is that not every human being is intelligent
Unless they're in a vegetative state, they are more intelligent than any artificial system, so far.
> Is that the way you formulate tasks for software engineers?
The only way to make a specification so precise that it does exactly what you want is to implement it, and then the code itself becomes the specification.
So, what you mean by saying something is more intelligent than something else? What criteria are you using to evaluate that?
And yes, it's the way programming works - when you know what you are trying to achieve. Software is about input and output and unless they are deterministic, you can't write code.
And yes, it's the way programming works - when you know what you are trying to achieve. Software is about input and output and unless they are deterministic, you can't write code.
> So, what you mean by saying something is more intelligent than something else?
Obviously people say that all the time despite our lack of a definition for exactly what it is, so we apparently don't need an exact definition.
> and unless they are deterministic, you can't write code.
You might be interested to know that the behavior of your OS isn't deterministic once you add a second CPU.
Obviously people say that all the time despite our lack of a definition for exactly what it is, so we apparently don't need an exact definition.
> and unless they are deterministic, you can't write code.
You might be interested to know that the behavior of your OS isn't deterministic once you add a second CPU.
SHRDLU was a toy program that was not on the right track.
Let's see anything better.
I agree.
What program isn't a toy? Stop being opprobrious and come up with a better example.
It simply does not exist yet. There were no good airplanes before Orville and Wilbur showed up on the scene. I hope there will be some breakthroughs soon, but scientific discoveries can't be planned/scheduled, so who knows how long it'll be...
SHRDLU is nothing more than a really well-done ELIZA for a really small domain.
SHRDLU is nothing more than a really well-done ELIZA for a really small domain.
Why do you say it is just an Eliza?
Or you might come up with something that works, by avoiding having your mind crammed and distorted by the classical mess.
It's one thing to notice that beginners can sometimes make contributions by luck, just by not knowing how hard something is supposed to be. It's another to use this reasoning to glorify lack of knowledge.
Whenever I see someone succeed rapidly I assume it isn't a newcomer, that there are depths of effort I cannot yet see.
Whenever I see someone succeed rapidly I assume it isn't a newcomer, that there are depths of effort I cannot yet see.
Many, many people have gone into the field to study the traditional techniques, and glorifying this knowledge has not led to AI.
I don't think a beginner is going to create AI by "luck". But a beginner's mind is not cluttered in the same way as an expert's. This may be what it takes to get working AI.
Expert knowledge is appropriate when discussing proven solutions. AI is speculative at this point.
I don't think a beginner is going to create AI by "luck". But a beginner's mind is not cluttered in the same way as an expert's. This may be what it takes to get working AI.
Expert knowledge is appropriate when discussing proven solutions. AI is speculative at this point.
(response: http://news.ycombinator.com/item?id=90476)
(response: http://news.ycombinator.com/item?id=90481)
Ignoring existing work in AI is as if you would try building a flying machine, ignoring existing knowledge in aerodynamics. You'd be right back where Leonardo was. People are doing that, nevertheless, but if you do that in AI, chances are you'll be just fabricating crackpot theories.
The difference between the field of AI and the field of aerodynamics is the state-of-the-art in the latter has produced working results -- to say the least!
Meanwhile humans farm gold and solve CAPTCHAs.
Meanwhile humans farm gold and solve CAPTCHAs.
Why would you want someone to do this? Wouldn't that simply lead one to remake all the mistakes that have been made in the field?
Doing what has been done in the field would repeat the mistakes.
Doing something original might actually be, you know, original.
Doing something original might actually be, you know, original.
"Do not deny the classical approach, simply as a reaction, or you will have created another pattern and trapped yourself there." -- Bruce Lee
http://en.wikipedia.org/wiki/Bruce_Lee#Philosophy
That's referring to refusing to use something you KNOW works, simply because it is traditional. Quite different from seeking your own path.
If you follow the classical path, you will become bogged down in the vast field of AI.
Ramanujan started from first principles and worked from there. My guess is that the person(s) who create "actual" AI will use that approach.
If you follow the classical path, you will become bogged down in the vast field of AI.
Ramanujan started from first principles and worked from there. My guess is that the person(s) who create "actual" AI will use that approach.
That's a valid point.
But there's value in studying things that didn't work. A huge number of obvious approaches to AI have been tried already, and the insights on why they don't work, or why they work on this problem but not this other one, are often the result of huge quantities of time spent on subtle traps and dead ends. To ignore them risks wasting your time all over again.
I think AI boils down to just the problem of managing complexity that all of cs is about. The eventual solution will be vast, and will require designing lots of separate subsystems that work in very different ways and yet need to communicate in subtle ways.
That emphasis on scale goes for research in general, I think. I recommend this video of Malcolm Gladwell talking about the nature of genius and how it's changed over the years. http://www.newyorker.com/online/video/conference/2007/gladwe... He compares the decoding of the rosetta stone and linear B 40 years ago with Andrew Wiles proving Fermat's last theorem in the past decade, and how fundamentally different their respective approaches are.
Both approaches may work, but you have to decide what attitude you want to take. You can either try to be a single monster-mind like Ramanujam sweeping through vast areas of research, or you can assume that won't happen and resign yourself to a lot of effort and learning before you're able to synthesize something useful.
To summarize: I agree that you want to avoid cluttering your mind with the ideology of past approaches. There's huge value though in simply studying the episodic history of a field, to be aware of what has been tried, what worked and what didn't.
But there's value in studying things that didn't work. A huge number of obvious approaches to AI have been tried already, and the insights on why they don't work, or why they work on this problem but not this other one, are often the result of huge quantities of time spent on subtle traps and dead ends. To ignore them risks wasting your time all over again.
I think AI boils down to just the problem of managing complexity that all of cs is about. The eventual solution will be vast, and will require designing lots of separate subsystems that work in very different ways and yet need to communicate in subtle ways.
That emphasis on scale goes for research in general, I think. I recommend this video of Malcolm Gladwell talking about the nature of genius and how it's changed over the years. http://www.newyorker.com/online/video/conference/2007/gladwe... He compares the decoding of the rosetta stone and linear B 40 years ago with Andrew Wiles proving Fermat's last theorem in the past decade, and how fundamentally different their respective approaches are.
Both approaches may work, but you have to decide what attitude you want to take. You can either try to be a single monster-mind like Ramanujam sweeping through vast areas of research, or you can assume that won't happen and resign yourself to a lot of effort and learning before you're able to synthesize something useful.
To summarize: I agree that you want to avoid cluttering your mind with the ideology of past approaches. There's huge value though in simply studying the episodic history of a field, to be aware of what has been tried, what worked and what didn't.
If you follow your own path, you will later find various areas of overlap and intersection, but you may avoid sinking into the tar-pit as you would when starting out with ALL the dead weight. It takes tremendous effort to load your brain with all the existing knowledge, and once you do, you may find yourself only able to think in terms of it, having conditioned yourself to other people's failed modes of thinking.
Yes, learning by doing is good. But learning by doing involves feedback loops between reading about what others did and trying to do your own. Artists must both learn to appreciate art and keep practicing their own craft.
I'm not saying that you have to learn everything before doing anything. That's a strawman. I'm saying you shouldn't ignore any source of knowledge, that you should look everywhere taking the good and leaving out the bad. And that's a Bruce Lee quote as well.
The concern about contagion is valid; my attitude towards it is that it happens not so much from what you read but who you talk to and the social circles you move in. Keep those diverse and you should be fine.
I'm not saying that you have to learn everything before doing anything. That's a strawman. I'm saying you shouldn't ignore any source of knowledge, that you should look everywhere taking the good and leaving out the bad. And that's a Bruce Lee quote as well.
The concern about contagion is valid; my attitude towards it is that it happens not so much from what you read but who you talk to and the social circles you move in. Keep those diverse and you should be fine.
> The idea that you have to learn everything before doing anything is a strawman.
If you don't learn everything, then you must have some independent criterion with which to pick and choose; and having such a facility, one may as well simply start down whichever path one feels is most appropriate anyway.
In other words, if you can somehow rule out large swaths of possibilities without even trying them, then you must have some intuitive feel for the problem (or at least, have an intuitive feel for what you think the solution is likely to resemble, whether or not you turn out to be correct). In such a case, you are investigating based on your own intimations and your efforts will develop from there, rather than from from a top-down view -- which has so far been amply demonstrated not to work.
Edit: The recommendation to look "everywhere" is untenable, because at some point you have to stop casting about if you are going to get anything done yourself.
If you don't learn everything, then you must have some independent criterion with which to pick and choose; and having such a facility, one may as well simply start down whichever path one feels is most appropriate anyway.
In other words, if you can somehow rule out large swaths of possibilities without even trying them, then you must have some intuitive feel for the problem (or at least, have an intuitive feel for what you think the solution is likely to resemble, whether or not you turn out to be correct). In such a case, you are investigating based on your own intimations and your efforts will develop from there, rather than from from a top-down view -- which has so far been amply demonstrated not to work.
Edit: The recommendation to look "everywhere" is untenable, because at some point you have to stop casting about if you are going to get anything done yourself.
I wasn't recommending looking everywhere, I was recommending being open to looking anywhere that catches your fancy.
The thing that is independent/unique about somebody isn't a static criterion that remains unchanging over time, it's more the trajectory you end up taking that's constantly evolving over time. All I'm saying is: don't exclude reading about past failures when you make your choices on what to do next. It's fine to say ok, let's stop reading for a while and try building this one thing my mind has settled on. That's a great habit to acquire.
The unpronounceable author of Flow: http://www.amazon.com/Flow-Psychology-Experience-Mihaly-Csik... says the most successful people alternate periods in closed states when they focus on a specific concrete project with periods in open states where they demonstrate huge appetite for new ideas.
I'm starting to think we're saying the same thing in different words and with slightly different emphasis. It is thus with most interesting conversations.
The thing that is independent/unique about somebody isn't a static criterion that remains unchanging over time, it's more the trajectory you end up taking that's constantly evolving over time. All I'm saying is: don't exclude reading about past failures when you make your choices on what to do next. It's fine to say ok, let's stop reading for a while and try building this one thing my mind has settled on. That's a great habit to acquire.
The unpronounceable author of Flow: http://www.amazon.com/Flow-Psychology-Experience-Mihaly-Csik... says the most successful people alternate periods in closed states when they focus on a specific concrete project with periods in open states where they demonstrate huge appetite for new ideas.
I'm starting to think we're saying the same thing in different words and with slightly different emphasis. It is thus with most interesting conversations.
> I'm starting to think we're saying the same thing in different words and with slightly different emphasis. It is thus with most interesting conversations.
I LIKE TURTLES
I LIKE TURTLES
I think that is a function of attitude, and not a function of how much existing material you've read.
It's pretty hard to learn something just by reading. It takes doing. That's where you get stuck in someone else's mindset.
If you can get stuck in a mindset by following a specific trajectory of action then it doesn't matter whether you're following somebody deliberately or just blindly finding the same path.
In fact, reading what somebody did will often help you avoid going down a path. I've read of many many more algorithms, successful and unsuccessful, than I've ever implemented.
In fact, reading what somebody did will often help you avoid going down a path. I've read of many many more algorithms, successful and unsuccessful, than I've ever implemented.
You don't really understand something until you implement it. And AI is unlikely to be an algorithm like those we are familiar with...
> You don't really understand something until you implement it.
Yup. We've already established that one has to prioritize what to implement. The priority clearly shouldn't always go towards doing something rather than reading what somebody else did. (Children would never learn to write that way.) Instead we prioritize differently at different times, and we try to have lots of feedback between reading and doing, by things like active reading.
Yup. We've already established that one has to prioritize what to implement. The priority clearly shouldn't always go towards doing something rather than reading what somebody else did. (Children would never learn to write that way.) Instead we prioritize differently at different times, and we try to have lots of feedback between reading and doing, by things like active reading.