Computer beats Jeopardy champs(cbc.ca)
cbc.ca
Computer beats Jeopardy champs
http://www.cbc.ca/technology/story/2011/01/14/computer-jeopardy.html
16 comments
Anyone know how the buzzers work? It's my impression that they are disabled until a moment after the question has been read completely, and then the first to buzz in gets a chance to answer. I wonder if they took steps to ensure that the computer doesn't have an edge in buzzing in first.
There is an excellent comment on Reddit by an ex-Jeopardy player who explains how the buttons work and why it matters with AI.
http://www.reddit.com/r/cogsci/comments/f253c/watch_ibms_wat...
http://www.reddit.com/r/cogsci/comments/f253c/watch_ibms_wat...
I think that's how it works (they only get enabled after the question has been read, while you get penalised somehow if you buzz in too early. I imagine they built in some random delays so the computer didn't always buzz in at the millisecond.
That would have been very sporting of them as I recall seeing an interview with Jeopardy winners where they mentioned the crucial role the buzzer plays in the game and how skill with it provides a competitive advantage.
Engadget has a video of the actual contest, and an interview with the IBM guys:
http://www.engadget.com/2011/01/13/ibms-watson-supercomputer...
http://www.engadget.com/2011/01/13/ibms-watson-supercomputer...
Hardly amazing that a computer with access to billions of pages of information can out-trivia humans.
I suppose the story here is supposed to be the natural language processing, but really the space which is covered by Jeopardy questions is pretty confined. Color me unimpressed.
I suppose the story here is supposed to be the natural language processing, but really the space which is covered by Jeopardy questions is pretty confined. Color me unimpressed.
Have you watched Jeopardy ever? The 'questions' are often pretty obscurely worded, and the IBM computer often comes to completely bonkers conclusions despite your supposed 'confined question space'.
I want to see it go on 'are you smarter than a fifth grader'. The sorts of questions they should ask aren't ones that require knowledge but creativity, understanding, and elision of concepts. For example:
If Dog is three and elephant is eight then what is ostrich?
Guarantee a fifth grader gets seven, but Watson is stumped.
If Dog is three and elephant is eight then what is ostrich?
Guarantee a fifth grader gets seven, but Watson is stumped.
I have watched Jeopardy. It's questions are no more obscurely worded than what my mom types into a search engine. The search engine is often able to produce reasonable results, which is impressive in isolation and I recognize the difficulty of that, but not something to make a huge PR event out of, as there are many applications with similar parsing ability (like search engines).
Jeopardy always words something in the same format with a lot of information about the answer. Perhaps I am too far removed, but to me it doesn't seem like a huge advance in natural language research or anything worth all this attention. It's clearly a PR event purchased by IBM to try to sell more software along that line, hardly revolutionary.
Jeopardy always words something in the same format with a lot of information about the answer. Perhaps I am too far removed, but to me it doesn't seem like a huge advance in natural language research or anything worth all this attention. It's clearly a PR event purchased by IBM to try to sell more software along that line, hardly revolutionary.
Ignorance is bliss so they say. This is not about a computer beating humans in trivia questions, but about computers learning to understand humans. This is a major milestone in natural language processing and imagine what this can do when it will be available on a wider scale.
Watch this please:
Building Watson - A Brief Overview of the DeepQA Project http://www.youtube.com/watch?v=3G2H3DZ8rNc
Watch this please:
Building Watson - A Brief Overview of the DeepQA Project http://www.youtube.com/watch?v=3G2H3DZ8rNc
Yes, but that's the point, it doesn't actually "understand". When it answers a question in the category of "rhyme time" it's not because it actually understands what a rhyme is, but because it's been programmed to. If Jeopardy renamed the rhyme time category "time rhyme" (requiring the contestant to rhyme AND invert) watson would be up the creek, whereas someone like Ken Jennings could probably even figure it out without Alex Trebek explaining the category, and you or I would figure out the scheme after flubbing the first one.
So what exactly then is the definition of understanding, vs being programmed to act like one understands? Do I, for instance, understand pain when I remove the hand from the hot water, or is it because the reflexes in my brainstem are programmed to? Or do I understand mathematical concepts like the monads I use in Haskell when I probably will fail horribly to descripe them adequately or manipulate them in a proper mathematicial setting?
Appreciate what you're saying and I did get that Watson doesn't truly "understand" us yet, but still, what has been achieved so far is pretty impressive and would have useful applications even in its current form. Curious to see what IBM will do next with this.