Ken Jennings on Watson(nydailynews.com)
nydailynews.com
Ken Jennings on Watson
http://www.nydailynews.com/opinions/2011/02/17/2011-02-17_ken_jennings_exclusive_oped_jeopardy_champ_says_computer_nemesis_watson_had_unfa.html
41 comments
I'd be interested to know if Jennings had any influence on the title -- the phrase "unfair advantage" isn't used in the text -- he calls it an "advantage", but "unfair" lends it an entirely different meaning. The article is a very even-handed discussion of Watson; the headline is needlessly provocative, and makes Jennings sound like a whiner.
In his Q&A session after day 1, Jennings said that "I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."
So my guess is that the daily news had more to do with that title than Ken. It doesn't seem like him, at any rate.
So my guess is that the daily news had more to do with that title than Ken. It doesn't seem like him, at any rate.
Judging from the other content advertised on the page, the Daily News is not a master of subtlety.
I'm certain it was a copywriter who created that headline. Would you read "How to learn to play Piano" or "They Laughed Until I sat down to play?"
I know HN tends to dislike overtly dramatic headlines but copywriting is also an interesting art.
I know HN tends to dislike overtly dramatic headlines but copywriting is also an interesting art.
Died with his buzzer in his hand, Lord Lord...
Nice image by Jennings. But I was surprised to hear him complain about the buzzer reaction time advantage. In an earlier interview with the Washington Post he discounted this, saying that the best human players anticipate the timing while Watson has to wait for the signal.
"On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best... But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."
http://live.washingtonpost.com/jeopardy-ken-jennings.html?hp...
Nice image by Jennings. But I was surprised to hear him complain about the buzzer reaction time advantage. In an earlier interview with the Washington Post he discounted this, saying that the best human players anticipate the timing while Watson has to wait for the signal.
"On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best... But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."
http://live.washingtonpost.com/jeopardy-ken-jennings.html?hp...
Like sciolistse said I think the "unfair" part was a bit of editorializing.
From a competition perspective, I think it was fair: every player is entitled to capitalize his own advantages within the rules.
However, I think this would have been more interesting intellectually to isolate Watson's "thinking", with everything else on a level playing field.
However, I think this would have been more interesting intellectually to isolate Watson's "thinking", with everything else on a level playing field.
In summary: the buzzer.
You could pretty clearly see this if you were watching the game. Ken looked pretty bummed out every time he clearly knew the answer, but missed buzzing in by a few milliseconds or so.
Out of curiosity, how does Jeopardy work? Can you buzz in as soon as you know the answer, or do you have to wait for the host to finish?
You could pretty clearly see this if you were watching the game. Ken looked pretty bummed out every time he clearly knew the answer, but missed buzzing in by a few milliseconds or so.
Out of curiosity, how does Jeopardy work? Can you buzz in as soon as you know the answer, or do you have to wait for the host to finish?
You have to wait for the host to finish.
Or, more precisely, you have to wait for the guy backstage who pushes his own button after the host has finished. If you push your button before he pushes his, you're locked out for something like half a second. His button is also connected to some light bulbs which are never shown on camera, which tell the contestants what's going on.
(But there are some human contestants who ignore the lights and try to time it anyway --- having practiced a lot at home, where they couldn't see them. One of them was a recent college champion, who was one of the pre-show speakers at a group viewing I was at for Monday's show.)
Or, more precisely, you have to wait for the guy backstage who pushes his own button after the host has finished. If you push your button before he pushes his, you're locked out for something like half a second. His button is also connected to some light bulbs which are never shown on camera, which tell the contestants what's going on.
(But there are some human contestants who ignore the lights and try to time it anyway --- having practiced a lot at home, where they couldn't see them. One of them was a recent college champion, who was one of the pre-show speakers at a group viewing I was at for Monday's show.)
Watson received an electronic signal that the buzzers were active. The human contestants have to time it via Trebek's voice or the light.
An electronic signal and a photonic signal ("the light") both travel at the same speed.
[deleted]
Once Trebek finishes, the buzzers start working.
John Henry beat the machine he was up against: http://en.wikipedia.org/wiki/John_Henry_(folklore)
They should have had a human from the IBM team manning the buzzer based on a visual cue that Watson is ready to answer. Much like Deep Blue didn't actually move the chess pieces with a robotic arm.
This is a great idea, actually. This was supposed to be a demonstration of natural language parsing and information retrieval, not a demonstration of a computers ability to react quickly.
We already know that computers are good at the latter.
We already know that computers are good at the latter.
Or to give Watson a taste of the human world perhaps they could route the signal via a congested cell carrier data plan.
Why? In this case, Watson use a buzzer just like the human contestants. It still has to take time to figure out the answer.
This makes we wish we could rate Watson's performance against the players in a way that didn't involve the buzzer. For instance, if Watson and each player were able to give responses for every single clue in the game, who would have the highest percentage of correct responses and/or the highest score?
Except, that's not Jeopardy. That's "Answer these Trivia Questions". A legitimate game, but not one that is likely to get much viewership.
Everyone misses what should be knocked about Watson. The advantage isn't the buzzer per se. But rather the text message to get the question and the signal to indicate the question had been read.
The way it should work in the future is that Watson reads the question from the screen, listens to Alex, and sees the signal from the light.
Given the standard way Jeopardy text is written though, I suspect that all of these additional requirements won't change the outcome by much.
Watson is just a better Jeopardy player than humans. Are there other trivia games where humans are better? Probably so. But people don't watch these other games in prime time.
Everyone misses what should be knocked about Watson. The advantage isn't the buzzer per se. But rather the text message to get the question and the signal to indicate the question had been read.
The way it should work in the future is that Watson reads the question from the screen, listens to Alex, and sees the signal from the light.
Given the standard way Jeopardy text is written though, I suspect that all of these additional requirements won't change the outcome by much.
Watson is just a better Jeopardy player than humans. Are there other trivia games where humans are better? Probably so. But people don't watch these other games in prime time.
Well, right, that wouldn't be Jeopardy, but ultimately, I don't really care about Jeopardy, thus my question. What I (and most of us here, I think) would find interesting is how Watson's language processing abilities compares to that of (very smart) people. I don't really care how fast Watson can ring a buzzer. If there was a way to gauge that, I'd find it interesting.
I assume you mean "natural" language processing.
For that trivia is the wrong test. It's better to just give it paragraphs and ask it questions about the paragraph. And of course have increasing levels of difficult. More like the SAT reading comprehension test.
This would extract how it stores data brought in from structured/unstructed sources and the actual task of processing natural language.
I suspect it will be a fair bit worse at processing natural language than even the average person.
For that trivia is the wrong test. It's better to just give it paragraphs and ask it questions about the paragraph. And of course have increasing levels of difficult. More like the SAT reading comprehension test.
This would extract how it stores data brought in from structured/unstructed sources and the actual task of processing natural language.
I suspect it will be a fair bit worse at processing natural language than even the average person.
His analogy about Kasparov needing to solve a long division problem is whack. Buzzing is part of the game of Jeapordy. For all we know he wouldn't have been a champion in a version of the game that didn't involve buzzing.
No use in complaining, this was a cool PR stunt. You don't hear the Washington Generals bitching about the Globetrotters.
I think I remember reading that Watson didn't buzz in until it was sure it had an answer which was giving the human opponents an advantage because they could buzz in based of thinking that by the time they are asked for their answer they can have come up with the correct one.
Looks like they must have overcome that issue.
Looks like they must have overcome that issue.
[deleted]
Ugh. He should have skipped writing about this. No matter the validity of his words, whatever he says will be percieved as sour grapes.
What in the world. The challenge was the natural language processing. Considering that Watson did that part admirably well, complaining about computers having better reflexes is just silly.
The buzzer timing is a larger part of winning than knowing the answers. I would say, even though Jennings never does, that it was a huge advantage.
But the buzzer timing is also part of Jeopardy. It's just a part of the game. You can change the game to disadvantage the computer, but then why not change other rules like require all responses to be handwritten using a hand with human blood flowing through it to a pumping heart, which is connected to a working human brain.
The point is that while Watson is a big step forward in language processing and information retrieval, it's unfair to conclude that he does these things as well as the best humans. Jeopardy isn't a particularly good test of these abilities (though it is good theater) because the additional variable of buzzer-timing screws up your results.
As was pointed out by another commenter, if the game had been "everyone who knows the answer write it down and get credit", Watson probably wouldn't have won, and absolutely not by so much.
So I think Jennings is just cautioning us not to assume that AI has achieved human level language and information capabilities when really a totally unrelated and unimpressive other dimension of the game is the culprit.
As was pointed out by another commenter, if the game had been "everyone who knows the answer write it down and get credit", Watson probably wouldn't have won, and absolutely not by so much.
So I think Jennings is just cautioning us not to assume that AI has achieved human level language and information capabilities when really a totally unrelated and unimpressive other dimension of the game is the culprit.
I seem to recall Kasparov complaining of unfairness/cheating too, when he was beat by Deep Blue.
I'd be more impressed by this massive IBM advertisement, if the computer received its information the exact way the other contestants did (verbally / visually looking at screen).
It wouldn't make much difference. OCR on the text on the screen would be fine.
If it had to do speech recognition for Alex Trebek, that would make it more difficult.
If it had to do speech recognition for Alex Trebek, that would make it more difficult.
Yep, it would be more difficult and probably make its buzzer reaction time a little more "honest".
the OCR would be nearly perfect since there's very little variation in the way the text is displayed. The voice part wouldn't matter since the all top level players use the visual clue to determine the answer long before Trebek finishes reading it.
More honest? It would make it impossible, since if the computer had to parse Trebek's voice, it would finish getting the entire clue at the same time that the humans are pressing the buzzer.
Are you the same guy who left this comment: http://www.crunchgear.com/2011/02/14/angry-nerds-ibms-watson...