One question I have about Watson that I don't recall being mentioned in any videos or articles so far - what sort of interface does Watson receive the questions over? Is Watson performing speech recognition or getting the text of the question via some sort of interface?
This was answered in the Nova special, in passing. The question is fed to Watson as text at the same moment it appears on the display that the contestants can see. Likewise for correct answers. Watson is not performing speech recognition.
The real question I have is according to that episode Watson does not take into account the category only the 'answer' and the already shown questions. It seems _really_ odd that they would ignore this bit of information.
I'm currently finding out what information we're allowed to share about how the avatar works and what went into developing it. The problem is we're so far down the totem pole I probably won't know for a while yet. :-/
One thing I'm interested in is any skew in the questions from normal. In particular I hope they ask linguistically tricky questions where you can't even figure out what's being asked at first. I felt like they went a bit easy on that front in the preview round:
I doubt they are skewing the questions. That kind of defeats the point. I suspect Jeopardy is constantly adding to a pool of questions. My hope is that they simply grab from that pool just like any other episode.
Note, the hardest questions for Watson are short questions -- the reduced time hurts Watson more than most human competitors.
The writers had no knowledge of which clues would be used for the Watson episodes; they only mention that they don't use audio or video clues. I certainly haven't done any analysis of it, but I didn't feel like they were any less common in the preview round than normal. Between all the games I'm sure we'd see more "linguistically tricky" clues. I'm hoping for some good "before and after" questions.
It’s a shame that this is (necessarily!) such an insular challenge. Everybody knows what chess is all about, I fear that the impact of this game will be limited to the US or the Anglosphere. Just as an example, there has been no Jeopardy on German TV since 2000, it’s not really a part of German pop culture and because of all the puns it doesn’t translate well.
(Question for native speakers: When watching the practice round [0] are you generally able to keep up and answer the questions? The speed with which the game was moving made it nearly impossible for me to follow or enjoy the game. I would like to know what the experience is like for native speakers.)
Well, that's kind of the point. Chess is a hard game, but it doesn't require broad knowledge of a human culture. If you're going to build a machine that exhibits such knowledge, you've got to pick a culture.
I can definitely follow along with this or any game of Jeopardy!, but my problem is that I just don't know all the answers. To me this is part of the appeal.
This is definitely faster-paced than regular episodes of Jeopardy!, but I don't think it's prohibitively fast for native speakers.
It's too bad that Jeopardy! doesn't fit in with international pop culture better. I think it's a perfect fit for a lot of AI problems.
Typically real jeopardy will show the text on the screen. I can get about 30 (sometimes up to 35ish) "questions" for a normal show but it is all about reading it to get those extra few brain cycles.
Don't fret; you can still tune in after another decade or so for a machine to beat us at GO. Given IBM's trending business interest in corporate AI I would not be surprised if it were another IBM sponsored team that finally builds the successful bot.
If you're okay with waiting a few hours/days, The Pirate Bay seems to have a good selection of torrents of recent Jeopardy! episodes, though I've not yet tried it.
You'll also be able to see a full transcript of the board, as well as all questions provided by contestants, at www.j-archive.com.
I had thought from the counter that IBM had run on the Watson page that there might be but it seems that us non-US residents will have to wait. Still very much looking forward to this. History in the making.
Hulu refers to crackle.com. Crackle has "minisodes" which seem to date from 2002-2003.
It is owned by Sony, distributed by CBS and seems to be syndicated, which means there is no network that carries it specifically, and thus no online distribution, as each market has a different station (Eg: Fox might buy it in one market while ABC might buy it in another).
In case anyone is interested to know Watson's opponents: Watson will compete against Brad Rutter, the current biggest all-time money winner on Jeopardy!, and Ken Jennings, the record holder for the longest championship streak [source: wikipedia]
I'm glad they touched on the idea that this is actually a "Human vs Human" competition... Really good Jeopardy players vs. a team of humans that built their own "Jeopardy Player".
Watson winning the tournament is a triumph for humanity, not just for machines.
Did anyone else notice that the vignettes about Watson's creation featured IBM researchers using MacBook Pros? So much for "International Business Machines." ;)
As a former IBM employee, you can pretty much use what ever type of machine you want for your work as long as it doesn't interfere with what you do. There is a significant Mac subculture internal to IBM, and they love and suppor it. However a huge percent of their 600,000 employee's still use IBM branded laptop's and desktop's.
Much of success on Jeopardy is not just deciphering clues in the answers, but your timing on ringing in to give the question. I'd imagine a machine could get really good at getting the timing down. Does Jeopardy have a way of varying who rings in first to keep things more fair?
Something I wrote from a previous thread. The one piece of info that wasn't in this comment is that Watson can NOT anticipate when Trebek is about to finish the question. Watson must wait until it gets a signal that the buzzer is now available, and only then can it begin the process to physically depress the button.
Remember that Watson also has to depress a physical button (the same buzzer everyone else uses).
The eye to finger path for humans is about 200ms. It probably takes about 100ms for Watson to physically press the button. So Watson is about 100ms faster. But that also gives humans about a 100ms window in which to beat Watson. This means that you need to start your press 100-200ms before Trebek finishes his last word.
That's pretty good sized window for most people given you are reading the question along with Trebek. If the person who turns the light on is very consistent, I think a human who is good at this could consistently beat Watson.
The timing of when players are allowed to click in is controlled by a human. Someone backstage decides at what moment Alex is finished speaking and then opens the clickers. If you click too early, you get ~ 300ms delay penalty which gives the other players a chance to click in.
Click timing is indeed very important but I do not think Watson has any special advantage there.
That’s Jeopardy. Whoever buzzes first gets to answer, I don’t see what’s unfair about that. Computers can react faster than humans, why should they be denied that advantage?
In case anyone missed it the first time around, the nytimes mag had a pretty good writeup on Watson back in June -- might be worth instapapering and reading later if you're going to catch the broadcast this week:
As an AI researcher I'm excited to watch this week. Even if it's not the most elegant artificial Jeopardy player imaginable, it raises the public profile of a lot of AI & ML topics and might encourage and inspire other groups to tackle ambitious projects.
I think it's fun to think about how Watson type intelligence will be at the average consumer's figertips (and affordable, to boot) in less that 20 years.
A quick look at the TOP500 supercomputers puts about a 10,000x increase in raw flops since 1993. Using that as a rough benchmark (a very rough one), we're looking at some impressive stuff in the next twenty years. It's not unreasonable to think that:
current consumer processor in FLOPS * 10,000 < Watson's grid's FLOPS
Naturally, that is a very rough estimate and has no scientific bearing at all.
I just walked into my local bar, asked the bartender to switch to channel 7, and voila! There weren't too many people around, but it was fun (and between breaks I chatted with a couple of drunk chicks who were trying to drown their V-Day sorrows in shots of vodka).
Assuming the space required to run watson halves every two years and v:: of the human brain is v1 = 1500cm³, and Σv of Watson's servers = v2 = 90 x (17.5cm x 44.0cm x 73.0xcm) = 5.05e6 cm^3.
So Watson will be the size of a human brain in t = - ln(v1/v2)/ln(2).
I'm sure there are those that disagree strongly, but I feel like this isn't as much an advance for AI as it is another interesting combination of filtered human-structured knowledge and computation power. Just as Deep Blue brute-forced chess, this is mostly a brute-force of another, albeit more open-ended domain (i.e. Q/A).
I'm not arguing that this isn't an impressive accomplishment, but that the statistical-learning stream of research is likely a conceptual local optima that yields the best results in the near term but is probably unrelated to the way we ultimately achieve a creative, general AI.
[+] [-] brown9-2|15 years ago|reply
http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth....
One question I have about Watson that I don't recall being mentioned in any videos or articles so far - what sort of interface does Watson receive the questions over? Is Watson performing speech recognition or getting the text of the question via some sort of interface?
[+] [-] lrm242|15 years ago|reply
[+] [-] jasonkester|15 years ago|reply
Amazing that PBS is locking down their videos. They're public television fer cryin' out loud.
[+] [-] crocowhile|15 years ago|reply
[+] [-] icefox|15 years ago|reply
[+] [-] mcav|15 years ago|reply
[+] [-] unknown|15 years ago|reply
[deleted]
[+] [-] swanson|15 years ago|reply
Game over, Watson.
[+] [-] zach|15 years ago|reply
That or the Star Trek self-destruct sequence.
[+] [-] nostrademons|15 years ago|reply
[+] [-] unknown|15 years ago|reply
[deleted]
[+] [-] waxpraxis|15 years ago|reply
http://automatastudios.com/automata-gives-ibms-watson-a-face...
[+] [-] kj12345|15 years ago|reply
http://www.youtube.com/watch?v=WFR3lOm_xhE
[+] [-] kenjackson|15 years ago|reply
Note, the hardest questions for Watson are short questions -- the reduced time hurts Watson more than most human competitors.
[+] [-] fefzero|15 years ago|reply
[+] [-] ugh|15 years ago|reply
(Question for native speakers: When watching the practice round [0] are you generally able to keep up and answer the questions? The speed with which the game was moving made it nearly impossible for me to follow or enjoy the game. I would like to know what the experience is like for native speakers.)
[0] http://www.youtube.com/watch?v=WFR3lOm_xhE
[+] [-] metageek|15 years ago|reply
Well, that's kind of the point. Chess is a hard game, but it doesn't require broad knowledge of a human culture. If you're going to build a machine that exhibits such knowledge, you've got to pick a culture.
[+] [-] fefzero|15 years ago|reply
This is definitely faster-paced than regular episodes of Jeopardy!, but I don't think it's prohibitively fast for native speakers.
It's too bad that Jeopardy! doesn't fit in with international pop culture better. I think it's a perfect fit for a lot of AI problems.
[+] [-] 3pt14159|15 years ago|reply
[+] [-] bayleo|15 years ago|reply
[+] [-] chaosmachine|15 years ago|reply
[+] [-] andrewce|15 years ago|reply
You'll also be able to see a full transcript of the board, as well as all questions provided by contestants, at www.j-archive.com.
[+] [-] jc123|15 years ago|reply
[+] [-] jackowayed|15 years ago|reply
[+] [-] patrickod|15 years ago|reply
[+] [-] unknown|15 years ago|reply
[deleted]
[+] [-] nika|15 years ago|reply
It is owned by Sony, distributed by CBS and seems to be syndicated, which means there is no network that carries it specifically, and thus no online distribution, as each market has a different station (Eg: Fox might buy it in one market while ABC might buy it in another).
This is near as I can tell looking into it.
[+] [-] shortlived|15 years ago|reply
[+] [-] sambeau|15 years ago|reply
[+] [-] 3pt14159|15 years ago|reply
Has a nice little video too.
[+] [-] quickpost|15 years ago|reply
Watson winning the tournament is a triumph for humanity, not just for machines.
[+] [-] savrajsingh|15 years ago|reply
[+] [-] bjg|15 years ago|reply
[+] [-] umjames|15 years ago|reply
[+] [-] kenjackson|15 years ago|reply
Remember that Watson also has to depress a physical button (the same buzzer everyone else uses). The eye to finger path for humans is about 200ms. It probably takes about 100ms for Watson to physically press the button. So Watson is about 100ms faster. But that also gives humans about a 100ms window in which to beat Watson. This means that you need to start your press 100-200ms before Trebek finishes his last word.
That's pretty good sized window for most people given you are reading the question along with Trebek. If the person who turns the light on is very consistent, I think a human who is good at this could consistently beat Watson.
[+] [-] cryptoz|15 years ago|reply
Click timing is indeed very important but I do not think Watson has any special advantage there.
[+] [-] ugh|15 years ago|reply
[+] [-] cj|15 years ago|reply
If it rings in every time there's no way it will win.
[+] [-] powrtoch|15 years ago|reply
[+] [-] icedpulleys|15 years ago|reply
http://www.nytimes.com/2010/06/20/magazine/20Computer-t.html...
As an AI researcher I'm excited to watch this week. Even if it's not the most elegant artificial Jeopardy player imaginable, it raises the public profile of a lot of AI & ML topics and might encourage and inspire other groups to tackle ambitious projects.
[+] [-] phren0logy|15 years ago|reply
http://twitter.com/robotwisdom/status/37200070742904832
[+] [-] olalonde|15 years ago|reply
[+] [-] bryanh|15 years ago|reply
A quick look at the TOP500 supercomputers puts about a 10,000x increase in raw flops since 1993. Using that as a rough benchmark (a very rough one), we're looking at some impressive stuff in the next twenty years. It's not unreasonable to think that:
Naturally, that is a very rough estimate and has no scientific bearing at all.[+] [-] KevBurnsJr|15 years ago|reply
[+] [-] ajays|15 years ago|reply
[+] [-] wmf|15 years ago|reply
[+] [-] gursikh|15 years ago|reply
Assuming the space required to run watson halves every two years and v:: of the human brain is v1 = 1500cm³, and Σv of Watson's servers = v2 = 90 x (17.5cm x 44.0cm x 73.0xcm) = 5.05e6 cm^3.
So Watson will be the size of a human brain in t = - ln(v1/v2)/ln(2).
We're looking at 11.7 years. So, 2023.
* Source: http://en.wikipedia.org/wiki/Watson_%28artificial_intelligen...
"Watson is made up of a cluster of ninety IBM Power 750 servers..."
* Source: http://www-03.ibm.com/systems/power/hardware/750/specs.html
"175 mm x 440 mm x 730 mm"
[+] [-] jal278|15 years ago|reply
I'm not arguing that this isn't an impressive accomplishment, but that the statistical-learning stream of research is likely a conceptual local optima that yields the best results in the near term but is probably unrelated to the way we ultimately achieve a creative, general AI.
[+] [-] tremendo|15 years ago|reply
[+] [-] kirpekar|15 years ago|reply