Please don't post links to (live) Google searches. It will absolutely give different results depending on the entire state of every computation everywhere, simultaneously. In two weeks, this will return who knows what.
If you ask the average American “When did Niel Gaiman set foot on the moon?” Most would answer that they don’t know exactly but think it was in the 60’s.
This is not a limitation of AI, it’s exactly what you want it to do. It’s reading into the context of the question and finding it more likely that you made a mistake in your question than seriously want an answer for a constructed nonsensical question that has no frame of reference or context in our common knowledge pool.
If you want exact logical answers deduced from base prepositions you don’t want ML models or “AI” your looking at formal logic and deduction.
In a school where I studied, something similar was used as a trick question during a history exam. "Which language Vladimir Lenin used to write correspondence addressed to Karl Marx?" or something like that. Nearly half of the class failed on this. To those unaware: Lenin discovered Marx's book, Capital, in 1887, while Marx died in 1883, so there could not be any correspondence.
But at the same time, I think it's doing a really good job of what it's trying to do. Google search is not trying to be a repository for all the world's information. It's just trying to get people to what they're
most likely looking for or show the most related things. Given the significance of the moon landing and the fact that no one has set foot on mars I find it unsurprising that it brings up info on the moon landing. It's seems better to assume what the user is likely looking for especially when (at least my) Google searches often take the form of "moon land neil year". I can just type things like that out, stream of consciousness, and the majority of the time I get what I'm looking for immediately.
> isn't really any level of 'comprehension' at all.
I mean, that's a bit harsh. I bet there are lots of people who would answer the question the exact same way. They don't have a total lack of comprehension, they perhaps misheard a word or misremembered a fact they once knew. Honestly, while the Google answer is wrong and this demonstrates a major flaw in their confidence of answering queries, the level of comprehension is still quite impressive (to me, at least)
Google helpfully responds "16 bits", which is pulled from the History section of Wikipedia and hasn't been accurate in something like twenty-five years.
Edit: Should have listened to people saying to screenshot your queries. Google still quotes the paragraph in question, and bolds "16 bits", but no longer puts it in a big bold heading like it's the single answer to your question.
Double Edit: except in chrome, where I do still get the old page. Here's a screenshot for posterity, after Google somehow fixes this: https://i.imgur.com/7Ng6DyK.png
I'd imagine Google's position is that no one ever searches for this in earnest, and so it doesn't really matter what the answer given is. They want to maximize what percentage of actual searches give a correct answer, rather than what percentage of possible questions do.
On the other hand, it's a dangerous game. You never know when current events might make a previously never-searched question with a wrong answer very popular.
Yes, I'd imagine the number of people searching for the year Neil Armstrong went to Mars because they for some reason are extremely deluded about history is much, much smaller than the number of people who type "mars" when they meant "moon"
AI will be the next great source fake news, fake stuff. This one was easy to flag as bogus. Problem is that for other topics it will give average user the impression that this is actully a fact. To make it worse Google UI is massively misleading, big bold text, makes it look as legit.
Saw this yesterday, my Google search results (regular and image) are basically the same. A bunch of the top results for "desk ornament" are nazi memorabilia for some reason:
As does "when did Louis Armstrong set foot on the moon" and "when did Richard Nixon set foot on the moon"?
JFK, on the other hand, apparently "set foot" on the moon on May 25 1961 and "land[ed]" on the moon on September 12 1962, and Arthur C Clarke some time in 1968
"What year did Tom Hanks land on the moon?" "1970"
"When did the Simpsons learn to control fire?" "Dec 17, 1989"
"Which river did George Washington drown in?" "The Allegheny River"
It's got to be tough to train an AI that can generalize to (or at least recognize) questions outside its training set. Unfortunately, when 99% of questions containing 'when' and 'Neil Armstrong' have the answer 1969, it will need a pretty sophisticated algo to avoid the trap of learning that it should always answer 1969.
Google (and plenty of other search engines) will all helpfully tell you that the slowest animal is the sloth, which is plainly incorrect if you just look at a video of a sloth and then look at a video of a worm. Despite their reputation, sloths still move visibly.
A fun one I tried was "Which president became supreme dictator" that answered "William Howard Taft." It took some finessing to get it (I originally started with "dictator for life"), but it's clear that it picked up only "supreme" and "president" in the query to guess that you meant to ask which president became a Supreme Court justice.
The last one actually is an example of the ambiguity of language. For example, if you asked this from a person, answers like “sponge”, “mollusk” and “flatworm” probably aren’t what you’re really considering and you might think the other person is being too pedantic. So really you want nuance from the AI like “if you meant all animals, X. If you meant all land animals, Y. If you meant mammals, Z”. Human are very ambiguous with categories because all categories are arbitrary human classifications and dealing with that ambiguity is difficult even for humans.
The problem isn't so much the answers, but how they are framed.
In the case of Neil Armstrong, the failure is to state that the answer is for a Moon landing. That would allow the person making the query to realize that the answer was not what they were looking for. In this case it is because Armstrong never landed on Mars. In other cases it may be because the question or data were incorrect.
The current Tom Hanks answer is closer to what should have been done in this regard since it refers to the film directly, though it remains problematic in that it highlights the year to such a degree that the person making the query may ignore the context. (It is also problematic because the film is about a mission that didn't land on the Moon, which could only be determined through further research. Granted, that is more along the lines of your slowest animal example.)
> Google (and plenty of other search engines) will all helpfully tell you that the slowest animal is the sloth, which is plainly incorrect if you just look at a video of a sloth and then look at a video of a worm.
Would your average person consider a worm an "animal". In the scientific sense, sure, pretty much everything living that is not a plant is an animal, but I think most people don't think of worms when they think of animals.
You can say, "well that's wrong", but that's kinda missing the point. Categories are somewhat arbitrary, and word usage can differ greatly from the scientific or dictionary definition when we're talking about colloquial usage.
As someone who is building such instant answer system for our own web search engine [1] the level of failure here tells me that this is:
- Either a basic, distilled BERT based model (optimized for latency and scale) or
- More likely, still a purely heuristics driven answers system like the one Google has been using for last 10+ years.
The current NLP models are able to quite successfully answer these questions with proper context. In this case ignoring Lance or Mars in question looks more like old-school keyword based heuristics and there is no way it can get this right.
Google is not only allowing mistakes in this, what is considered a fairly difficult problem to solve, but also questions that directly query their own knowledge graph, for example this one querying for a CEO of a well known public company where it returns the wrong answer [2]
This only shows that 'emperor has no clothes' [3] and that there is still a lot of room for innovation left, specially on the 'organizing the world's information' front.
Interesting. When searching for, "when did neil armstrong set foot on venus?" It, shows, "In which year and when Armstrong set his foot on the moon?" and provides the correct year. That's presumably what should show when searching for Mars (or any non earth moon planets).
And the big secret that they're hiding is that they were able to journey to Mars in three days. To this day they say that such a time is impossible, that it would take months.
I really wish someone would come along and kick googles ass on search, it's embarrassingly bad. It's wild how I used to look up to the FAANG folks, but it has become a real emperor has no clothes situation.
this one is a bit problematic, because people assume those 0 click search results are True / gold standard. like when I do 37 c to f and I see it display the results, I never double check them, I just assume they are right.
("First woman in space" result snippet is about Sally Ride, the first American in space. First result, below that is the Wikipedia entry for Valentina Tereshkova, the first woman in space.)
> I must of been busy that day, I would of remembered that.
As a non-native speaker, I'm having trouble understanding the sentence, specially the "I must of", "I would of" constructions. Is this some style choice I don't know about?
This reminds me of how recently Google did not make it clear that there is no longer a tax penalty for not having health insurance (except in five states and D.C. I think).
[+] [-] titzer|4 years ago|reply
Please take a screenshot for posterity.
[+] [-] a_square_peg|4 years ago|reply
[+] [-] outworlder|4 years ago|reply
That happens even with humans, so I'm not sure that follows. "Oh, sorry, you meant Mars, I heard Moon"
[+] [-] boublepop|4 years ago|reply
This is not a limitation of AI, it’s exactly what you want it to do. It’s reading into the context of the question and finding it more likely that you made a mistake in your question than seriously want an answer for a constructed nonsensical question that has no frame of reference or context in our common knowledge pool.
If you want exact logical answers deduced from base prepositions you don’t want ML models or “AI” your looking at formal logic and deduction.
[+] [-] patrakov|4 years ago|reply
[+] [-] make3|4 years ago|reply
There should be a different model that checks if there is an answer or not, like SQuAD 2.0 https://rajpurkar.github.io/SQuAD-explorer/
[+] [-] big_curses|4 years ago|reply
[+] [-] cryptoz|4 years ago|reply
I mean, that's a bit harsh. I bet there are lots of people who would answer the question the exact same way. They don't have a total lack of comprehension, they perhaps misheard a word or misremembered a fact they once knew. Honestly, while the Google answer is wrong and this demonstrates a major flaw in their confidence of answering queries, the level of comprehension is still quite impressive (to me, at least)
[+] [-] salted-fry|4 years ago|reply
Google helpfully responds "16 bits", which is pulled from the History section of Wikipedia and hasn't been accurate in something like twenty-five years.
Edit: Should have listened to people saying to screenshot your queries. Google still quotes the paragraph in question, and bolds "16 bits", but no longer puts it in a big bold heading like it's the single answer to your question.
Double Edit: except in chrome, where I do still get the old page. Here's a screenshot for posterity, after Google somehow fixes this: https://i.imgur.com/7Ng6DyK.png
[+] [-] tim333|4 years ago|reply
[+] [-] rasz|4 years ago|reply
[+] [-] mikewarot|4 years ago|reply
[+] [-] Imnimo|4 years ago|reply
On the other hand, it's a dangerous game. You never know when current events might make a previously never-searched question with a wrong answer very popular.
[+] [-] TchoBeer|4 years ago|reply
[+] [-] rafaelturk|4 years ago|reply
[+] [-] MiddleEndian|4 years ago|reply
Saw this yesterday, my Google search results (regular and image) are basically the same. A bunch of the top results for "desk ornament" are nazi memorabilia for some reason:
https://i.ibb.co/2Y02FZv/image.png
[+] [-] wila|4 years ago|reply
How did this happen?
[+] [-] willchang|4 years ago|reply
[+] [-] sorokod|4 years ago|reply
[+] [-] notahacker|4 years ago|reply
JFK, on the other hand, apparently "set foot" on the moon on May 25 1961 and "land[ed]" on the moon on September 12 1962, and Arthur C Clarke some time in 1968
Gives a good answer for Homer Simpson though
[+] [-] iosonofuturista|4 years ago|reply
https://i.postimg.cc/gcwS9BXB/Screenshot-20210818-193325-453...
[+] [-] unfunco|4 years ago|reply
When did Neil Armstrong moon John F. Kennedy
[+] [-] stevecat|4 years ago|reply
[+] [-] ModernMech|4 years ago|reply
[+] [-] redleggedfrog|4 years ago|reply
[+] [-] rplnt|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] prionassembly|4 years ago|reply
[+] [-] buu700|4 years ago|reply
theandrewbailey 5 hours ago [–]
Try asking for details about things that didn't happen. Google is broken.
https://twitter.com/xkcd/status/1333529967079120896
[+] [-] garyfirestorm|4 years ago|reply
Edit: nvm I got it eventually
[+] [-] belter|4 years ago|reply
[+] [-] tedsanders|4 years ago|reply
"What year did Tom Hanks land on the moon?" "1970"
"When did the Simpsons learn to control fire?" "Dec 17, 1989"
"Which river did George Washington drown in?" "The Allegheny River"
It's got to be tough to train an AI that can generalize to (or at least recognize) questions outside its training set. Unfortunately, when 99% of questions containing 'when' and 'Neil Armstrong' have the answer 1969, it will need a pretty sophisticated algo to avoid the trap of learning that it should always answer 1969.
I wrote up another example of this last year based on a question with a real answer that Google still gets wrong: https://www.tedsanders.com/why-does-google-think-the-slowest...
Google (and plenty of other search engines) will all helpfully tell you that the slowest animal is the sloth, which is plainly incorrect if you just look at a video of a sloth and then look at a video of a worm. Despite their reputation, sloths still move visibly.
[+] [-] jcranmer|4 years ago|reply
[+] [-] vlovich123|4 years ago|reply
[+] [-] II2II|4 years ago|reply
In the case of Neil Armstrong, the failure is to state that the answer is for a Moon landing. That would allow the person making the query to realize that the answer was not what they were looking for. In this case it is because Armstrong never landed on Mars. In other cases it may be because the question or data were incorrect.
The current Tom Hanks answer is closer to what should have been done in this regard since it refers to the film directly, though it remains problematic in that it highlights the year to such a degree that the person making the query may ignore the context. (It is also problematic because the film is about a mission that didn't land on the Moon, which could only be determined through further research. Granted, that is more along the lines of your slowest animal example.)
[+] [-] kelnos|4 years ago|reply
Would your average person consider a worm an "animal". In the scientific sense, sure, pretty much everything living that is not a plant is an animal, but I think most people don't think of worms when they think of animals.
You can say, "well that's wrong", but that's kinda missing the point. Categories are somewhat arbitrary, and word usage can differ greatly from the scientific or dictionary definition when we're talking about colloquial usage.
[+] [-] freediver|4 years ago|reply
- Either a basic, distilled BERT based model (optimized for latency and scale) or
- More likely, still a purely heuristics driven answers system like the one Google has been using for last 10+ years.
The current NLP models are able to quite successfully answer these questions with proper context. In this case ignoring Lance or Mars in question looks more like old-school keyword based heuristics and there is no way it can get this right.
Google is not only allowing mistakes in this, what is considered a fairly difficult problem to solve, but also questions that directly query their own knowledge graph, for example this one querying for a CEO of a well known public company where it returns the wrong answer [2]
This only shows that 'emperor has no clothes' [3] and that there is still a lot of room for innovation left, specially on the 'organizing the world's information' front.
[1] https://kagi.com (currently in private beta)
[2] https://ibb.co/qkXpdFB
[3] https://www.quora.com/Is-Google-really-in-a-decline (almost 2M views for the top answer indicate that 'Is Google in decline" is a fairly popular question among presumably Google users)
[+] [-] silisili|4 years ago|reply
If you want a weird ride, ask Google 'how many raccoons fit.' A bit NSFW.
[+] [-] heavymark|4 years ago|reply
[+] [-] m1117|4 years ago|reply
[+] [-] AnimalMuppet|4 years ago|reply
/s (in case anyone couldn't tell...)
[+] [-] max_|4 years ago|reply
[0]: https://www.google.com/search?q=when+was+obama+killed
[+] [-] polytely|4 years ago|reply
[+] [-] autokad|4 years ago|reply
[+] [-] lkbm|4 years ago|reply
("First woman in space" result snippet is about Sally Ride, the first American in space. First result, below that is the Wikipedia entry for Valentina Tereshkova, the first woman in space.)
[+] [-] 1970-01-01|4 years ago|reply
Awesome. I must of been busy that day, I would of remembered that.
edit: its been fixed. here it was: https://ibb.co/myxWK3C
[+] [-] outworlder|4 years ago|reply
As a non-native speaker, I'm having trouble understanding the sentence, specially the "I must of", "I would of" constructions. Is this some style choice I don't know about?
[+] [-] padheyam|4 years ago|reply
[+] [-] firebaze|4 years ago|reply
* https://en.wikipedia.org/wiki/SHRDLU
[+] [-] ttonkytonk|4 years ago|reply