I think this field is suffering from some confusion of terminology. In my mind there are three subfields that are crystallizing that each have different goals and thus different methods.
The first one is Data Science. More and more businesses store their data electronically. Data Scientists aim to analyze this data to derive insights from it. Machine Learning is one of the tools in their tool belt, however often they prefer models that are understandable and not a black box. Sometimes they prefer statistics because it tells you if your insights are significant.
The second one is Machine Learning Engineering. ML Engineers are Software Engineers that use Machine Learning to build products. They might work on spam detection, recommendation engines or news feeds. They care about building products that scale and are reliable. They will run A/B tests to see how metrics are impacted. They might use Deep Learning, but they will weight the pros and cons against other methods.
Then there are AI Researchers. Their goal is to push the boundaries of what computers can do. They might work on letting computers recognize images, understand speech and translate languages. Their method of choice is often Deep Learning because it has unlocked a lot of new applications.
I feel like this post is essentially someone from the first group criticizing the last group, saying their methods are not applicable to him. That is expected.
> I feel like this post is essentially someone from the first group criticizing the last group, saying their methods are not applicable to him.
Probably more accurate to say that it's the first group criticizing others in the first group who try to act like people in the third group. Data scientists who use deep learning for everything, when a more interpretable model would do just as well.
And the criticism you note isn't one-directional in the field at large. I'm finding that ML/AI researchers deriding ML/Data engineers and "scientists" as not doing "real" ML or AI is becoming a thing, similar to how some computer scientists deride engineering as not doing real computing.
Not a bad way to put it. You could make his argument valid though by discussing the economic value of the problems solved by the first category of people vs. the third. Right now I agree with him that the first is bigger than the third. But I believe that balance is starting to tip and the potential value of the third will keep increasing relatively to the first to the point of dwarfing it (hence no bust).
I understand that most people working with deep learning wouldn't want this type of thinking to spread amongst the public, and I surely don't want it either.
But you have to be totally unaware of reality to think that DL is the definitive tool for AI.
Most impressive results in DL in the past 2 years happended like this:
>deepmind steals people from the top ML research teams in univerisites around the world
>these people are given an incredible amount of money to solve an incredibly complex task
>a 6000 layers deep network is run for 6 months on a GPU cluster the size of Texas
>Google drops in their marketing team
>media says Google solved the AI problem
>repeat every 6 months to keep the company hot and keep the people flow constant
>get accepted at every conference on earth because you're deepmind (seriously, have you seen the crap that they get to present at NIPS and ICML? The ddqn paper is literally a single line modification to another paper's algorithm, while we plebeians have to struggle like hell to get the originality points)
I'll be impressed when they solve Pacman on a Raspberry Pi, otherwise they are simply grownups playing with very expensive toys.
Deep learning is cool, I truly believe that, and I love working with neural networks, but anyone with a base knowledge of ML knows better than to praise it as the saviour of AI research.
Rant over, I'm gonna go check how my autoencoder is learning now ;)
When I was at Watson this is the first thing I told every customer: before you start with AI are you already doing the more mundane data science on your structured data? If not, you shouldn't go right away for the shiny object.
This said I still believe the article is mistaken in its evaluation of potential impact (and its fuzzy metaphore of pipes). Unstructured or semi-structured or dirty data is much more prevalent than cleaned structured data on which you can do simple regression to get insight.
Ultimately the class of problems solved by more advanced AI will be incommensurably bigger than the class of problems solved by simple machine learning. I could make a big laundry list but just start thinking of anything that involves images, sound, or text (ie most form of human communication).
And before you do mundane data science on your structured data, you should figure out if there is a better way to get cleaner raw data, more data, as well as more accurate data.
For example, I predict stereo vision algorithms will die out soon, including deep-learning-assisted stereo vision. It's useful for now but not something to build a business around. Better time-of-flight depth cameras will be here soon enough. It's just basic physics. I worked on one for my PhD research. You can get pretty clean depth data with some basic statistics and no AI algorithm wizardry. We're just waiting for someone to take it to a fab, build driver electronics, and commercialize it.
So the basic criticism here is than Hanson, in suggesting firms clean up their data and then apply simple analytics to it, is defining away the problem that ML solves?
Most firms that think they want advanced AI/ML really just need linear regression
That's how AI always looks in the rearview mirror. Like a trivial part of today's furniture. Pointing a phone at a random person on the street and getting their identity is already in the realm of "just machine learning" and my phone recognizing faces is simply "that's how phones work, duh" ordinary. When I first started reading Hacker News a handful of years ago, one of the hot topics was computer vision at the level of industrial applications like assembly lines. Today, my face unlocks the phone in my pocket...and, statistically, yours does not. AI is just what we call the cutting edge.
Open the first edition of Artificial Intelligence: A Modern Approach and there's a fair bit of effort to apply linear regression selectively in order to be computationally feasible. That just linear regression is just linear regression these days because my laptop only has 1.6 teraflops of GPU and that's measley compared to what $20k would buy.
The way in which AI booms go bust is that after a few years everybody accepts that computers can beat humans at checkers. The next boom ends and everybody accepts that computers can beat humans at chess. After this one, it will be Go and when that happens computers will still be better at checkers and chess too.
Robin's post reveals a couple fundamental misunderstandings. While he may be correct that, for now, many small firms should apply linear regression rather than deep learning to their limited datasets, he is wrong in his prediction of an AI bust. If it happens, it will not be for the reasons he cites.
He is skeptical that deep learning and other forms of advanced AI 1) will be applicable to smaller and smaller datasets, and that 2) they will become easier to use.
And yet some great research is being done that will prove him wrong on his first point.
One-shot learning, or learning from a few examples, is a field where we're making rapid progress, which means that in the near future, we'll obtain much higher accuracy on smaller datasets. So the immense performance gains we've seen by applying deep learning to big data will someday extend to smaller data as well.
Secondly, Robin is skeptical that deep learning will be a tool most firms can adopt, given the lack of specialists. For now, that talent is scarce and salaries are high. But this is a problem that job markets know how to fix. The data science academies popping up in San Francisco exist for a reason: to satisfy that demand.
And to go one step further, the history of technology suggests that we find ways to wrap powerful technology in usable packages for less technical people. AI is going to be just one component that fits into a larger data stack, infusing products invisibly until we don't even think about it.
And fwiw, his phrase "deep machine learning" isn't a thing. Nobody says that, because it's redundant. All deep learning is a subset of machine learning.
I'm skeptical of claims about a one-shot learning silver bullet, unless people are talking about something different from how it has been classically presented, .e.g. Patrick Winton's MIT lectures. Yes, you can learn from a few examples, but only because you've imparted your expert knowledge, maintain a large number of heuristics, control the search space effectively, etc. There's a lot of domain-specific work required for each system, so I consider it more an approach of classical AI and not something that figures out everything from the data alone, like deep learning.
But again, maybe people are talking about something different than my above description when they talk about one-shot learning today. Either way, I don't think having to rely on a lot of domain specific knowledge is necessarily a bad thing.
>> One-shot learning, or learning from a few examples, is a field where we're making rapid progress, which means that in the near future, we'll obtain much higher accuracy on smaller datasets.
I'm really not convinced by one-shot learning, or rather I really don't see how it is possible to show that any technique used generalises well to unseen data, when you're supposed to have access to only very little data during development.
Even with very thorough cross-validation, if your development (training, test and validation) set are altogether, say, 0.1 of the unseen data you hope to predict, your validation results are going to be completely meaningless.
I think there are a lot of programmers try playing around with "deep learning" and it doesn't work for them. But they lack the knowledge necessary to make it work, such as calculus, statistics, signal processing theory, ect.
This article matches what I've been seeing anecdotally (especially at smaller tech firms and universities in the Midwest US).
I've been hearing more folks in research and industry express the importance of applying simpler techniques (like linear regression and decision trees) before reaching for the latest state-of-the-art approach.
AGREE (but it's irrelevant to (1), and also a pretty old conjecture.)
5) Subpoint: It's not true that 47 percent of total US employment is at risk .. to computerisation .. perhaps over the next decade or two.
PROBABLY (that this number/timeframe is optimistic means very little. one decade after the Internet many people said it hadn't upended industry as predicted. whether it took 10, 20, or 30 years, the important fact is that the revolution happened.)
It would be interesting to know if those who are agree in the comments agree with the sensational headline or point 1, or the more obvious and less consequential points 2-5.
I think a lot of people are young and don't realize what it was like working 20 years ago. When I started working in 1994, most people didn't even have computers on their desk let alone an email address. The web was brand new (we used things like gopher and archie), and only a handful of people had ever used or seen the internet.
Sure, I suppose it's possible that advances we've seen in AI won't be translate into huge productivity gains, but I would think that extremely unlikely.
another point is that Linear Regression IS Machine/statistical Learning. Sure its been around for more than 100 years before computation, but regression algorithms are learning algorithms.
Arguing for more linear regression to solve a firms problems, is equivalent to arguing for machine learning. Now, if instead he wanted to argue that the vast majority of a businesses prediction problems can be solved by simple algorithms, that is most likely true. but economic impact of this is still a part of the economic impact of machine learning.
After a good look behind the curtain of Deep Learning, I've come to agree with Robin. No, Deep Learning will not fail. But it will fail to live up to its promise to revolutionize AI, and it won't replace statistics or GOFAI in many tasks that require intelligence.
Yes, DL has proven itself to perform (most?) gradient-based tasks better than any other algorithm. It maximizes the value in large data, minimizing error brilliantly. But ask it to address a single feature not present in the zillion images in ImageNet, and it's lost. (E.g. Where is the person in the image looking? To the left? The right? No DN using labels from ImageNet could say.) This is classic AI brittleness.
With all the hoolpa surrounding DL's successes at single task challenges (mostly on images), we've failed to notice that nothing has really changed in AI. The info available from raw data remains as thin as ever. I think soon we'll all see that even ginormous quantities of thinly labeled supervised data can take your AI agent only so far -- a truly useful AI agent will need info that isn't present in all the labeled images on the planet. In the end the agent still needs a rich internal model of the world that it can further enrich with curated data (teaching) to master each new task or transfer the skill to a related domain. And to do that, it needs the ability to infer cause and effect, and explore possible worlds. Without that, any big-data-trained AI will always remain a one trick pony.
Alas, Deep Learning (alone) can't fill that void. The relevant information and inferential capability needed to apply it to solve new problems and variations on them -- these skills just aren't present in the nets or the big data available to train them to high levels of broad competence. To create a mind capable of performing multiple diverse tasks, like the kinds a robot needs in order to repair a broken toaster, I think we'll all soon realize that DL has not replaced GOFAI at all. A truly useful intelligent agent still must learn hierarchies of concepts and use logic, if it's to do more than play board games.
> Good CS expert says: Most firms that think they want advanced AI/ML really just need linear regression on cleaned-up data.
Cleaning up data is very expensive. And without that, the analysis is good for nothing. AI helps provide good analysis without having to cleaning up data manually. I don't see how that is going away.
There is a never ending confusion caused by the term "AI" to begin with. Term coined by John McCarthy to raise money in the 60's is really good at driving imagination, yet at the same time causes hype and over-expecations.
This field is notorious for its hype-bust cycles and I don't see any reason why this time would be different. There are obviously applications and advancements no doubt about it, but the question is do those justify the level of excitement, and the answer is probably "no".
When people hear AI they inevitably think "sentient robots". This will likely not happen within the next 2-3 hype cycles and certainly not in this one.
Check out this blog for a hype-free, reasonable evaluation of the current AI:
Thanks. This is why I love HN for finds like this, after poking around a bit this site looks like a really good blog that doesn't hype AI nor deny it which is congruent with my views.
The more I get into machine learning and deep learning it seems like there is an incredible amount of configuration to get some decent results. Cleaning and storing the data takes a long time. And then you need to figure out exactly what you want to predict. If you predict some feature with any sort of error in your process the entire results will be flawed.
There are a few very nice applications of the AI techniques, however most data sets don't fit well with machine learning. What you see is that in tutorials use the Iris data set so much because it breaks into categories very easily. In the real world, most things are in a maybe state rather than yes/no.
Here's why the pipes metaphor is a bad one: we already are doing everything we can and ever will do with pipes. Pipes have been around for a really long time, we know what they are capable of, we've explored all of their uses.
OTOH, the current progress in AI has enabled us to do things we couldn't do before and is pointing towards totally new applications. It's not about making existing functionality cheaper, or incrementally improving results in existing areas, it's about doing things that have been heretofore impossible.
I agree that deep nets are overkill for lots of data analysis problems, but the AI boom is not about existing data analysis problems.
> we already are doing everything we can and ever will do with pipes.
I'm pretty sure industrial engineers would disagree here. The romans certainly didn't clean their lead pipes with plasma and neither did they coat them with some fancy nano materials to reduce stickyness.
If there is a curse of our industry, it is almost willful ignorance of just how hard the physical engineering fields are.
The simple things with pipes are simple. Yes. However, to think we haven't made advances, or have no more to make, is borderline insulting to mechanical engineers and plumbers.
Ironically, deep learning will likely help lead to some of those advances.
> I agree that deep nets are overkill for lots of data analysis problems, but the AI boom is not about existing data analysis problems.
This hits the nail on the head for me. The author's observations in the first 9/10 of the article could all be perfectly valid, but the conclusions he draws I. The last two don't follow for exactly this reason.
The interaction of a system of many small pipes to accomplish computation is an active area of research, and new improvements are used in devices pretty routinely. (Really, the behavior of networks of pipes in general is still pretty open, if you want instantaneous details rather than statistical averages.)
Along similar lines, HFLP systems and systems that require laminar flow to be effective are both more recent techniques that come out of a better understanding and engineering of pipes. HFLP upgrades are a current engineering change over very recent and modern high-pressure systems.
Another way the analogy breaks down: Let's think about cars/trucks. Let's say in the 1930's , they're only availble to big business. What do we get ? A few big companies(Walmart/Sears/etc) doing all retail, at significantly lower prices. A big change.
It seems a little odd that the author is focusing on machine learning not being terribly good for prediction from data to counter the "this time is different" argument. The reason this time is different is we are in a period when AI is surpassing human intelligence field by field and that only happens once in the history of the planet. AI is better at chess and go for example, is slowly getting there in driving and will probably surpass general thinking at some point in the future though there's a big question mark as to when.
Journalists and investors only seem to get excited about buzzwords - Maybe that's because they don't actually understand technology.
To say that technology is like an iceberg is a major understatement.
The buzzwords which tech journalists, tech investors and even tech recruiters use to make decisions are shallow and meaningless.
I spoke to a tech recruiter before and he told me that the way recruiters qualify resumes is just by looking for keywords, buzzwords and company names; they don't actually understand what most of the terms mean. This approach is probably good enough for a lot of cases, but it means that you're probably going to miss out on really awesome candidates (who don't use these buzzwords to describe themselves).
The same rule applies to investors. By only evaluating things based on buzzwords; you might miss out on great contenders.
I've read every comment in this thread and its filled mostly with peoples self congratulatory intellectual views. Nobody, not even Robin Hansen himself has given a good, detailed argument as to why the current progress in Machine learning will stop.
I doubt you'll get that, because nobody thinks that progress in machine learning will stop.
An AI winter doesn't mean that progress stops. It means that businesses and the general public become disillusioned by AI's or ML's failure to live up to the popular hype, and stop throwing so much money at it. The hype then dies down. Research continues, though, until enough progress is made that machine learning starts to produce results that excite the public again, and the cycle goes into another hype phase.
I think the core argument against is the same as it was in 1990 (Dreyfus, Heidegger) - basically AI can solve problems in micro-worlds (chess then, Go now; blocks worlds then image annotation now), but it's not clear these micro-worlds can be fused together, unless they are embodied in an entity who lives in the world. The current deep learning & robotics work is exciting and promising, but it's still a long way from an embodied general intelligence. So - progress might hit the same wall - we can build better subsystems , but not the whole system
Have you ever played one of those strategy games with a tech tree? Research A and it lets you research B, C, and D; research C and D and it lets you research E; etc?
That's based on the way discoveries in the real world build on eachother. And in the real world, the research tree seems to be "lumpy".
Think "agricultural revolution", "industrial revolution", etc. Something new comes available, and everyone rushes to pick off all the new low-hanging fruit. Eventually the easiest gains are all taken, and people lose interest and move to other things. And as people keep picking away more slowly at the more difficult/involved things, eventually someone will find something that -- probably combined with some completely different existing knowledge -- opens up another new field. And it repeats.
Right now we're in the "low-hanging fruit" phase of (1) computers that are powerful enough to run neural networks, combined with (2) feedback algorithms that allow networks with lots of layers to learn effectively. Sooner or later the gains will get a bit tougher as we understand the field better, and then research will slow even further as many researchers find something else new and shiny -- and with better returns -- to focus on.
The progress in AI didn't stop during the past AI winters, it just slowed as funding dried up when people realized the AI at the time couldn't possibly live up to the hype.
Those are just the two I had off the top of my head. We apply ML tasks for object/scene classification and they blow away humans. Not only that we're already structuring a GAN for "procedural" 3D model generation - in theory this will decimate the manual 3D reconstruction process.
I started question the credibility of the article when the author mentioned "deep machine learning". Not an expert in ML, but it should be "deep learning" referring to a type of neural network based machine learning technique with deep hidden layers.
A little off topic but I think the VR boom will bust much more sooner than AI.
I can't think of normal people wearing those heavy gears in their normal life. There will be its use cases in specialized applications like education, industry, games but I don't think it will get popular like an iPhone.
AR is still OK since it augments real life but there is a long way before it will become mainstream.
I reckon someone said the same about the first mobile phones when they were attached to a suitcase. Who would want to carry a suitcase around just to make calls. Now, comparing the current VR to the iPhone is a huge leap, the VR technology is still evolving and I think it will surely get lighter.
have you tried it? i own an oculus and every family member i've seen has been shocked and loved it. Obviously the oculus is prohibitively expensive but with the release of Playstation VR i think the mainstream is poised to adopt it. I really believe the next game consoles that come out will simply be vr headsets
I don't think there is a VR boom. There's just early adopters right now. I'd agree with you if you said that a VR boom will never happen... I think VR will stay niche among a core set of gamers.
I have a hard time understanding why even technical people use the term "AI" today. Its use should be limited to sensational media and cheesy sci-fi. It's roughly equivalent to saying "computery thingamabob". I would call a pocket calculator an AI too. Why not? It carries out certain mental tasks better than our brains do.
* The article is correct and the current singularity (as described by Kurzweil) will hit a plateau. No further progress will be made and we'll have machines that are forever dumber than humans.
* The singularity will continue up until SAI. So help them human race if we shackle it with human ideologies and ignorance.
There is no way to tell. AlphaGo immensely surprised me - from my perspective the singularity is happening, but there is no telling just how far it can go. AlphaGo changed my perspective of Kurzweil from a lunatic to someone who might actually have a point.
Where the line is drawn is "goal-less AI," possibly the most important step toward SAI. Currently, all AI is governed by a goal (be it a goal or a fitness function). The recent development regarding Starcraft and ML is ripe for the picking, either the AI wins or not - a fantastic fitness function. The question is, how would we apply it to something like Skyrim: where mere continuation of existence and prosperity are equally as viable goals (as-per the human race). "Getting food" may become a local minimum that obscures any further progress - resulting in monkey agents in the game (assuming the AI optimizes for the food minimum). In a word, what we are really questioning is: sapience.
I'm a big critic of Bitcoin, yet so far I am still wrong. The same principle might apply here. It's simply too early to tell.
[+] [-] ma2rten|9 years ago|reply
The first one is Data Science. More and more businesses store their data electronically. Data Scientists aim to analyze this data to derive insights from it. Machine Learning is one of the tools in their tool belt, however often they prefer models that are understandable and not a black box. Sometimes they prefer statistics because it tells you if your insights are significant.
The second one is Machine Learning Engineering. ML Engineers are Software Engineers that use Machine Learning to build products. They might work on spam detection, recommendation engines or news feeds. They care about building products that scale and are reliable. They will run A/B tests to see how metrics are impacted. They might use Deep Learning, but they will weight the pros and cons against other methods.
Then there are AI Researchers. Their goal is to push the boundaries of what computers can do. They might work on letting computers recognize images, understand speech and translate languages. Their method of choice is often Deep Learning because it has unlocked a lot of new applications.
I feel like this post is essentially someone from the first group criticizing the last group, saying their methods are not applicable to him. That is expected.
[+] [-] kevinchen|9 years ago|reply
Probably more accurate to say that it's the first group criticizing others in the first group who try to act like people in the third group. Data scientists who use deep learning for everything, when a more interpretable model would do just as well.
[+] [-] sidlls|9 years ago|reply
And the criticism you note isn't one-directional in the field at large. I'm finding that ML/AI researchers deriding ML/Data engineers and "scientists" as not doing "real" ML or AI is becoming a thing, similar to how some computer scientists deride engineering as not doing real computing.
[+] [-] ktRolster|9 years ago|reply
[+] [-] pesenti|9 years ago|reply
[+] [-] bogomipz|9 years ago|reply
You mentioned that that Deep learning was a method of choice for AI researched because Deep it has unlocked a lot of new application.
I have a question - is it also a "method of choice" for researchers because its not well understood yet why Deep Learning actually works?
[+] [-] HemanHeartYou|9 years ago|reply
[deleted]
[+] [-] _fh5n|9 years ago|reply
>deepmind steals people from the top ML research teams in univerisites around the world
>these people are given an incredible amount of money to solve an incredibly complex task
>a 6000 layers deep network is run for 6 months on a GPU cluster the size of Texas
>Google drops in their marketing team
>media says Google solved the AI problem
>repeat every 6 months to keep the company hot and keep the people flow constant
>get accepted at every conference on earth because you're deepmind (seriously, have you seen the crap that they get to present at NIPS and ICML? The ddqn paper is literally a single line modification to another paper's algorithm, while we plebeians have to struggle like hell to get the originality points)
I'll be impressed when they solve Pacman on a Raspberry Pi, otherwise they are simply grownups playing with very expensive toys.
Deep learning is cool, I truly believe that, and I love working with neural networks, but anyone with a base knowledge of ML knows better than to praise it as the saviour of AI research.
Rant over, I'm gonna go check how my autoencoder is learning now ;)
[+] [-] pesenti|9 years ago|reply
This said I still believe the article is mistaken in its evaluation of potential impact (and its fuzzy metaphore of pipes). Unstructured or semi-structured or dirty data is much more prevalent than cleaned structured data on which you can do simple regression to get insight.
Ultimately the class of problems solved by more advanced AI will be incommensurably bigger than the class of problems solved by simple machine learning. I could make a big laundry list but just start thinking of anything that involves images, sound, or text (ie most form of human communication).
[+] [-] dheera|9 years ago|reply
For example, I predict stereo vision algorithms will die out soon, including deep-learning-assisted stereo vision. It's useful for now but not something to build a business around. Better time-of-flight depth cameras will be here soon enough. It's just basic physics. I worked on one for my PhD research. You can get pretty clean depth data with some basic statistics and no AI algorithm wizardry. We're just waiting for someone to take it to a fab, build driver electronics, and commercialize it.
[+] [-] tptacek|9 years ago|reply
[+] [-] brudgers|9 years ago|reply
That's how AI always looks in the rearview mirror. Like a trivial part of today's furniture. Pointing a phone at a random person on the street and getting their identity is already in the realm of "just machine learning" and my phone recognizing faces is simply "that's how phones work, duh" ordinary. When I first started reading Hacker News a handful of years ago, one of the hot topics was computer vision at the level of industrial applications like assembly lines. Today, my face unlocks the phone in my pocket...and, statistically, yours does not. AI is just what we call the cutting edge.
Open the first edition of Artificial Intelligence: A Modern Approach and there's a fair bit of effort to apply linear regression selectively in order to be computationally feasible. That just linear regression is just linear regression these days because my laptop only has 1.6 teraflops of GPU and that's measley compared to what $20k would buy.
The way in which AI booms go bust is that after a few years everybody accepts that computers can beat humans at checkers. The next boom ends and everybody accepts that computers can beat humans at chess. After this one, it will be Go and when that happens computers will still be better at checkers and chess too.
[+] [-] vonnik|9 years ago|reply
Robin's post reveals a couple fundamental misunderstandings. While he may be correct that, for now, many small firms should apply linear regression rather than deep learning to their limited datasets, he is wrong in his prediction of an AI bust. If it happens, it will not be for the reasons he cites.
He is skeptical that deep learning and other forms of advanced AI 1) will be applicable to smaller and smaller datasets, and that 2) they will become easier to use.
And yet some great research is being done that will prove him wrong on his first point.
https://arxiv.org/abs/1605.06065 https://arxiv.org/abs/1606.04080
One-shot learning, or learning from a few examples, is a field where we're making rapid progress, which means that in the near future, we'll obtain much higher accuracy on smaller datasets. So the immense performance gains we've seen by applying deep learning to big data will someday extend to smaller data as well.
Secondly, Robin is skeptical that deep learning will be a tool most firms can adopt, given the lack of specialists. For now, that talent is scarce and salaries are high. But this is a problem that job markets know how to fix. The data science academies popping up in San Francisco exist for a reason: to satisfy that demand.
And to go one step further, the history of technology suggests that we find ways to wrap powerful technology in usable packages for less technical people. AI is going to be just one component that fits into a larger data stack, infusing products invisibly until we don't even think about it.
And fwiw, his phrase "deep machine learning" isn't a thing. Nobody says that, because it's redundant. All deep learning is a subset of machine learning.
[+] [-] orthoganol|9 years ago|reply
I'm skeptical of claims about a one-shot learning silver bullet, unless people are talking about something different from how it has been classically presented, .e.g. Patrick Winton's MIT lectures. Yes, you can learn from a few examples, but only because you've imparted your expert knowledge, maintain a large number of heuristics, control the search space effectively, etc. There's a lot of domain-specific work required for each system, so I consider it more an approach of classical AI and not something that figures out everything from the data alone, like deep learning.
But again, maybe people are talking about something different than my above description when they talk about one-shot learning today. Either way, I don't think having to rely on a lot of domain specific knowledge is necessarily a bad thing.
[+] [-] ma2rten|9 years ago|reply
(https://arxiv.org/abs/1507.02672, https://arxiv.org/abs/1511.06430)
I was really impressed by their results, but I haven't seen it being applied successfully to other applications.
[+] [-] YeGoblynQueenne|9 years ago|reply
I'm really not convinced by one-shot learning, or rather I really don't see how it is possible to show that any technique used generalises well to unseen data, when you're supposed to have access to only very little data during development.
Even with very thorough cross-validation, if your development (training, test and validation) set are altogether, say, 0.1 of the unseen data you hope to predict, your validation results are going to be completely meaningless.
[+] [-] jostmey|9 years ago|reply
[+] [-] jeyoor|9 years ago|reply
I've been hearing more folks in research and industry express the importance of applying simpler techniques (like linear regression and decision trees) before reaching for the latest state-of-the-art approach.
See also this response to the author's tweet on the subject: https://twitter.com/anderssandberg/status/803311515717738496
[+] [-] WhitneyLand|9 years ago|reply
Notice now you can cogently disagree with the main idea while agreeing with most of the sub points (paraphrasing below):
1) Most impactful point: The economic impact innovations in AI/machine learning will have over the next ~2 decades are being overestimated.
DISAGREE
2) Subpoint : Overhyped (fashion-induced) tech causes companies to waste time and money.
AGREE (well, yes, but does anyone not know this?)
3) Subpoint: Most firms that want AI/ML really just need linear regression on cleaned-up data.
PROBABLY (but this doesn't prove or even support (1))
4) Subpoint: Obstacles limit applications (though incompetence)
AGREE (but it's irrelevant to (1), and also a pretty old conjecture.)
5) Subpoint: It's not true that 47 percent of total US employment is at risk .. to computerisation .. perhaps over the next decade or two.
PROBABLY (that this number/timeframe is optimistic means very little. one decade after the Internet many people said it hadn't upended industry as predicted. whether it took 10, 20, or 30 years, the important fact is that the revolution happened.)
It would be interesting to know if those who are agree in the comments agree with the sensational headline or point 1, or the more obvious and less consequential points 2-5.
[+] [-] Patrick_Devine|9 years ago|reply
Sure, I suppose it's possible that advances we've seen in AI won't be translate into huge productivity gains, but I would think that extremely unlikely.
[+] [-] jbrambleDC|9 years ago|reply
Arguing for more linear regression to solve a firms problems, is equivalent to arguing for machine learning. Now, if instead he wanted to argue that the vast majority of a businesses prediction problems can be solved by simple algorithms, that is most likely true. but economic impact of this is still a part of the economic impact of machine learning.
[+] [-] randcraw|9 years ago|reply
Yes, DL has proven itself to perform (most?) gradient-based tasks better than any other algorithm. It maximizes the value in large data, minimizing error brilliantly. But ask it to address a single feature not present in the zillion images in ImageNet, and it's lost. (E.g. Where is the person in the image looking? To the left? The right? No DN using labels from ImageNet could say.) This is classic AI brittleness.
With all the hoolpa surrounding DL's successes at single task challenges (mostly on images), we've failed to notice that nothing has really changed in AI. The info available from raw data remains as thin as ever. I think soon we'll all see that even ginormous quantities of thinly labeled supervised data can take your AI agent only so far -- a truly useful AI agent will need info that isn't present in all the labeled images on the planet. In the end the agent still needs a rich internal model of the world that it can further enrich with curated data (teaching) to master each new task or transfer the skill to a related domain. And to do that, it needs the ability to infer cause and effect, and explore possible worlds. Without that, any big-data-trained AI will always remain a one trick pony.
Alas, Deep Learning (alone) can't fill that void. The relevant information and inferential capability needed to apply it to solve new problems and variations on them -- these skills just aren't present in the nets or the big data available to train them to high levels of broad competence. To create a mind capable of performing multiple diverse tasks, like the kinds a robot needs in order to repair a broken toaster, I think we'll all soon realize that DL has not replaced GOFAI at all. A truly useful intelligent agent still must learn hierarchies of concepts and use logic, if it's to do more than play board games.
[+] [-] chime|9 years ago|reply
Cleaning up data is very expensive. And without that, the analysis is good for nothing. AI helps provide good analysis without having to cleaning up data manually. I don't see how that is going away.
[+] [-] felippee|9 years ago|reply
This field is notorious for its hype-bust cycles and I don't see any reason why this time would be different. There are obviously applications and advancements no doubt about it, but the question is do those justify the level of excitement, and the answer is probably "no".
When people hear AI they inevitably think "sentient robots". This will likely not happen within the next 2-3 hype cycles and certainly not in this one.
Check out this blog for a hype-free, reasonable evaluation of the current AI:
http://blog.piekniewski.info/2016/11/17/myths-and-facts-abou...
[+] [-] dmfdmf|9 years ago|reply
[+] [-] rampage101|9 years ago|reply
There are a few very nice applications of the AI techniques, however most data sets don't fit well with machine learning. What you see is that in tutorials use the Iris data set so much because it breaks into categories very easily. In the real world, most things are in a maybe state rather than yes/no.
[+] [-] shmageggy|9 years ago|reply
OTOH, the current progress in AI has enabled us to do things we couldn't do before and is pointing towards totally new applications. It's not about making existing functionality cheaper, or incrementally improving results in existing areas, it's about doing things that have been heretofore impossible.
I agree that deep nets are overkill for lots of data analysis problems, but the AI boom is not about existing data analysis problems.
[+] [-] the8472|9 years ago|reply
I'm pretty sure industrial engineers would disagree here. The romans certainly didn't clean their lead pipes with plasma and neither did they coat them with some fancy nano materials to reduce stickyness.
[+] [-] taeric|9 years ago|reply
The simple things with pipes are simple. Yes. However, to think we haven't made advances, or have no more to make, is borderline insulting to mechanical engineers and plumbers.
Ironically, deep learning will likely help lead to some of those advances.
[+] [-] throwaway729|9 years ago|reply
This hits the nail on the head for me. The author's observations in the first 9/10 of the article could all be perfectly valid, but the conclusions he draws I. The last two don't follow for exactly this reason.
[+] [-] SomeStupidPoint|9 years ago|reply
Along similar lines, HFLP systems and systems that require laminar flow to be effective are both more recent techniques that come out of a better understanding and engineering of pipes. HFLP upgrades are a current engineering change over very recent and modern high-pressure systems.
[+] [-] petra|9 years ago|reply
[+] [-] fooker|9 years ago|reply
How about a space elevator ?
[+] [-] tim333|9 years ago|reply
[+] [-] jondubois|9 years ago|reply
To say that technology is like an iceberg is a major understatement.
The buzzwords which tech journalists, tech investors and even tech recruiters use to make decisions are shallow and meaningless.
I spoke to a tech recruiter before and he told me that the way recruiters qualify resumes is just by looking for keywords, buzzwords and company names; they don't actually understand what most of the terms mean. This approach is probably good enough for a lot of cases, but it means that you're probably going to miss out on really awesome candidates (who don't use these buzzwords to describe themselves).
The same rule applies to investors. By only evaluating things based on buzzwords; you might miss out on great contenders.
[+] [-] RushAndAPush|9 years ago|reply
[+] [-] bunderbunder|9 years ago|reply
An AI winter doesn't mean that progress stops. It means that businesses and the general public become disillusioned by AI's or ML's failure to live up to the popular hype, and stop throwing so much money at it. The hype then dies down. Research continues, though, until enough progress is made that machine learning starts to produce results that excite the public again, and the cycle goes into another hype phase.
[+] [-] jimduk|9 years ago|reply
[+] [-] tbrownaw|9 years ago|reply
Have you ever played one of those strategy games with a tech tree? Research A and it lets you research B, C, and D; research C and D and it lets you research E; etc?
That's based on the way discoveries in the real world build on eachother. And in the real world, the research tree seems to be "lumpy".
Think "agricultural revolution", "industrial revolution", etc. Something new comes available, and everyone rushes to pick off all the new low-hanging fruit. Eventually the easiest gains are all taken, and people lose interest and move to other things. And as people keep picking away more slowly at the more difficult/involved things, eventually someone will find something that -- probably combined with some completely different existing knowledge -- opens up another new field. And it repeats.
Right now we're in the "low-hanging fruit" phase of (1) computers that are powerful enough to run neural networks, combined with (2) feedback algorithms that allow networks with lots of layers to learn effectively. Sooner or later the gains will get a bit tougher as we understand the field better, and then research will slow even further as many researchers find something else new and shiny -- and with better returns -- to focus on.
[+] [-] goatlover|9 years ago|reply
[+] [-] AndrewKemendo|9 years ago|reply
ML companies are already tackling tasks which have major cost implications:
https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...
http://med.stanford.edu/news/all-news/2016/08/computers-trou...
Those are just the two I had off the top of my head. We apply ML tasks for object/scene classification and they blow away humans. Not only that we're already structuring a GAN for "procedural" 3D model generation - in theory this will decimate the manual 3D reconstruction process.
[+] [-] yegle|9 years ago|reply
[+] [-] h43k3r|9 years ago|reply
I can't think of normal people wearing those heavy gears in their normal life. There will be its use cases in specialized applications like education, industry, games but I don't think it will get popular like an iPhone.
AR is still OK since it augments real life but there is a long way before it will become mainstream.
[+] [-] luisramalho|9 years ago|reply
[+] [-] hacker_9|9 years ago|reply
[+] [-] thesimpsons1022|9 years ago|reply
[+] [-] crimsonalucard|9 years ago|reply
[+] [-] Spooky23|9 years ago|reply
[+] [-] euske|9 years ago|reply
[+] [-] zamalek|9 years ago|reply
* The article is correct and the current singularity (as described by Kurzweil) will hit a plateau. No further progress will be made and we'll have machines that are forever dumber than humans.
* The singularity will continue up until SAI. So help them human race if we shackle it with human ideologies and ignorance.
There is no way to tell. AlphaGo immensely surprised me - from my perspective the singularity is happening, but there is no telling just how far it can go. AlphaGo changed my perspective of Kurzweil from a lunatic to someone who might actually have a point.
Where the line is drawn is "goal-less AI," possibly the most important step toward SAI. Currently, all AI is governed by a goal (be it a goal or a fitness function). The recent development regarding Starcraft and ML is ripe for the picking, either the AI wins or not - a fantastic fitness function. The question is, how would we apply it to something like Skyrim: where mere continuation of existence and prosperity are equally as viable goals (as-per the human race). "Getting food" may become a local minimum that obscures any further progress - resulting in monkey agents in the game (assuming the AI optimizes for the food minimum). In a word, what we are really questioning is: sapience.
I'm a big critic of Bitcoin, yet so far I am still wrong. The same principle might apply here. It's simply too early to tell.