The title is clickbait, but the problem with the religious fervor of machine learning/deep learning is that entrepreneurs/VC keep arguing that deep learning is magic and can solve any problem if you just stack enough layers. (see also: https://www.reddit.com/r/ProgrammerHumor/comments/5si1f0/mac...)
Meanwhile, statistical methods for non-image/text data with identifiable features can often work better than neural networks, but they are not as sexy. (Good discussion on HN about this: https://news.ycombinator.com/item?id=13563892)
It's worth noting that ML doesn't just mean neural networks - those just get the most hype. I'm also curious if you could give an example of traditional models (presumably logreg, SVM, decision forest/trees) outperforming DNN on classification tasks. I read your thread and agree with the points, but I'm curious to see if anyone's outlined examples of when DNN fail in particular (not necessarily RNN/CNN).
Aside - I think it's really ironic for people with no technical know how to go on a talk show and say "learn ML or you're toast". First, the inertia of the world economy is much greater than 3 years. Second, it's unrealistic to ask every 7-11 cashier to go learn ML. Third, programming is a complex art and we're surely not 3 years away from the singularity. Most people I know who have some ML knowledge also realize how far we are from a computer program that can read code, read specs, and then decide how to implement or fix existing code. And last, even if all of the above were moot, I think it's more likely that we write a program that successfully helms a Fortune 500 company before we write one that produces or improves on software. The former could be more easily reduced to a game of chess while the latter requires some massive novel research into "creativity". Maybe it's Mark Cuban who should be learning ML!
Yeah, we just came across someone on HN trying to write a testing DSL with NLP. [0] He's apparently gotten funding for this.
To be frank, the tech community has really been leaving a sour taste in my mouth the last few years. Everyone is anxious to bandwagon on these buzzwords that provide little to no benefit or are applicable only to a small segment of the tech population.
In most cases, even a superficial understanding of the problem space should make it obvious that $This_Weeks_Sexy_Solution is a very bad fit. Do you need a server that persists data and is individually addressable? Then why are you using Kubernetes and Docker, which are still struggling to figure out these very basic things? k8s and Docker have very specific uses, but unless you're Google, they're probably not the right fit for your production environment right now.
This phenomenon seemed to hit a critical mass with document databases and single-page apps, and it continues to iterate with every open-source release that comes out of Google or Facebook. Since Google released TensorFlow last year and the compounded hype of [much worse than advertised attempts at] conversational speech recognition in Siri and Alexa, the "machine learning" bandwagon is starting to try to edge into the spotlight, and it sounds like it's already a mandatory part of any VC pitch.
Fortunately, machine learning is pretty hard and you get into hairy math practically right away, so I don't think the legs on this one will last as long. But we'll see. We'll at least have a lot of faux-ML going around and a lot of people making spurious claims on their resumes.
The economic collisions that make Silicon Valley and the tech industry in general a hive for inexperienced and insecure youth are bearing some really interesting effects this way. How can a company that's not "blown about by every wind of [tech fad]" fully exploit its relative sanity for competitive advantage?
For whatever reason, statistics seems to be in the midst of a serious branding problem. Maybe traditional frameworks of inference are too conservative for non-researcher's tastes?
I'm curious what Fisher, Neyman, and Pearson would say about the current state of the field. Especially considering how often they and other statisticians disagreed with each other throughout the 20th century.
I have no experience with the entrepreneur/VC side of this coin, but Data Scientists know the differences between traditional regression based ML and CNNs. We know that doing image recognition or NLP is best suited to nvidia/tensorflow/keras/deeplearning/buzzword/buzzword. We also know that SVMs & Random Forests & regression lines work just as well as they always have to make predictions based on your click-thru data.
Maybe we should explain the cost/benefit of the buzzwords vs. the science?
Sexy is in the eye of the beholder. Seriously, I think the problem of ML is that it often goes against the so-called principle parsimony, which plays a critical role in statistical modelling. Especially in many areas where the whole point isn't strictly to predict future outcomes, but to understand how things work and to test hypotheses. Understanding leads to better prediction anyways. If you can predict something but you don't know why or how, I would take such predictions with a grain of salt.
Just call it "statistical machine learning" or some other vacuous term and people who know enough will respect it (maybe after rolling their eyes a bit), and people who don't will be happy to hear you are using machine learning.
Reading HN I worry that we're going to have the opposite problem - a glut of people will try and (badly?) learn ML and then realize there aren't enough ML jobs.
I've a PhD and held ML-engineer positions in a few different companies - I've good industry awareness.
Most applied ML, for most companies, right now, is actually relatively simple models (hand-coded rules! logistic regression! You'd be shocked how common these are.) The bulk of the work is data cleaning, gathering, integration, deployment, productisation, reliability, avoiding pathological cases, special-casing, Product, UX. You do need ML specialists who understand the stuff, to make it all work and come together - but the ratio of ML specialists to the wider team is low. Maybe 1 or 2 specialist on a team of 10 for an ML heavy product.
This is going to remain the case IMO. Yes, there will be small teams, in highly resourced organizations (GOOG, FB etc), academic research labs, or occasional hard-tech startups, who do new model development. Maybe if AI becomes huge, you'll see more traditional Fortune 500s spin up similar efforts.
But there'll be a much wider set of people&businesses applying and tuning well understood approaches, rather than doing new model development. And you just don't need as many ML specialists, for that approach.
Even with deep learning, the tooling will advance. I mean, even look at all the research papers describing applications at the moment - so many of them are using pre-trained models. Industry will be similar. Tooling will advance, and you'll be able to do increasingly more with off-the-shelf pieces.
I think ML is absolutely going to have a big impact - I buy at least some of the hype. But should all developers, or even a substantial minority of developers, start learning ML as a career imperative? I don't think so.
Finally, it takes serious time to learn this stuff.
Its easy to dabble (and worthwhile doing - its fun; and sometimes you can do powerful things in a using tools in a very blackbox manner!). But actually thoroughly learning it takes time. It takes serious time to build statistical intuition, as just one example.
We could easily end up with a great many career developers who have a specialization in ML, frustrated they never get to use it.
This is not my field, but a serious question--I once read that part of the motivation of pharmaceutical companies in hiring researchers was not so that they would all produce ground breaking independent research, but rather because they would be capable of reading the literature (again, not my field). Is that true at all for machine learning? Would companies hire people who would be up to date with the literature so that they could implement algorithms that others have developed in an academic context and put them into production?
I mean, even look at all the research papers describing applications at the moment - so many of them are using pre-trained models.
Which is probably going to blow up in their collective faces at some point - it's at best a problematic technique. Many of these papers have little or no serious analysis of generalization issues or validation.
I think to the broader point that generally learning how to validate a ML technique is going to be very useful to a lot of people, implementing them less so.
I'm going to go against the grain here and (gasp) not hate on deep learning. People should realize that although many older statistical methods and "traditional" machine learning methods such as LDA, SVMs and decision trees may be good enough for business tasks, they are not the cutting edge of AI research. I think people are forgetting exactly how difficult image classification and object detection was before the advent of deep learning. People hating on "stack more layers" forget that "stack more layers" is EXACTLY what improved imagenet performance to such as massive extent. ResNets pushed the limits by figuring out how to stack more layers in a beneficial way.
And let's take a look at AlphaGo, how would you do that with SVMs or decision trees? Just get over the fact that deep learning provides a level of "intuition" (Go's search space is famously greater than the estimated number of stars in the universe).
I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of course...) and would prefer that their preferred methods would still reign supreme.
I'll end this wall of text on the note that OpenAI's Ian Goodfellow notes that all projects at OpenAI use deep learning right now, but they are not dogmatic and will consider other approaches if they work well. I think this is the path that should be taken. On the other hand, I also see a bright future for uniting traditional techniques with deep learning such as attaching a SVM to a CNN and combining decision trees with CNNs both of which have resulted in good results.
>> I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of course...) and would prefer that their preferred methods would still reign supreme.
I think that's definitely a part of it, and I feel that way sometimes myself (not that I'm a PhD). But there's another side of that reluctance that lies on the axis of model accountability and explicability. A lot of modern ML/Deep Learning doesn't -feel- like we're understanding anything any more than we did ten years ago. Yes our black-box results are better according to the tests we've laid out for them, but there's something more slippery about the 'why', beyond the handwave of 'complexity'. Maybe this is just the way it will be going forward (in the spirit of Quantum's "shut up and calculate"), but it is not easy to give up something that you can wrap your head around with something that kind of just takes care of itself, especially if you're in the business of seeking knowledge instead of results.
You can argue against Cuban's point that anyone who doesn't know machine learning will be "dinosaur in 3 years", while also being enthusiastic about deep learning.
> "traditional" machine learning methods ... may be good enough for business tasks, they are not the cutting edge of AI research
That's true - but so?
Under discussion is whether every developer should drop what they are doing and learn ML/DL, not what the cutting edge of AI research is?
In that context a lot of the low-hanging fruit for the next while is going to involve actually deploying simpler ML, with all the non-ML work that entails - and maybe using off-the-shelf DL to extract information from unstructured data.
> older ML PhDs are angry that deep learning is so easy
That's very ungenerous. People tend to do PhDs in things like ML (often making sacrifices to do so) because they want to advance the field - not to make it harder! People are excited. They are wary about hype causing another AI winter, or a lot of important projects failing for silly reasons.
Agree on the idea of Classical Machine Learning experts being a little pissed regarding the average joe getting better results with such ease. A lot of these people have put in years of study/work in those techniques and it must be sad to see some form of sudden paradigm shift making it worth less.
On the other hand it is true that DL is not required for many (most?) companies as they don't work with the kind of problems you list (vision, robotics)
I would be far more inclined to heed this advice if Cuban have any indication of understanding ML as anything more than a magic black box; fairy dust to be sprinkled into every pitch deck to solve any objection or solve difficult problems. The bandwagon is passing through folks, jump on board with Mark or you will have fewer buzzwords with which to craft your deck...
Sort of a content-free article, but the headline is an interesting bold claim that conjures a lot of thoughts:
- I know enough machine learning to be dangerous, but I'm hardly ever asked to use it. I designed a Bayesian classifier for my own startup around 6 years ago, analyzing political donor networks. I've completed the Stanford ML course. Back in college I did a math minor, so I'm comfortable with linear algebra, calculus, etc. I'm pretty comfortable with statistics of both kinds. But my bread-and-butter is freelance web development . . . and I'm not really even sure how to find work doing more MLy things.
- I've read over and over that the most time-consuming part of ML work is data collection & cleanup, and that matches my own experience. It is the same thing that killed so many data warehouse projects in the 90s. You don't need a Ph.D. to do it, but it is a tough and costly prerequisite. So it seems like you'll need non-ML programmers even for specifically ML projects.
- In a similar vein, Google has written about the challenges of "operationalizing" machine learning projects.[1] Having a little experience collaborating with a team doing an ML project, where they did the ML engine and I did the user-facing application, I can say that many ML experts are not experts in building reliable, production-ready software.
- Will there ever be a Wordpress of machine learning? If there is, the author will be rich, but you won't need a Ph.D. to operate it. But because ML requires hooks into your existing systems, I don't know if this will ever happen. What will happen I think is plugins to existing e-commerce systems for product recommendation or other off-the-shelf ML-powered features. These already exist, but I assume they will become more prevalent and powerful over time. In any case, the mainstreaming of ML for business will be inversely correlated with the expense to implement it, which suggests it will be easier and easier for non-expert developers to use (and misuse).
EDIT: Added the (now-)third bullet point I forgot before.
One of the things I've realised is that the more I'm looking around, the more I find opportunities where people "should" have seen the opportunities of basic bayesian models, simple clustering algorithms etc. and other simple mathematical/statistical methods 20 years, but didn't, and still don't. That has massively changed my perspective on how quickly the onslaught of machine learning will come.
E.g. when I was reading up on genetic algorithms etc. 20 years ago we also expected the "revolution" to be right around the corner, and that things like genetic programming would change the world in a few years time. And while various of those methods found use some places, most places that could have used at least some of the simpler ones, still don't.
In other words, I think talking about a 3 year timeline is crazy. It's getting more attention, sure, but there is so much low-hanging fruit that most developers could be busy for the next 20 years putting in place the most trivial algoriths all over the place and we still wouldn't have picked off even the low hanging fruit where the computational resources and algorithms and data to make a big impact were well within reach 20 years ago.
This certainly means there is plenty of room for a lot developers to do very cool stuff and build careers on machine learning today, but it also mean most developers will not have to learn the state of the art - or anything near it - for a very long time.
As a concrete example I give to people, consider all of the search boxes out there on various sites - product searches, location searches, site searches - that are straight keyword based searches that don't take into account any clickstream data to improve ranking. The proportion of search boxes I see that take advantage of the available data is vanishingly small, even though very basic analysis can improve the perceived relevance of the results massively.
We certainly will see more companies invest in proper machine learning as the payoff gets higher and difficulty in taking advantage of it drops. But we will also see a huge proportion of sites that could use it continue to ignore it for years to come.
There are big business opportunities in finding ways of making a dent in that portion of the market, though, and so learning this stuff can certainly be well worth it on a personal level, but I don't believe in his timeline in terms of the overall market.
Statements like these suggest that another AI Winter is coming (sooner than 3 years I guess)
"""He thinks even programming is vulnerable to being automated and reducing the number of available programming jobs."""
I believed something similar could happen within 1-2 years of learning/writing AI programs (more than 12 years back). I believed it so much that it consumed most of my weekends as I took on the Genetic Programming approach. Yes! computers can write programs - BUT trying reading those. Eventually after spending hours or days, you will be able to read those programs and you might find a simple "hello-world" program represented by a complex mathematical equation. Good luck trying to get such program fixed by humans. Imagine an experience decoding deep-learning-neural-nets.
However, that is black-box from a programmer perspective.
From a business/management personnel perspective - the code is a black box anyways. When they get NNs that can generate required software, they will replace the people-manager with a NN-manager (who is a programmer btw!)
> Statements like these suggest that another AI Winter is coming (sooner than 3 years I guess)
I wouldn't bet against it. It's very risky to extrapolate progress in AI.
A DARPA project named CHI [1] succeeded in automatically producing programs from a high-level knowledge description language. And when they fed it its own description, CHI rewrote itself, with a better implementation. Do you fear for your programming job yet? Well, that was 36 years ago.
That's as strong as a statement as Trump: Learn Machine Learning or you'll be a Dinosaur in 3 years...
Maybe if it was coming from Bill Gates, Mark Zuckerberg, or another tech titan with some actual coding experience and a deeper level of learning about what ML even is. Cuban's a business man, and most CEO's I know don't have a clue about the stacks that run their own company, let alone what's popular.
That said, I do think ML will be important, but I develop ecommerce apps and things of the such in Laravel, unless I move into AI and Neural nets I don't see needing to know a lot about ML (though I wouldn't mind moving in that direction as that space picks up) -- but there's still plenty of opportunities without it.
Assuming we're all going to be deep learning programmers is quite a bit foolhardy. I think what's really relevant to consider is AI winters can and do happen[1]. I would not disagree deep learning has done some amazing things, what I would say is it does have limitations.
What causes AI winters is when an advance such as deep learning can be applied to new problems and leads to increased interest. And while this new thing is really good as a subset of problems and impresses the public, of course it can't displace humans at everything and naturally has it's limitations.
So funding pours in, everyone gets hyped, and then those natural limits are (re)discovered and everyone gets all anti-AI research. Of course many people knew the limitations all along, but the dream is gone and so is a lot of funding until the next thing comes along.
This is probably natural to a lot of fields but AI just seems more prone to these boom and bust cycles because it's really exciting stuff.
Exactly. 25 years later, I'm still waiting to be replaced by an Expert System. A lot of the tech that fueled the hype train (logic programming! genetic algorithms!) is still really neat, but it didn't work out the way we expected. Same thing will happen with ML. It's the Prolog of tomorrow.
Good luck with that. Any place doing serious ML will require the person to have a PhD or have publications and presentations at conferences like NIPS/ICML. Even most CS grads with a bachelors do not have the math background required unless they double majored in math or stats.
This is more VC/founders who are hyping up AI and need more ML folks so they can drive down costs.
The notion that you can only do "serious ML" if you have a PhD/published papers is a No True Scotsman fallacy.
The modern tools for ML/deep learning are accessible to all open source and well documented. And as I note in my top-level comment, old-fashioned statistical methods like linear regression are more than sufficient for real-world business problems, and definitely do not require a PhD to grok.
I didn't double major in math or stats. I squeaked through linear algebra without understanding the material, and then re-studied it independently a few years ago and actually understood a little more.
Yet, I can still apply machine learning to solve Ax=b problems. More importantly, I can use business analysis and write code to transform business problems into an Ax=b problem, and then optimize it.
You don't need a PhD to grok optimizing a vector to transform a matrix of inputs into a vector of observed outputs, then apply that trained vector going forward. Neural nets are slightly more complicated than a straight linear regression, but only slightly. I'd call decision tree methods like GBM even more complex, but still eminently grokkable for a decent programmer.
> Any place doing serious ML will require the person to have a PhD or have publications and presentations at conferences like NIPS/ICML.
Stop spouting this bullshit. You don't need a PhD, and you don't need to advance the field to be doing 'serious ML'. All you need to be able to do is know how and when to apply it to solve crucial business problems.
This misunderstanding is what needs to go the way of the dinosaur. You don't need a PhD to know how to clean data, prepare features, or generate synthetic features. You don't need a PhD to tune hyperparameters. You don't need a PhD to know how to separate your test data from your training data. You don't even need a PhD to wire things together at a high level with something like tensorflow.
Obviously you will want a theoretical expert on your team. But if your startup is counting on having a room full of them, good luck.
Good thing the vast majority of businesses won't need "serious" ML, but instead will require only simple implementations to help solve business problems.
Utter nonsense. A PhD signals two things: that a person has the same degree of mastery of core material as a person with a master's degree and that he or she has the determination to do additional original research sufficient to produce a 100 page paper.
It isn't required for any serious research effort, except by the accident of inertia. And it certainly isn't a necessary indicator of determination.
Honest question : once the technics will settle a bit, and libraries are created, what will be needed, apart from knowing that machine learning algorithm are based on some kind of stasticial inference, with a few settings here and there ?
I mean, we don't need a phd in image compression to create a service that streams videos. We just use libraries. Same for everything in computer science, it always end up packaged in some kind of reusable code or service, and only some specialists remain working in the field to work on marginal improvements.
Once you've plugged your data into an ML system and gives you a classifier, are you done? What if the results are not good enough, do you just move on to another problem?
If you don't just move on to another problem you will need people who know how to push these systems further.
Unlike video streaming libraries, there's no certain way for people to verify the outcome of an ML function is correct (e.g. Would this model work on other datasets? Is the statistical inference correct?). You need at least some basic knowledge about the underlying math to avoid pitfalls.
(Pre-apology. I am not trolling, please don't get my comment below more than what I intended: another perspective to look at the strong reaction Cuban's comments incited.)
Reading through the comments, I see that Cuban's statement upset and even angered several HN commenters. That is a strong emotional reaction.
I am not saying it is the 5 stages of grief, but the first 3 fits: denial, anger, bargaining, depression and acceptance.
Also from Howard Aiken: Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats.
I 'm not even sure why it provokes such reaction. Neural networks have existed for 4 decades ; without the deep- part, but we know their potential power and we still have not been replaced. Granted, this time it's different. I think what worries most of us is that neural nets need lots of data and we don't have access to it. Still, they are easy to learn and we should be learning about them (note to self).
I think prosthetic knowledge will become deeper and more accurate in the long run. And if it scales we don't need many people building the general purpose AI.
With this prosthetic knowledge we will have to learn much more what to ask and know how much the machine knows.
One has to quickly grasp the abstract that is one level too high or detailed for the machine to find and than find the seperate answers of the level below to recombine them. You can't yet ask where to open a restaurant, but you can google for demographics and write a program to map ratios between income, foot-traffic density and restaurant density.
Once we can ask what and where is the most profitable business in to start in town, we probably still don't get a step by step guide how to do it, interior design included. Where the rubber meets the road there is still a lot of oppertunities to decide on and complex data we can grasp more easy than the machine.
Just like there isn't such a thing as polymaths anymore because the world has so many specialized skills, I think there won't be the possibilities of things like "full stack developer" in the near future because of the complexities for development. Some things that contribute to the complexities will be: ubiquitous controls, advanced AI, internet of things, augmented reality, machine learning, and new technologies we don't even have yet. We are in the golden age now where a single person can sometimes create a better website or app compared to a whole development team at a fortune 500 company. I think our Internet world will become so complex in the very near future that a single person simply won't be able be capable and they'll have to become specialized in only a portion of it.
[+] [-] minimaxir|9 years ago|reply
Meanwhile, statistical methods for non-image/text data with identifiable features can often work better than neural networks, but they are not as sexy. (Good discussion on HN about this: https://news.ycombinator.com/item?id=13563892)
[+] [-] hasbroslasher|9 years ago|reply
Aside - I think it's really ironic for people with no technical know how to go on a talk show and say "learn ML or you're toast". First, the inertia of the world economy is much greater than 3 years. Second, it's unrealistic to ask every 7-11 cashier to go learn ML. Third, programming is a complex art and we're surely not 3 years away from the singularity. Most people I know who have some ML knowledge also realize how far we are from a computer program that can read code, read specs, and then decide how to implement or fix existing code. And last, even if all of the above were moot, I think it's more likely that we write a program that successfully helms a Fortune 500 company before we write one that produces or improves on software. The former could be more easily reduced to a game of chess while the latter requires some massive novel research into "creativity". Maybe it's Mark Cuban who should be learning ML!
[+] [-] cookiecaper|9 years ago|reply
To be frank, the tech community has really been leaving a sour taste in my mouth the last few years. Everyone is anxious to bandwagon on these buzzwords that provide little to no benefit or are applicable only to a small segment of the tech population.
In most cases, even a superficial understanding of the problem space should make it obvious that $This_Weeks_Sexy_Solution is a very bad fit. Do you need a server that persists data and is individually addressable? Then why are you using Kubernetes and Docker, which are still struggling to figure out these very basic things? k8s and Docker have very specific uses, but unless you're Google, they're probably not the right fit for your production environment right now.
This phenomenon seemed to hit a critical mass with document databases and single-page apps, and it continues to iterate with every open-source release that comes out of Google or Facebook. Since Google released TensorFlow last year and the compounded hype of [much worse than advertised attempts at] conversational speech recognition in Siri and Alexa, the "machine learning" bandwagon is starting to try to edge into the spotlight, and it sounds like it's already a mandatory part of any VC pitch.
Fortunately, machine learning is pretty hard and you get into hairy math practically right away, so I don't think the legs on this one will last as long. But we'll see. We'll at least have a lot of faux-ML going around and a lot of people making spurious claims on their resumes.
The economic collisions that make Silicon Valley and the tech industry in general a hive for inexperienced and insecure youth are bearing some really interesting effects this way. How can a company that's not "blown about by every wind of [tech fad]" fully exploit its relative sanity for competitive advantage?
[0] https://news.ycombinator.com/item?id=13572415 ; my reply downthread at https://news.ycombinator.com/item?id=13573978
[+] [-] thearn4|9 years ago|reply
I'm curious what Fisher, Neyman, and Pearson would say about the current state of the field. Especially considering how often they and other statisticians disagreed with each other throughout the 20th century.
[+] [-] zebrafish|9 years ago|reply
Maybe we should explain the cost/benefit of the buzzwords vs. the science?
[+] [-] lottin|9 years ago|reply
[+] [-] mattnewton|9 years ago|reply
[+] [-] wakkaflokka|9 years ago|reply
[+] [-] russdill|9 years ago|reply
[+] [-] HeavenBanned|9 years ago|reply
[deleted]
[+] [-] feral|9 years ago|reply
I've a PhD and held ML-engineer positions in a few different companies - I've good industry awareness.
Most applied ML, for most companies, right now, is actually relatively simple models (hand-coded rules! logistic regression! You'd be shocked how common these are.) The bulk of the work is data cleaning, gathering, integration, deployment, productisation, reliability, avoiding pathological cases, special-casing, Product, UX. You do need ML specialists who understand the stuff, to make it all work and come together - but the ratio of ML specialists to the wider team is low. Maybe 1 or 2 specialist on a team of 10 for an ML heavy product.
This is going to remain the case IMO. Yes, there will be small teams, in highly resourced organizations (GOOG, FB etc), academic research labs, or occasional hard-tech startups, who do new model development. Maybe if AI becomes huge, you'll see more traditional Fortune 500s spin up similar efforts.
But there'll be a much wider set of people&businesses applying and tuning well understood approaches, rather than doing new model development. And you just don't need as many ML specialists, for that approach.
Even with deep learning, the tooling will advance. I mean, even look at all the research papers describing applications at the moment - so many of them are using pre-trained models. Industry will be similar. Tooling will advance, and you'll be able to do increasingly more with off-the-shelf pieces.
I think ML is absolutely going to have a big impact - I buy at least some of the hype. But should all developers, or even a substantial minority of developers, start learning ML as a career imperative? I don't think so.
Finally, it takes serious time to learn this stuff. Its easy to dabble (and worthwhile doing - its fun; and sometimes you can do powerful things in a using tools in a very blackbox manner!). But actually thoroughly learning it takes time. It takes serious time to build statistical intuition, as just one example.
We could easily end up with a great many career developers who have a specialization in ML, frustrated they never get to use it.
[+] [-] ylem|9 years ago|reply
[+] [-] ska|9 years ago|reply
I think to the broader point that generally learning how to validate a ML technique is going to be very useful to a lot of people, implementing them less so.
[+] [-] deepnotderp|9 years ago|reply
And let's take a look at AlphaGo, how would you do that with SVMs or decision trees? Just get over the fact that deep learning provides a level of "intuition" (Go's search space is famously greater than the estimated number of stars in the universe).
I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of course...) and would prefer that their preferred methods would still reign supreme.
I'll end this wall of text on the note that OpenAI's Ian Goodfellow notes that all projects at OpenAI use deep learning right now, but they are not dogmatic and will consider other approaches if they work well. I think this is the path that should be taken. On the other hand, I also see a bright future for uniting traditional techniques with deep learning such as attaching a SVM to a CNN and combining decision trees with CNNs both of which have resulted in good results.
[+] [-] dmreedy|9 years ago|reply
I think that's definitely a part of it, and I feel that way sometimes myself (not that I'm a PhD). But there's another side of that reluctance that lies on the axis of model accountability and explicability. A lot of modern ML/Deep Learning doesn't -feel- like we're understanding anything any more than we did ten years ago. Yes our black-box results are better according to the tests we've laid out for them, but there's something more slippery about the 'why', beyond the handwave of 'complexity'. Maybe this is just the way it will be going forward (in the spirit of Quantum's "shut up and calculate"), but it is not easy to give up something that you can wrap your head around with something that kind of just takes care of itself, especially if you're in the business of seeking knowledge instead of results.
[+] [-] feral|9 years ago|reply
> "traditional" machine learning methods ... may be good enough for business tasks, they are not the cutting edge of AI research
That's true - but so? Under discussion is whether every developer should drop what they are doing and learn ML/DL, not what the cutting edge of AI research is?
In that context a lot of the low-hanging fruit for the next while is going to involve actually deploying simpler ML, with all the non-ML work that entails - and maybe using off-the-shelf DL to extract information from unstructured data.
> older ML PhDs are angry that deep learning is so easy
That's very ungenerous. People tend to do PhDs in things like ML (often making sacrifices to do so) because they want to advance the field - not to make it harder! People are excited. They are wary about hype causing another AI winter, or a lot of important projects failing for silly reasons.
[+] [-] reachtarunhere|9 years ago|reply
On the other hand it is true that DL is not required for many (most?) companies as they don't work with the kind of problems you list (vision, robotics)
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] romaniv|9 years ago|reply
This is a poor example. There are Go programs that work very well and use exclusively MCTS without any neural networks.
[+] [-] evgen|9 years ago|reply
[+] [-] pjungwir|9 years ago|reply
- I know enough machine learning to be dangerous, but I'm hardly ever asked to use it. I designed a Bayesian classifier for my own startup around 6 years ago, analyzing political donor networks. I've completed the Stanford ML course. Back in college I did a math minor, so I'm comfortable with linear algebra, calculus, etc. I'm pretty comfortable with statistics of both kinds. But my bread-and-butter is freelance web development . . . and I'm not really even sure how to find work doing more MLy things.
- I've read over and over that the most time-consuming part of ML work is data collection & cleanup, and that matches my own experience. It is the same thing that killed so many data warehouse projects in the 90s. You don't need a Ph.D. to do it, but it is a tough and costly prerequisite. So it seems like you'll need non-ML programmers even for specifically ML projects.
- In a similar vein, Google has written about the challenges of "operationalizing" machine learning projects.[1] Having a little experience collaborating with a team doing an ML project, where they did the ML engine and I did the user-facing application, I can say that many ML experts are not experts in building reliable, production-ready software.
- Will there ever be a Wordpress of machine learning? If there is, the author will be rich, but you won't need a Ph.D. to operate it. But because ML requires hooks into your existing systems, I don't know if this will ever happen. What will happen I think is plugins to existing e-commerce systems for product recommendation or other off-the-shelf ML-powered features. These already exist, but I assume they will become more prevalent and powerful over time. In any case, the mainstreaming of ML for business will be inversely correlated with the expense to implement it, which suggests it will be easier and easier for non-expert developers to use (and misuse).
EDIT: Added the (now-)third bullet point I forgot before.
[1] https://research.google.com/pubs/pub43146.html
[+] [-] vidarh|9 years ago|reply
E.g. when I was reading up on genetic algorithms etc. 20 years ago we also expected the "revolution" to be right around the corner, and that things like genetic programming would change the world in a few years time. And while various of those methods found use some places, most places that could have used at least some of the simpler ones, still don't.
In other words, I think talking about a 3 year timeline is crazy. It's getting more attention, sure, but there is so much low-hanging fruit that most developers could be busy for the next 20 years putting in place the most trivial algoriths all over the place and we still wouldn't have picked off even the low hanging fruit where the computational resources and algorithms and data to make a big impact were well within reach 20 years ago.
This certainly means there is plenty of room for a lot developers to do very cool stuff and build careers on machine learning today, but it also mean most developers will not have to learn the state of the art - or anything near it - for a very long time.
As a concrete example I give to people, consider all of the search boxes out there on various sites - product searches, location searches, site searches - that are straight keyword based searches that don't take into account any clickstream data to improve ranking. The proportion of search boxes I see that take advantage of the available data is vanishingly small, even though very basic analysis can improve the perceived relevance of the results massively.
We certainly will see more companies invest in proper machine learning as the payoff gets higher and difficulty in taking advantage of it drops. But we will also see a huge proportion of sites that could use it continue to ignore it for years to come.
There are big business opportunities in finding ways of making a dent in that portion of the market, though, and so learning this stuff can certainly be well worth it on a personal level, but I don't believe in his timeline in terms of the overall market.
[+] [-] anupshinde|9 years ago|reply
"""He thinks even programming is vulnerable to being automated and reducing the number of available programming jobs."""
I believed something similar could happen within 1-2 years of learning/writing AI programs (more than 12 years back). I believed it so much that it consumed most of my weekends as I took on the Genetic Programming approach. Yes! computers can write programs - BUT trying reading those. Eventually after spending hours or days, you will be able to read those programs and you might find a simple "hello-world" program represented by a complex mathematical equation. Good luck trying to get such program fixed by humans. Imagine an experience decoding deep-learning-neural-nets. However, that is black-box from a programmer perspective.
From a business/management personnel perspective - the code is a black box anyways. When they get NNs that can generate required software, they will replace the people-manager with a NN-manager (who is a programmer btw!)
[+] [-] primaryobjects|9 years ago|reply
Using Artificial Intelligence to Write Self-Modifying/Improving Programs http://www.primaryobjects.com/2013/01/27/using-artificial-in...
[+] [-] makeset|9 years ago|reply
I wouldn't bet against it. It's very risky to extrapolate progress in AI.
A DARPA project named CHI [1] succeeded in automatically producing programs from a high-level knowledge description language. And when they fed it its own description, CHI rewrote itself, with a better implementation. Do you fear for your programming job yet? Well, that was 36 years ago.
[1] www.dtic.mil/get-tr-doc/pdf?AD=ADA105661
[+] [-] gremlinsinc|9 years ago|reply
Maybe if it was coming from Bill Gates, Mark Zuckerberg, or another tech titan with some actual coding experience and a deeper level of learning about what ML even is. Cuban's a business man, and most CEO's I know don't have a clue about the stacks that run their own company, let alone what's popular.
That said, I do think ML will be important, but I develop ecommerce apps and things of the such in Laravel, unless I move into AI and Neural nets I don't see needing to know a lot about ML (though I wouldn't mind moving in that direction as that space picks up) -- but there's still plenty of opportunities without it.
[+] [-] marricks|9 years ago|reply
What causes AI winters is when an advance such as deep learning can be applied to new problems and leads to increased interest. And while this new thing is really good as a subset of problems and impresses the public, of course it can't displace humans at everything and naturally has it's limitations.
So funding pours in, everyone gets hyped, and then those natural limits are (re)discovered and everyone gets all anti-AI research. Of course many people knew the limitations all along, but the dream is gone and so is a lot of funding until the next thing comes along.
This is probably natural to a lot of fields but AI just seems more prone to these boom and bust cycles because it's really exciting stuff.
[1] https://en.wikipedia.org/wiki/AI_winter
[+] [-] sevensor|9 years ago|reply
[+] [-] itg|9 years ago|reply
This is more VC/founders who are hyping up AI and need more ML folks so they can drive down costs.
[+] [-] minimaxir|9 years ago|reply
The modern tools for ML/deep learning are accessible to all open source and well documented. And as I note in my top-level comment, old-fashioned statistical methods like linear regression are more than sufficient for real-world business problems, and definitely do not require a PhD to grok.
[+] [-] jbooth|9 years ago|reply
Yet, I can still apply machine learning to solve Ax=b problems. More importantly, I can use business analysis and write code to transform business problems into an Ax=b problem, and then optimize it.
You don't need a PhD to grok optimizing a vector to transform a matrix of inputs into a vector of observed outputs, then apply that trained vector going forward. Neural nets are slightly more complicated than a straight linear regression, but only slightly. I'd call decision tree methods like GBM even more complex, but still eminently grokkable for a decent programmer.
[+] [-] Avalaxy|9 years ago|reply
Stop spouting this bullshit. You don't need a PhD, and you don't need to advance the field to be doing 'serious ML'. All you need to be able to do is know how and when to apply it to solve crucial business problems.
[+] [-] solipsism|9 years ago|reply
Obviously you will want a theoretical expert on your team. But if your startup is counting on having a room full of them, good luck.
[+] [-] forgetsusername|9 years ago|reply
Good thing the vast majority of businesses won't need "serious" ML, but instead will require only simple implementations to help solve business problems.
[+] [-] sidlls|9 years ago|reply
It isn't required for any serious research effort, except by the accident of inertia. And it certainly isn't a necessary indicator of determination.
[+] [-] bsaul|9 years ago|reply
I mean, we don't need a phd in image compression to create a service that streams videos. We just use libraries. Same for everything in computer science, it always end up packaged in some kind of reusable code or service, and only some specialists remain working in the field to work on marginal improvements.
Why would ML be any different ?
[+] [-] Eridrus|9 years ago|reply
If you don't just move on to another problem you will need people who know how to push these systems further.
[+] [-] pesfandiar|9 years ago|reply
[+] [-] msvan|9 years ago|reply
[+] [-] geodel|9 years ago|reply
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] badthingfactory|9 years ago|reply
[+] [-] brilliantcode|9 years ago|reply
Luddites of 18th century thought they would never be replaced and continued on their trajectory.
[+] [-] ploika|9 years ago|reply
A lot of white-collar jobs may be automated (or otherwise changed beyond recognition due to technology) after about thirty years maybe, but not three.
[+] [-] mad44|9 years ago|reply
Reading through the comments, I see that Cuban's statement upset and even angered several HN commenters. That is a strong emotional reaction.
I am not saying it is the 5 stages of grief, but the first 3 fits: denial, anger, bargaining, depression and acceptance.
Also from Howard Aiken: Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats.
[+] [-] return0|9 years ago|reply
[+] [-] dkarapetyan|9 years ago|reply
[+] [-] anotheryou|9 years ago|reply
With this prosthetic knowledge we will have to learn much more what to ask and know how much the machine knows.
One has to quickly grasp the abstract that is one level too high or detailed for the machine to find and than find the seperate answers of the level below to recombine them. You can't yet ask where to open a restaurant, but you can google for demographics and write a program to map ratios between income, foot-traffic density and restaurant density.
Once we can ask what and where is the most profitable business in to start in town, we probably still don't get a step by step guide how to do it, interior design included. Where the rubber meets the road there is still a lot of oppertunities to decide on and complex data we can grasp more easy than the machine.
[+] [-] xs|9 years ago|reply
[+] [-] taytus|9 years ago|reply