I found this really frustrating to read; on 'training your own models':
> Deep learning is a class of machine learning that has gotten a lot of well-deserved attention in recent years because it's working to solve a wide variety of AI problems in vision, natural language processing, and many others.
What others? Be specific.
The problem is not that people don't know how to build neural networks.
The tensorflow tutorial is extremely comprehensive and there are lots of resources for people who know what problem they're solving to try to build a solution.
The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.
Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.
That's. All. It. Does.
It takes an input array and outputs an output array; a magic f(x) function.
The thing people don't understand well is not how do I implement f(x); the problem is how do I convert my domain problem (I have 2000000 shelf products in my database and a tonnes of sales information) how to convert that into an input array of numbers? What is the output array of numbers I want out of the function?
The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.
That's why its good for images; because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem.
...but if your problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.
tldr; The problem people are having is figuring out input and output for domains other than image processing; not how to build a NN, or how to choose which NN to build.
Guide misses the point entirely.
(You might argue this is somewhat uncharitable to what is actually a well written guide to ML; but you can read the about section for yourself and see what you think; I feel like the guide needs to focus sharply on practical business outcomes if it wants to do what it says its doing)
Technical but important quibble. You've described supervised learning: given pairs (X, Y), learn the function F such that Y = F(X).
Unsupervised learning takes samples and generates more of the same kind of data. We can see unsupervised learning as being given samples (X) and training a function X = F(noise) that transforms a random vector into an output whose distribution resembles that of the original generating function.
Reinforcement learning sees a reward R(X) of policies X and learns to find X such that R(X) is high.
All of these have been known to operate on trees or other representations instead of arrays.
This matters because there are business problems that are not supervised learning. Reinforcement learning problems especially. Sometimes I can say how well an output is doing and I want an output that does better.
Thanks for writing this. It sounds like what I've experienced in this kind of work isn't just accidental. I am working on leveling up math skills to be able to understand the process for TensorFlow etc, but I get the feeling that once it's finally tractable I'll still have the same problems of finding how to transform the inputs into whatever works best for the model, what to output, and how to use the output. This seems to be the part that can't be solved by off-the-shelf packages. And the part where many or most exciting discoveries will be made in the future.
>>The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.
Yes, exactly.
As a "Pull" programmer [1], I can't tell how I would use this technology to solve problems I have or my company has. This makes it difficult for me to get motivated to learn it, despite all the buzz and excitement.
I went through the tensorflow tutorial. While it is interesting, I didn't find it useful, the way learning a new programming language or framework or library would be.
I could not agree more. There should be more articles like this one (https://hackernoon.com/how-to-better-classify-coachella-with...) clearly outlining the steps what needs to be done to solve a real world problem. The algorithms by themselves are very powerful.
>...but if you problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.
Totally agree. As someone who does scientific modeling, fitting models is easy. Finding the model that correctly describes my problem and can be trained with existing or easily acquired input data is hard.
> The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.
> because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem
Nitpick maybe, but you do not represent each class by a number. This would create a bias to consider subsequent categories as similar (why would aardvark=0 be more similar to antilope=1 than cat=2?). Instead you have one dimension per possible-value (aka one-hot encoding) and the value in that dimension represents how likely the photo belongs to that class (with values summing to 1, softmax, if it's a one-class problem, or no such guarantee if it's a multiclass problem).
> Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.
> That's. All. It. Does.
No. That's. WHAT. It. Does.
Your emotional overreaction to this site has induced bias leading to a false negative -- mistaking hype for signal. Fact is, using deep learning, newbie grad students have routinely and resoundingly outperformed nearly all the state-of-the-art techniques in signal processing in less than FIVE years of trying. Image analysis and speech recognition have been revolutionized; 40+ years of R&D expertise has been largely supplanted in the blink of an eye. Dismiss this tsunami at your own peril.
Does DL scale to harder problems? Why yes, it does. No computer implementation of the game Go had outperformed the best humans until deep learning was used, then it beat the best humans almost immediately (unlike chess which took decades to reach that level using traditional AI methods). I have little doubt that significant achievements await DL's application to more complex tasks like language analysis too. To believe otherwise would be severely short-sighted.
Does deep learning also solve trivial pattern rec problems like regression or curve fitting as efficiently as some linear ML models? No, but who cares? By that reasoning, we should dismiss Einstein because many 12 year old children can multiply numbers faster and for less money.
The ultimate goal of deep learning is to remove the need to transform input.
The traditional computer vision pipeline works like this: you preprocess the image, featurize it with features like SIFT, and then run an SVM classifier. However, this pipeline is totally beaten by end-to-end neural networks.
Same will hold true for other domains. As long as we've discovered the right architecture, aka. the family for f(x), learning still boils down to training your models.
This is an important insight, however I think more of a research topic. The guide was careful to provide guidance on problems that have a very firm foundation. It didn't want to discuss something that would just drag its audience into the weeds.
The dimension of the input and output arrays needn't be different, right? There's no reason you can't input a image into a convolutional neural network and output an altered image of the same size.
> tldr; The problem people are having is figuring out input and output for domains other than image processing.
This and some of your other comments in this thread lead me to believe you are unaware of how DL is being used in NLP. I would argue real industry use of DL for NLP dwarfs that of DL for images. Personal assistants, search, customer service, sales, chat are the big general cases, let alone company-specific tasks (.e.g. financial data, reviews) or other NLP domains that are not particularly industry-adopted (.e.g. translation, summarization). There's a lot of demand for NLP DL experts, and it's no surprise this year that Socher's NLP DL class had 700 Stanford students enrolled.
Artificial Intelligence is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.
Please don't write this off as a reddit-esque comment. I really think the analogy holds -- just take a look at Product Hunt front page, or HN for that matter.
Some people are really doing it, but they're not really doing it properly. Some of them will get better at it, and some will realize they started doing it way too early.
In a few years though, most people will be doing it, and it'll be great (they'll exagerate how much they do it though).
It was pretty incredible to open up https://vimeo.com/215926017 and click at a random timestamp (29:30) and hear "we're trying to use AI to diagnose cancer by examining the DNA that's free-floating in your bloodstream."
This is clearly the future, if it can work. That's probably one of the biggest startup ideas that you could possibly have. I hope that whoever's behind it is successful.
Freenome is the company, they're doing interesting work but it's still early days. This isn't actually a very new idea, it's been floating around for years in academia (I've personally been hearing it about it for at least 5 years). The hard part isn't really the AI per se, it's getting enough DNA from the circulating copies (i.e. sample prep) and being lucky enough that the cancer you're trying to diagnose actually puts out enough DNA for you to measure at an early enough stage to be clinically relevant. In other words, if you can only get DNA when it's at stage 4 and metastatic, your DNA test isn't actually very useful. If you can get stage 1 cancers, now you're talking, but in those cases tumors tend to shed a lot less DNA (among other reasons because they're smaller).
A very good friend of mine is their CSO, I think they've got a really cool idea and could indeed go far, but it's a fiendishly hard problem.
In Los Angeles area, how does one get involved in these projects? I would like to volunteer at a local research lab. I am coming from a computer scientist background and would like to learn more about the medical space.
For the record, I've worked on building a classifier for nudity detection and not much get's better than starting with a pre-trained model and fine-tuning to your specific rules. I wouldn't be surprised if this is the future of deep learning: Start with the closest pre-trained model that has trained for months using custom clustered hardware and fine-tune further on commodity hardware.
While this site has some great info for non-technical folk who are serious about learning AI, I think the site will fly over the heads of most "line-of-business leaders" at whom it's aimed.
In my experience, those folks think of AI like...
My company is collecting tons of data in [insert data warehouses here]. How can I bolt on some "AI" and find out what's "interesting"?
> My company is collecting tons of data in [insert data warehouses here]. How can I bolt on some "AI" and find out what's "interesting"?
For quite a few business types, this can be further simplified to just:
How can I bolt on some "AI"?
It's a nice bonus if you have some data and can find interesting things from it. But the goal for a surprising number of managers is just to be able to say they have some advanced AI technology doing, well, who knows what, just so long as there is definitely some advanced AI technology doing it.
This is a great point, and I think I fall into that bucket you describe. I have just enough information to be dangerous.
There does seem to be a lack of available tools even for someone with a well-defined problem with available labelled data to validate implementing machine learning to a problem. I think that there's a need out there for a software with a GUI that can accept data and let you try some ML approaches with it and take the coding out of it - and it doesn't seem like it would be insanely difficult to build.
This is spot-on. There's an assumption that if you have large/big data, then AI will be your next big step. That's absolutely not a given. I've been told that AI is just "Statistics at scale", but you wouldn't call simple decision making like A/B testing "AI".
Also, I think there's an emerging category outside of MLaaS that focuses on ML/DL deployment (ops).
Here's a quick intro:
Seldon's mission is to help people predict and shape the future with machine learning. Our products increase efficiency and speed up the time to value for data scientists:
Seldon Core - a machine learning platform that helps your data science team deploy models into production. It provides an open-source data science stack that runs within a Kubernetes Cluster. You can use Seldon to deploy machine learning and deep learning models into production on-premise or in the cloud (e.g. GCP, AWS, Azure). Seldon is framework agnostic, it supports models built with TensorFlow, Keras, Vowpal Wabbit, XGBoost, Gensim and any other model-building tool — it even supports models built with commercial tools and services where the model is exportable. Fork Seldon on Github: https://github.com/SeldonIO/seldon-server/
Seldon Deploy - in development - a machine learning deployment platform for data science teams. Features include workflows with approvals and audit trails, advanced experiments, continuous integration and model explanations.
Seldon was part of the Barclays Techstars '16 and we’re working with customers in banking, technology and media sectors.
Does anyone have a list of VCs that are specializing in investing into AI startups?
I'm looking for a seed and it'd be great to have a list like that (I'm experienced in the field, first started with Deep Learning in 2010, it's on my GitHub...)
There are tons: Data Collective, Array Ventures, Susa Ventures (my fund), Switch Ventures, Amplify Partners, Zetta Venture Partners, AME Cloud Ventures, Ten One Ten, Bloomberg Beta, etc.
That's not an exhaustive list by any means, just some of the first ones that come to mind for me. To be honest, I think the majority of seed funds are interested in AI/ML startups these days. Which one is the best fit for you will depend on the specific flavor of company you're building, your location, your level of traction, et cetera.
If you have any questions, you can email me at: leo at susaventures.com.
"Deep learning is a name for machine learning techniques using many-layered artificial neural networks. Occasionally people use the term artificial intelligence, but unless you want to sound sci-fi, it is reserved for problems that are currently considered “too hard for machines” - a frontier that keeps moving rapidly." from http://p.migdal.pl/2017/04/30/teaching-deep-learning.html
In particular, two decades ago, letter recognition was considered a holy grail of AI, see:
"Indeed, I once even proposed that the toughest challenge facing AI workers is to answer the question: “What are the letters ‘A’ and ‘I’?" - Douglas R. Hofstadter (1995)
Not it as simple assignment to anyone learning ConvNets (BTW: I really recommend notMNIST as a starting point).
I've seen that Hofstadter quote used in this context before, and always assumed it was being quoted out of context. Do you know where it comes from? I suspect the idea is related to his "Letter Spirit" project - if so approaches like notMNIST don't really speak to it.
Hofstadter often wrote about the difficulty in semantic understanding, and how AI approaches often approached things in such a superficial way that they miss the deeper meaning by design. Many of his projects were aimed at taking on these much harder problems head on. I suspect this quote, while also being a play on words, is aimed at this deeper understanding.
For a business guy this is a very useful resource, nice and clear. Can understand some of the criticism but perhaps people are expecting too much.
I specifically liked the Image Feature Analysis tool and the fact this is exposed for others to use on github. Thinking about linking it to my home security camera.
Site is incomplete. Misses out on whole categories of software classified as AI (like Cognitive Computing with approaches like associative memory).
Sad.
[+] [-] shadowmint|8 years ago|reply
> Deep learning is a class of machine learning that has gotten a lot of well-deserved attention in recent years because it's working to solve a wide variety of AI problems in vision, natural language processing, and many others.
What others? Be specific.
The problem is not that people don't know how to build neural networks.
The tensorflow tutorial is extremely comprehensive and there are lots of resources for people who know what problem they're solving to try to build a solution.
The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.
Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.
That's. All. It. Does.
It takes an input array and outputs an output array; a magic f(x) function.
The thing people don't understand well is not how do I implement f(x); the problem is how do I convert my domain problem (I have 2000000 shelf products in my database and a tonnes of sales information) how to convert that into an input array of numbers? What is the output array of numbers I want out of the function?
The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.
That's why its good for images; because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem.
...but if your problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.
tldr; The problem people are having is figuring out input and output for domains other than image processing; not how to build a NN, or how to choose which NN to build.
Guide misses the point entirely.
(You might argue this is somewhat uncharitable to what is actually a well written guide to ML; but you can read the about section for yourself and see what you think; I feel like the guide needs to focus sharply on practical business outcomes if it wants to do what it says its doing)
[+] [-] Eliezer|8 years ago|reply
Unsupervised learning takes samples and generates more of the same kind of data. We can see unsupervised learning as being given samples (X) and training a function X = F(noise) that transforms a random vector into an output whose distribution resembles that of the original generating function.
Reinforcement learning sees a reward R(X) of policies X and learns to find X such that R(X) is high.
All of these have been known to operate on trees or other representations instead of arrays.
This matters because there are business problems that are not supervised learning. Reinforcement learning problems especially. Sometimes I can say how well an output is doing and I want an output that does better.
[+] [-] guscost|8 years ago|reply
[+] [-] enraged_camel|8 years ago|reply
Yes, exactly.
As a "Pull" programmer [1], I can't tell how I would use this technology to solve problems I have or my company has. This makes it difficult for me to get motivated to learn it, despite all the buzz and excitement.
I went through the tensorflow tutorial. While it is interesting, I didn't find it useful, the way learning a new programming language or framework or library would be.
[1]http://v25media.s3.amazonaws.com/edw519_mod.html#chapter_71
[+] [-] jsemrau|8 years ago|reply
[+] [-] axiom92|8 years ago|reply
That's one general statement over another. Consider https://arxiv.org/abs/1508.00021 as a counterexample.
[+] [-] ferdterguson|8 years ago|reply
[+] [-] halflings|8 years ago|reply
> because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem
Nitpick maybe, but you do not represent each class by a number. This would create a bias to consider subsequent categories as similar (why would aardvark=0 be more similar to antilope=1 than cat=2?). Instead you have one dimension per possible-value (aka one-hot encoding) and the value in that dimension represents how likely the photo belongs to that class (with values summing to 1, softmax, if it's a one-class problem, or no such guarantee if it's a multiclass problem).
[+] [-] randcraw|8 years ago|reply
> That's. All. It. Does.
No. That's. WHAT. It. Does.
Your emotional overreaction to this site has induced bias leading to a false negative -- mistaking hype for signal. Fact is, using deep learning, newbie grad students have routinely and resoundingly outperformed nearly all the state-of-the-art techniques in signal processing in less than FIVE years of trying. Image analysis and speech recognition have been revolutionized; 40+ years of R&D expertise has been largely supplanted in the blink of an eye. Dismiss this tsunami at your own peril.
Does DL scale to harder problems? Why yes, it does. No computer implementation of the game Go had outperformed the best humans until deep learning was used, then it beat the best humans almost immediately (unlike chess which took decades to reach that level using traditional AI methods). I have little doubt that significant achievements await DL's application to more complex tasks like language analysis too. To believe otherwise would be severely short-sighted.
Does deep learning also solve trivial pattern rec problems like regression or curve fitting as efficiently as some linear ML models? No, but who cares? By that reasoning, we should dismiss Einstein because many 12 year old children can multiply numbers faster and for less money.
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] strin|8 years ago|reply
The traditional computer vision pipeline works like this: you preprocess the image, featurize it with features like SIFT, and then run an SVM classifier. However, this pipeline is totally beaten by end-to-end neural networks.
Same will hold true for other domains. As long as we've discovered the right architecture, aka. the family for f(x), learning still boils down to training your models.
[+] [-] davedx|8 years ago|reply
https://aigrant.org/
[+] [-] MrMike|8 years ago|reply
[+] [-] blazespin|8 years ago|reply
[+] [-] thousandautumns|8 years ago|reply
[+] [-] smefi|8 years ago|reply
[+] [-] orthoganol|8 years ago|reply
This and some of your other comments in this thread lead me to believe you are unaware of how DL is being used in NLP. I would argue real industry use of DL for NLP dwarfs that of DL for images. Personal assistants, search, customer service, sales, chat are the big general cases, let alone company-specific tasks (.e.g. financial data, reviews) or other NLP domains that are not particularly industry-adopted (.e.g. translation, summarization). There's a lot of demand for NLP DL experts, and it's no surprise this year that Socher's NLP DL class had 700 Stanford students enrolled.
[+] [-] tontonius|8 years ago|reply
Please don't write this off as a reddit-esque comment. I really think the analogy holds -- just take a look at Product Hunt front page, or HN for that matter.
[+] [-] rjtavares|8 years ago|reply
In a few years though, most people will be doing it, and it'll be great (they'll exagerate how much they do it though).
Yes, the analogy holds.
[+] [-] sillysaurus3|8 years ago|reply
This is clearly the future, if it can work. That's probably one of the biggest startup ideas that you could possibly have. I hope that whoever's behind it is successful.
[+] [-] entee|8 years ago|reply
A very good friend of mine is their CSO, I think they've got a really cool idea and could indeed go far, but it's a fiendishly hard problem.
[+] [-] sjg007|8 years ago|reply
[+] [-] ktu100|8 years ago|reply
[+] [-] joshatidealspot|8 years ago|reply
[+] [-] jonobelotti|8 years ago|reply
The start of your second sentence seems to me pretty inaccurate.
[+] [-] eanzenberg|8 years ago|reply
For the record, I've worked on building a classifier for nudity detection and not much get's better than starting with a pre-trained model and fine-tuning to your specific rules. I wouldn't be surprised if this is the future of deep learning: Start with the closest pre-trained model that has trained for months using custom clustered hardware and fine-tune further on commodity hardware.
[+] [-] mifeng|8 years ago|reply
In my experience, those folks think of AI like...
My company is collecting tons of data in [insert data warehouses here]. How can I bolt on some "AI" and find out what's "interesting"?
[+] [-] _delirium|8 years ago|reply
For quite a few business types, this can be further simplified to just:
How can I bolt on some "AI"?
It's a nice bonus if you have some data and can find interesting things from it. But the goal for a surprising number of managers is just to be able to say they have some advanced AI technology doing, well, who knows what, just so long as there is definitely some advanced AI technology doing it.
[+] [-] riphay|8 years ago|reply
There does seem to be a lack of available tools even for someone with a well-defined problem with available labelled data to validate implementing machine learning to a problem. I think that there's a need out there for a software with a GUI that can accept data and let you try some ML approaches with it and take the coding out of it - and it doesn't seem like it would be insanely difficult to build.
[+] [-] fudged71|8 years ago|reply
[+] [-] quocble|8 years ago|reply
[+] [-] ahousley|8 years ago|reply
Do you accept pull requests? I'd like to make a suggestion for the software and services page: http://aiplaybook.a16z.com/docs/reference/software
Also, I think there's an emerging category outside of MLaaS that focuses on ML/DL deployment (ops).
Here's a quick intro:
Seldon's mission is to help people predict and shape the future with machine learning. Our products increase efficiency and speed up the time to value for data scientists:
Seldon Core - a machine learning platform that helps your data science team deploy models into production. It provides an open-source data science stack that runs within a Kubernetes Cluster. You can use Seldon to deploy machine learning and deep learning models into production on-premise or in the cloud (e.g. GCP, AWS, Azure). Seldon is framework agnostic, it supports models built with TensorFlow, Keras, Vowpal Wabbit, XGBoost, Gensim and any other model-building tool — it even supports models built with commercial tools and services where the model is exportable. Fork Seldon on Github: https://github.com/SeldonIO/seldon-server/
Seldon Deploy - in development - a machine learning deployment platform for data science teams. Features include workflows with approvals and audit trails, advanced experiments, continuous integration and model explanations.
Seldon was part of the Barclays Techstars '16 and we’re working with customers in banking, technology and media sectors.
Disclosure: I’m CEO at Seldon.
[+] [-] dchichkov|8 years ago|reply
I'm looking for a seed and it'd be great to have a list like that (I'm experienced in the field, first started with Deep Learning in 2010, it's on my GitHub...)
[+] [-] lpolovets|8 years ago|reply
That's not an exhaustive list by any means, just some of the first ones that come to mind for me. To be honest, I think the majority of seed funds are interested in AI/ML startups these days. Which one is the best fit for you will depend on the specific flavor of company you're building, your location, your level of traction, et cetera.
If you have any questions, you can email me at: leo at susaventures.com.
[+] [-] fiatjaf|8 years ago|reply
[+] [-] saagarjha|8 years ago|reply
[+] [-] annnnd|8 years ago|reply
[+] [-] stared|8 years ago|reply
"Deep learning is a name for machine learning techniques using many-layered artificial neural networks. Occasionally people use the term artificial intelligence, but unless you want to sound sci-fi, it is reserved for problems that are currently considered “too hard for machines” - a frontier that keeps moving rapidly." from http://p.migdal.pl/2017/04/30/teaching-deep-learning.html
In particular, two decades ago, letter recognition was considered a holy grail of AI, see:
"Indeed, I once even proposed that the toughest challenge facing AI workers is to answer the question: “What are the letters ‘A’ and ‘I’?" - Douglas R. Hofstadter (1995)
Not it as simple assignment to anyone learning ConvNets (BTW: I really recommend notMNIST as a starting point).
[+] [-] ska|8 years ago|reply
Hofstadter often wrote about the difficulty in semantic understanding, and how AI approaches often approached things in such a superficial way that they miss the deeper meaning by design. Many of his projects were aimed at taking on these much harder problems head on. I suspect this quote, while also being a play on words, is aimed at this deeper understanding.
[+] [-] monkeydust|8 years ago|reply
I specifically liked the Image Feature Analysis tool and the fact this is exposed for others to use on github. Thinking about linking it to my home security camera.
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] thr0waway1239|8 years ago|reply
???? (release a self-anointed playbook)
Profit!
[+] [-] smefi|8 years ago|reply
[+] [-] amelius|8 years ago|reply
Deep Learning is more about pattern matching, not about "understanding" patterns.
[+] [-] JabavuAdams|8 years ago|reply
[+] [-] baxtr|8 years ago|reply
[+] [-] monkeydust|8 years ago|reply
[deleted]
[+] [-] beders|8 years ago|reply