If you're trying to learn about deep learning, I highly suggest using Python(Theano) or Lua(Torch). They're free and used by the experts in the field for research.
Even if you don't want to use the frameworks, you'll still have access to fast linear algebra routines.
Could someone can recommend me a book about deep learning and/or machine learning for this kind of open-source library ? I do not have any background in ML nor DL.
Are there any good Neural Network frameworks written in Ruby?
The ones I have used (ruby-fann and AI4r) dramatically slow down when you use them on large amount of data.
Nice article. If anyone is interested in understanding the theory and also dig deeper, the machine learning course on Coursera is a great place to start as well.
It's still widely used in academia (though I'm extremely glad IPython/Jupyter is being praised by Nature[1]), and it has a lot of "toolboxes" with a reasonable integration between them.
Once you step outside it, you need one library for the machine learning part of your project, another one for the computer vision aspects, yet another one for the reading of different data files. When you get to the 20th library you'd need, plus all the classes and wrappers you have to write on your own to achieve the same result, you start looking MATLAB under a softer light. That's even more for those who are no programming-inclined.
Actually matlab i had great experience learning NN in matlab. It is very fast with matrix multiplication and have comprehensive toolbox set. If you need prototype anything, matlab is your choice.
At times my mechanical engineering courses at university felt like MATLAB tutorials. You had to use it, no way around it. Good luck once you're out of university and want to start your own thing, you won't be able to afford it. The computer science courses in contrast preferred open source tools over proprietary ones.
ANNs are great for the right application. But I'm starting to fear "Deep Learning" is the new "Big Data" buzzword.
I believe ANNs are Turing Complete, meaning they should be able to compute anything (EDIT: + "that is computable by any other Turing Machine"). The questions are, can a training regimen be created to create the right ANN to solve "any" problem, and if so, is it an efficient means to solve that problem?
For example, it's fairly trivial to build an ANN to spit out the right results for a given polynomial function, i.e. "f(x, y, z) = ax + by + cz". Knowing the polynomial function ahead of time, you just generate a ton of input/output sets and feed them into the training of the ANN, and then from there on the ANN will spit them back out.
The problem with that is, you didn't learn anything new. You didn't learn how to solve a new problem. It's somewhat useful for teaching people how to program ANNs, but I personally think it's garbage for teaching how to understand ANNs.
ANNs make more sense when we already have the training data, but we don't know the underlying function that maps input to said outputs. In the trivial case of the polynomial function, if someone were to hand us the training set, we could use an ANN to figure out what the polynomial must be.
Except--for this particular example of a polynomial function--this isn't very efficient. For a polynomial of N terms, you only need N+1 sets of IO to trivially use algebra to determine the function. You can use any of the readily available linear algebra libraries to do such a thing. In fact, I wrote a project for a client that does just that: it uses a basic matrix library to crunch a set of GPS data to create a quadratic formula estimation of curves in roads, so that model can them be resampled, continuously, sans noise.
And if that function is not just a simple polynomial--if, say, it includes sines and cosines and square roots, etc.-- then the ANN is going to have to be large enough to include in it ad-hoc, arithmetic estimations of sine and cosine and square roots sufficient to give the right answers. It might even include several different estimating functions just for sine just because our mystery function requires more than one sine operation. It might even have corner cases where it gets the answer wrong, because you didn't have a sufficiently large data set for it to "figure out" things like the fact that sin(x) is approximately x for small values of x. If one knew the right formula (and yes, that's a big if), it'd be significantly more efficient to write a program that computed the values correctly.
All of this is not to poo-poo on ANNs. ANNs are great tools for when we don't know the function and when the function is sufficiently non-trivial to discover. The polynomial example is like trying to kill a fly on the wall with a swarm of nanonmachines designed to evolve and learn how to construct a flyswatter (which is part of the reason I dislike it as a learning tool). But write traditional code to do Optical Character Recognition, I dare you. ANNs are just highly specialized. Think of setting up your ANN like defining the full width and depth of the space of all possible programs that you'd like to search for the program that solves your problem. You then use feedback to "walk" across that space until you find something that looks like your desired program. We're entering an era where we have the memory and distributed processing capabilities to crank out some rather large ANNs. For some problems, we end up training a computer to write programs for us that we could have written on our own. This can impact the number of requests you can handle in a given amount of time.
Of course, that is not necessarily bad, either. "Throwing money at the problem" is not the wrong solution when you have a lot more money than time. Technology is supposed to serve us, not the other way around. Why spend a week discovering a formula to map your data when you can train an ANN in a few hours? And perhaps you don't have very high requirements for request handling. Maybe you only need to process one image a minute on your particular system. Have at it.
But you really, really need to know that is the case before you jump on the ANN bandwagon. You have to know what you want out of the ANN. If you don't have that ability to look at a set of inputs and express a desired set of outputs, then ANN isn't magic pixie dust that will solve that for you. If you have experts in your particular field telling you that your particular problem cannot be easily modeled, then ANNs might be helpful for you. If you are new to your field and you think "let's try an ANN", you're probably going to have a bad time. If you end up with an ANN that is estimating a relatively trivial program, and you're trying to provide a SaaS offering that is meant to scale to thousands or millions of concurrent users, the ANN approach could seriously harm your ability to scale.
It looks like your argument is essentially 'don't use a neural network when you can do a glm / other regression instead' which no serious person should disagree with.
I love matlab but specifically with neural networks, I made bad experiences. Just generally subpar performance on convergence speed and results. It's better to use caffe, which is the best neural network kit I know. Also, large parts of caffe are being implemented for GPUs such that performance becomes even better.
Just got 'Database Error'while trying to connect to this page.
Error establishing a database connection.
So, the number of connections to this page is limited? or what could be the issues throwing this kind of error?
can someone give me some real world business need where I can apply RNN and this type of knowledge? Obviously not looking for a hand out but open to exploring problems in the enterprise or any other potential problems worth solving which has a market.
I find that having a goal of what I want to solve or create motivates me to learn. Whereas if I'm studying Statistics but don't have a clear goal that motivates me (calculating sports betting odds) then it's that much harder to master and appreciate it's applications.
I guess to me, knowing the application of something before I dive both feet into learning it is actually the most important truth for beginners.
As a kid, did you want to make video games and then ended up learning programming but ultimately not making video games? No 8 year old thinks I'm going to implement lxml in javascript one day they just think of something they like or curious about (ex. video games).
[+] [-] ching_wow_ka|10 years ago|reply
Even if you don't want to use the frameworks, you'll still have access to fast linear algebra routines.
[+] [-] raphaelj|10 years ago|reply
[+] [-] temuze|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] akshayB|10 years ago|reply
[+] [-] an4rchy|10 years ago|reply
[+] [-] omegote|10 years ago|reply
[+] [-] dr_zoidberg|10 years ago|reply
Once you step outside it, you need one library for the machine learning part of your project, another one for the computer vision aspects, yet another one for the reading of different data files. When you get to the 20th library you'd need, plus all the classes and wrappers you have to write on your own to achieve the same result, you start looking MATLAB under a softer light. That's even more for those who are no programming-inclined.
[1] http://www.nature.com/news/interactive-notebooks-sharing-the...
[+] [-] artmageddon|10 years ago|reply
[0]: https://www.gnu.org/software/octave/
[+] [-] Dolores12|10 years ago|reply
[+] [-] fphilipe|10 years ago|reply
[+] [-] LesZedCB|10 years ago|reply
http://webcache.googleusercontent.com/search?q=cache:UhEgP6_...
[+] [-] moron4hire|10 years ago|reply
I believe ANNs are Turing Complete, meaning they should be able to compute anything (EDIT: + "that is computable by any other Turing Machine"). The questions are, can a training regimen be created to create the right ANN to solve "any" problem, and if so, is it an efficient means to solve that problem?
For example, it's fairly trivial to build an ANN to spit out the right results for a given polynomial function, i.e. "f(x, y, z) = ax + by + cz". Knowing the polynomial function ahead of time, you just generate a ton of input/output sets and feed them into the training of the ANN, and then from there on the ANN will spit them back out.
The problem with that is, you didn't learn anything new. You didn't learn how to solve a new problem. It's somewhat useful for teaching people how to program ANNs, but I personally think it's garbage for teaching how to understand ANNs.
ANNs make more sense when we already have the training data, but we don't know the underlying function that maps input to said outputs. In the trivial case of the polynomial function, if someone were to hand us the training set, we could use an ANN to figure out what the polynomial must be.
Except--for this particular example of a polynomial function--this isn't very efficient. For a polynomial of N terms, you only need N+1 sets of IO to trivially use algebra to determine the function. You can use any of the readily available linear algebra libraries to do such a thing. In fact, I wrote a project for a client that does just that: it uses a basic matrix library to crunch a set of GPS data to create a quadratic formula estimation of curves in roads, so that model can them be resampled, continuously, sans noise.
And if that function is not just a simple polynomial--if, say, it includes sines and cosines and square roots, etc.-- then the ANN is going to have to be large enough to include in it ad-hoc, arithmetic estimations of sine and cosine and square roots sufficient to give the right answers. It might even include several different estimating functions just for sine just because our mystery function requires more than one sine operation. It might even have corner cases where it gets the answer wrong, because you didn't have a sufficiently large data set for it to "figure out" things like the fact that sin(x) is approximately x for small values of x. If one knew the right formula (and yes, that's a big if), it'd be significantly more efficient to write a program that computed the values correctly.
All of this is not to poo-poo on ANNs. ANNs are great tools for when we don't know the function and when the function is sufficiently non-trivial to discover. The polynomial example is like trying to kill a fly on the wall with a swarm of nanonmachines designed to evolve and learn how to construct a flyswatter (which is part of the reason I dislike it as a learning tool). But write traditional code to do Optical Character Recognition, I dare you. ANNs are just highly specialized. Think of setting up your ANN like defining the full width and depth of the space of all possible programs that you'd like to search for the program that solves your problem. You then use feedback to "walk" across that space until you find something that looks like your desired program. We're entering an era where we have the memory and distributed processing capabilities to crank out some rather large ANNs. For some problems, we end up training a computer to write programs for us that we could have written on our own. This can impact the number of requests you can handle in a given amount of time.
Of course, that is not necessarily bad, either. "Throwing money at the problem" is not the wrong solution when you have a lot more money than time. Technology is supposed to serve us, not the other way around. Why spend a week discovering a formula to map your data when you can train an ANN in a few hours? And perhaps you don't have very high requirements for request handling. Maybe you only need to process one image a minute on your particular system. Have at it.
But you really, really need to know that is the case before you jump on the ANN bandwagon. You have to know what you want out of the ANN. If you don't have that ability to look at a set of inputs and express a desired set of outputs, then ANN isn't magic pixie dust that will solve that for you. If you have experts in your particular field telling you that your particular problem cannot be easily modeled, then ANNs might be helpful for you. If you are new to your field and you think "let's try an ANN", you're probably going to have a bad time. If you end up with an ANN that is estimating a relatively trivial program, and you're trying to provide a SaaS offering that is meant to scale to thousands or millions of concurrent users, the ANN approach could seriously harm your ability to scale.
[+] [-] strong_ai|10 years ago|reply
[+] [-] GFK_of_xmaspast|10 years ago|reply
[+] [-] rsy96|10 years ago|reply
You spent several paragraph criticizing ANNs, but regular ANNs are not deep learning at all.
[+] [-] ramgorur|10 years ago|reply
Their mcc compiler is even more crappier, it has so many memory leaks that even valgrind gives up and gets freezed.
I do not want to run a MATALBBED-ANN over large datasets, no way.
MATLAB scwhag: "Do you speak MATLAB ?"
me: "No, I don't speak MATLAB, and I don't want to"
[+] [-] Kenji|10 years ago|reply
[+] [-] gansai|10 years ago|reply
[+] [-] BowBun|10 years ago|reply
[+] [-] fsqcds|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] curiousjorge|10 years ago|reply
I find that having a goal of what I want to solve or create motivates me to learn. Whereas if I'm studying Statistics but don't have a clear goal that motivates me (calculating sports betting odds) then it's that much harder to master and appreciate it's applications.
I guess to me, knowing the application of something before I dive both feet into learning it is actually the most important truth for beginners.
As a kid, did you want to make video games and then ended up learning programming but ultimately not making video games? No 8 year old thinks I'm going to implement lxml in javascript one day they just think of something they like or curious about (ex. video games).
[+] [-] moron4hire|10 years ago|reply
https://en.wikipedia.org/wiki/Artificial_neural_network#Appl...
[+] [-] nnat425|10 years ago|reply
[deleted]
[+] [-] anacleto|10 years ago|reply
[deleted]