[hrokr]

I agree you'll definitely want to read Elements of Statistical Learning but there are a few more, namely Think Stats and Think Bayes.

Since no one has really said much about Bayes yet, I think it worth mentioning just how useful it is in DS and ML. A Bayesian approach makes a very good baseline and often one that is hard to beat.

If you're not particular fluent with Probability and Statistics now, let me suggest you add in Khan Academy (make sure to pick the CLEP version) and JBstatistics. Khan has the advantage of quizzes (so you're not just kidding yourself that you know the material). JBstatistics has the advantage of really good explanations. You'll probably want to watch Khan at x1.5 speed.