WingNews logo WingNews GitHub [2]
top | item 665029

Ask HN: good math books

111 points| bad_user | 17 years ago | reply

Dear HN, I need some insight from you.

While I have very solid foundation when it comes to algorithms, unfortunately my math skills are kind of rusty and lacking. I have big holes in my math education, partly because of bad luck with teachers in high-school/college. And the math I learned, I'm starting to forget because of lack of practice.

My domains of interest are data-mining, NLP and AI. These are domains where knowledge of mathematics is required to understand the underlying phenomenons. I mean, I got to some level of knowledge, but it is based on experimentation, empiric conclusions and cargo-culting.

Going to a local college for lectures is not an option.

I want some recommendations on good math books on: calculus, discrete math, probabilities, statistics

I'd also like to start from a comfortable level, so I prefer books that at least start with an introduction of the prerequisites.

46 comments

order
[+] caffeine|17 years ago|reply
Hi - I was in a similar situation a year ago (starting a Master's in ML, feeling crappy at math). What worked for me was:

- Studying is doing problems. Reading is not studying. When studying, read the minimum amount necessary to complete the problem. Do all problems CLOSED-book: no notes or reading while doing problems.

- A problem is not understood until you can do it on a blank piece of paper two days after reading the relevant material (so do problems several times to check).

- Study machine learning problems that use the math. The examples in math text books are pretty useless to you - do ML problems directly. When you don't get it, look the math up quickly in a reference book (A.E.M. by Kreyszig works), as much as needed to complete the problem, and move on.

- Implement regularly. Once you've done the problems concerning an algorithm or technique, implement it. Make sure your implementation is correct, and spend some effort making it quick. Read source code for other people's implementations.

This method is effective (for me) because I like programming and machine learning, I don't like math textbooks or the problems within, and I can't remember what I haven't solved.

Book recommendation: David MacKay's (free) book: Information Theory, Inference and Learning Algorithms (http://www.inference.phy.cam.ac.uk/mackay/itila/)

- Enjoyable: not terse or uptight, engaging reading.

- Problem-focused: the text is really explanations between interesting problems. The problems are spread throughout the text rather than bunched at the end - this is a huge advantage.

- Deep and unifying: You don't get this perspective often. Helps you think about AI/ML from a strong theoretical perspective. Will give you lots of "Aha!" moments.

- Math: Plenty of math, but not unnecessarily difficult. The practice you'll get doing these problems will serve you in other situations.

[+] pz|17 years ago|reply
i second the David MacKay recommendation. its a real pleasure to read. for me personally, his coverage of the fundamentals of information theory was amazing. the proofs are easy to follow and explained well so you can match the intuition to formula.

i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization (available here http://www.stanford.edu/~boyd/cvxbook/). This is maybe a little less fun but very deep and still well written and 'illustrated'.

My background is in ML and I came to realize over time that a lot of ML is just statistics and optimization theory rehashed by computer scientist.

[+] sanand|17 years ago|reply
My recommendations: Hardy, A Course of Pure Mathematics. Knuth, Graham and Patashnik, Concrete Mathematics. Hoffman and Kunze, Linear Algebra (there are several free online books on Linear Algebra and Vector Spaces, Strang, Axler, Lang Halmos and Shilov are also recommended). Feller, An Introduction to Probability Theory and Its Applications.

If you're not studying for exams and you have plenty of time on your hands, try the following algorithm: 1. Dive into some random chapter in the book and attempt the first exercise in that chapter. 2. Understand the problem by referring back and learning everything that might potentially have relevance to the exercise. 3. Attempt to solve exercise. 4. If unsuccessful, go to step 2. 5. If there are more exercises in the chapter, pick the next exercise and go to step 2. 6. If there are still unsolved exercises in the book, go to step 1.

Initially, this can cause excruciating pain and frustration. But every exercise is a little research problem and soon -- if you are the curious type who cannot rest until you have got to the bottom of a problem -- you'll find you've mastered the subject faster than you would have learnt its elements, had you taken a formal classroom course. The most important benefit of this method is that it will train you in some modes of thinking that are crucial in research. Think of it as callisthenics for problem solving type of research!

[+] rw|17 years ago|reply
A few suggestions (if only we all had more time):

Do ProjectEuler.net, as it provides a great mesh between algorithm design and mathematics; it might be within your expertise already, however. Look in to getting a good discrete math book or two. My circumstances led me to not use my discrete math book very much for class, so I won't give you a recommendation that I do not have.

Go through Scholarpedia. They are doing cool work over there.

Get some text(s) on statistical inference, especially Bayesian learning. If you have access to a good library system (say, state-wide library loan), you ought to have no problem getting most textbooks.

Seeing as how you're into NLP and AI, look at concurrency and what, say, Google is doing with MapReduce and PageRank (they publish reasonably-understandable papers). In general, push yourself to read peer-reviewed journal articles on the topics of your interest.

Peruse some of the organizations that do research into the philosophical and human issues surrounding AI and technology. For example, there's Less Wrong at lesswrong.com. They can come off as quite hubristic, but if you drill through the latent groupthink and individual over-assertiveness, you'll find a gigantic wealth of information on both what rationality is, and how to apply it to your life.

Additionally, check out (I've mentioned these before, so I'll quote myself verbatim):

"Problems On Algorithms" (Parberry, Gasarch): A slim volume containing ~700 exercises on proving various properties of algorithms. And it teaches you most of what is needed. Download it from: http://www.eng.unt.edu/ian/books/free/

"Elements of the Theory of Computation" (Papadimitriou): Classic text. Sure you've heard of Turing machines before, but what do you know about mu-recursive functions? It's very beautiful and if you have a person with whom to read the book and do the exercises, you will gain immensely. It will help unify, in your mind, many of the myriad articles you've probably read on Wikipedia about formal languages and complexity theory.

[+] lkozma|17 years ago|reply
You could read Gilbert Strang's Linear Algebra books, there are also video lectures available...

Gelman's Bayesian Data Analysis is a good statistical intro with a Bayesian bend,

maybe Freedman's Statistical models,

read some general machine learning intro book, such as Alpaydin or Bishop for a more probabilistic view.

For NLP especially, you'll need some information theory. Maybe start with MacKay's Information Theory, Inference, etc. (it is freely available online), very entertaining and accessible book and you can follow the pointers there for more details.

For general data mining and machine learning you need some basics of optimization, pick up any standard book on convex optimization to get an idea of the basic problems and methods.

that's it at a first try :)

[+] caffeine|17 years ago|reply
Don't forget Hastie & Tibshirani's Elements of Statistical Learning! :)
[+] chollida1|17 years ago|reply
I've seen a few people recommend The Princeton Companion to Mathematics(PCM) http://press.princeton.edu/titles/8350

I own this book and while I do agree it's a great book for getting a 10000 foot view about topics in mathematics it's not really a book one read to learn the math that the poster is asking for.

I agree with another post here that said studying is doing problems and being able to do them again a week later without having to consult the text again.

PCM isnt a good book for learning calculus, discrete math, probability or statistics( what the posters wants to learn) simply because it isn't designed to teach them. There are no problems/solutions in the book to try.

If you really want one book then I'd recommend Concrete Mathamatics http://www.amazon.com/Concrete-Mathematics-Foundation-Comput... by Graham, Knuth and Patashnik. It has relevant topics and problem sets that you can work through to see if you truly did learn anything when reading it.

[+] J_McQuade|17 years ago|reply
Well, if you're looking for an introduction to the basics of mathematics (which I would recommend for anyone looking to delve deeper into any specific area), you can't really beat 'How to Prove It' by Velleman. It covers the fundamental concepts of mathematical analysis and proof-writing (and, perhaps more importantly, reading proofs) and will help give you a great grounding for further reading, especially in the areas of discrete mathematics, set theory, and probability. I can't recommend it enough as an enjoyable introduction to the field.

Also, for a good probability textbook, I would point you to 'Probability and Random Processes' by Grimmett and Stirzaker. It may be a bit hard-going at first (which is why I recommend How to Prove It to get you used to rigorous maths), but it is a very thorough text that also happens to concentrate on many areas of interest to the average programmer (Markov chains and the like). Together with the companion book of exercises, it should be fantastic for self-teaching, too.

In calculus, you have so many options that I can only really suggest you go out and find one that you like. I used 'Early Transcendentals' by Stewart in my first year Calculus lectures and it's served as a good reference ever since, particularly for basic vector / multi-dimensional calculus. But, like I say, just look around.

[+] rw|17 years ago|reply
I second "How To Prove It".
[+] ionfish|17 years ago|reply
Velleman's book with Alexander George, Philosophies of Mathematics, is a good introduction to that subject, and has a nice walkthrough of ZFC.
[+] yoshimo111|17 years ago|reply
http://www.thatquiz.org/ for practice problems.

http://en.wikipedia.org/wiki/Mathomatic an open source math solver, will also output answers to programming code, can solve polynomials, do calculus, trig...etc. You can use it to check your answers.

http://www.math.com/ the website has a lot of topics.

You don't neccessarily 'need' a textbook. You can probably find the information you seek all online. If you have any questions you can ask yahoo answers.

http://tutorial.math.lamar.edu/links.aspx http://etc.usf.edu/math/index.htm an overwhelming list of math sites. SOS math has worked well at helping me in the past.

[+] joubert|17 years ago|reply
The Princeton Companion to Mathematics.
[+] tptacek|17 years ago|reply
What do people love about this book so much? (I haven't read it).
[+] jonsen|17 years ago|reply
I got i two days ago and have skimmed around: Absolutely recommendable!
[+] kylec|17 years ago|reply
OK, I'm about to order it, but I feel as though I might be a little unprepared for its contents - is there another book that I might be able to use to warm up?
[+] AlleyRow|17 years ago|reply
Check Out "Mathematical Tools for Physics" by James Nearing (available as a free download at http://www.physics.miami.edu/~nearing/mathmethods/). It covers a lot of great results and is one of the most easy to follow comprehensive summaries of an undergraduate math education that I've come across.
[+] pasbesoin|17 years ago|reply
Note also the links at the bottom of that page, to other books and sites listing freely available books.
[+] fogus|17 years ago|reply
I personally liked "A Logical Approach to Discrete Math" by Gries and Schneider which can be had for a good price on Amazon. As for statustics, I liked "Statistics: A Gentle Introduction" by Coolidge and "Principles of Statistics" by Bullmer which can be had for a great price even new.

Although only slightly related to your chosen topics, "Linear Algebra Done Right" by Axler is awesome.

-m

[+] jimbokun|17 years ago|reply
I just happened to find this book browsing through my campus library a few days ago:

Basic Training in Mathematics

A Fitness Program for Science Students

Shankar, R.

http://www.springer.com/physics/book/978-0-306-45036-5

Meant as a review/primer for undergraduates in the physical sciences and engineering. Mainly covers topics in calculus and linear algebra. Does a fairly good job of covering the connections between ideas and some of the intuition behind ideas, while keeping things pretty concise. That is my review after finishing the first chapter, which is as far as I've got.

The "Basic Training" perspective peeks through in comments like:

"Trigonometric functions, identities and derivatives, radian measures for angles. These will not be listed here since you must have already learned them by heart as a child."

So don't expect to be overly molly-coddled by this book.

I'm reading it for many of the same reasons you describe. Taking a machine learning course in the Fall, want to brush up on the underlying mathematics involved. Statistics, calculus and linear algebra are pervasive in most of the ML books I have looked at. I note that you did not list linear algebra, but that is the area where I have been spending the most time, mainly because I never took a course in it, where the other topics I have at least some background I can review.

[+] hamilton|17 years ago|reply
Everyone has given really great recommendations. I second caffeine's PROCESS. The way I learned most mathematics is from working through problems on a white board I bought. From there I got a great grasp of probability theory and linear algebra, which gave me the legs to study statistical learning.

While I have nothing to add on the math front, on the intuition / implementation front you should definitely check out Chris Manning's (currently) free textbook on Information Retrieval. He delves into the history, technical details, and state of the art of IR. Let's hope he opens up his NLP book as well (great CS prof at Stanford).

If humanly possible, I'd recommend making a friend who can help you solve confusion or quandary about certain problems. This is probably less likely for most people, but if you DO know someone with domain knowledge, I'd buy them dinner and pick their brain. I've fully traversed the learning curve from both wrestling with problems and talking to people much smarter than me.

http://www-csli.stanford.edu/~hinrich/information-retrieval-...

[+] ananthrk|17 years ago|reply
(Given that you are also interested in NLP) "Foundations of Statistical Natural Language Processing" is a good bet.
[+] menloparkbum|17 years ago|reply
I want some recommendations on good math books on: calculus, discrete math, probabilities, statistics

Calculus Made Easy by Silvanius P. Thompson (Feynman taught himself calc with this book)

Concrete Mathematics by Knuth, Graham, Patashnik

Fundamentals of Applied Probability Theory by Alvin Drake

You probably want to learn Linear Algebra - get Gilbert Strang's book.

The Schuam's outlines in each topic are good for review. I like the ancient ones, usually there are dozens of them for next to nothing at any good used bookstore.