Ask HN: Good books on machine learning?
I'm trying to learn about methods of machine learning. Do you guys have any suggestions for books or sources I should use?
4 comments
A very readable introduction to machine learning and recommendation systems is Programming Collective Intelligence, by Toby Seagram (O'Reilly). Worth looking at if you're just getting your feet wet.
There really isn't very much available on the practical side. So if you are looking to implement algorithms I suggest that you make use of the machine learning at ocw.mit.edu
Alternatively, if you want a good dose of a theoretical explanation of algorithms currently in use I highly recommend "Pattern Recognition and Machine Learning" by Christopher Bishop. It is definitely the best machine learning (and statistics) textbook that I have ever come across.
Alternatively, if you want a good dose of a theoretical explanation of algorithms currently in use I highly recommend "Pattern Recognition and Machine Learning" by Christopher Bishop. It is definitely the best machine learning (and statistics) textbook that I have ever come across.
I second the nomination of Bishop. It is the standard text. It is only two years old, and Bishop will teach you machine learning the way that the field practices it nowadays. In my lab of fourteen people, we must have six or so copies of Bishop.
I don't understand what is impractical about Bishop. If you are looking blindly to use an off-the-shelf machine learning implementation, that's one thing. Machine Learning has been described as the study of bias. If you want to understand when to pick certain techniques, and develop appropriate biases, then read Bishop.
"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman gives more of a statistician's approach. The treatment is simply less broad, and also more dated.
You can also look at Andrew Ng's video lectures: http://www.youtube.com/watch?v=UzxYlbK2c7E He is very well-respected in the field. For certain students, watching a lecture may be preferable to reading a book.
I don't understand what is impractical about Bishop. If you are looking blindly to use an off-the-shelf machine learning implementation, that's one thing. Machine Learning has been described as the study of bias. If you want to understand when to pick certain techniques, and develop appropriate biases, then read Bishop.
"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman gives more of a statistician's approach. The treatment is simply less broad, and also more dated.
You can also look at Andrew Ng's video lectures: http://www.youtube.com/watch?v=UzxYlbK2c7E He is very well-respected in the field. For certain students, watching a lecture may be preferable to reading a book.
A few other things (sorry, not to snipe too much! :) )
-I'm skeptical of the idea of a single "standard text" in such a fast-moving field. New machine learning techniques appear constantly and are often documented online years before they appear in books. Some computer scientists say they prefer conference proceedings over academic journals because the latter take so long.
-Further, I'm not sure that the goal of any text should be to cover topics X, Y and Z in any case, which doesn't seem possible for a book to do. What does seem feasible is to set up a framework for analyzing the performance of different techniques. So I'd like to hear a comparison of how Bishop does that vs. HTF.
-You're of course correct that HTF takes a statistician's POV on the field - the authors are all professors of statistics at Stanford. They are also accomplished - Friedman was a co-author on CART, for example. I would instead ask the question: what can you get out of the book and the framework it offers?
-I think that part of the framework in machine learning is to think about bias AND variance, and how to trade them off successfully. This is an important part of model selection, for example.
-I'm skeptical of the idea of a single "standard text" in such a fast-moving field. New machine learning techniques appear constantly and are often documented online years before they appear in books. Some computer scientists say they prefer conference proceedings over academic journals because the latter take so long.
-Further, I'm not sure that the goal of any text should be to cover topics X, Y and Z in any case, which doesn't seem possible for a book to do. What does seem feasible is to set up a framework for analyzing the performance of different techniques. So I'd like to hear a comparison of how Bishop does that vs. HTF.
-You're of course correct that HTF takes a statistician's POV on the field - the authors are all professors of statistics at Stanford. They are also accomplished - Friedman was a co-author on CART, for example. I would instead ask the question: what can you get out of the book and the framework it offers?
-I think that part of the framework in machine learning is to think about bias AND variance, and how to trade them off successfully. This is an important part of model selection, for example.
In terms of "datedness," I would point out that the second edition is now out:
This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates.
I'd be interested to hear what you mean by "simply less broad" as compared to Bishop's book, which (from having flipped through it) looks pretty comparable.
This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates.
I'd be interested to hear what you mean by "simply less broad" as compared to Bishop's book, which (from having flipped through it) looks pretty comparable.
Here are a set of links:
http://www.vetta.org/recommended-reading/
I'd second the recommendation of Bishop if you can hack the math, and also Elements of Statistical Learning, though I wouldn't attempt to learn techniques from the latter so much as look at a very interesting mathematical take on them.
gl
http://www.vetta.org/recommended-reading/
I'd second the recommendation of Bishop if you can hack the math, and also Elements of Statistical Learning, though I wouldn't attempt to learn techniques from the latter so much as look at a very interesting mathematical take on them.
gl
Try "The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman. Lots of math but an outstanding introduction.