Reading through and taking in "The Art of Computer Programming" seems to require massive amounts of time investment. In your opinion is the gems inside the text worth it?
Almost anywhere on the internet people will respond to this with basically "no, it's extremely inefficient to learn how to program this way", and they're right.
But.
If you want to learn _other_ things, these books are incredible. Maybe you want to learn deeply the combinatorial ideas behind data structures, or general techniques for squeezing small optimizations out of low level code. Or even maybe you want to see simple data structures used every-which-way to efficiently solve a huge range of mathematical problems. None of this is normal programming per se. I treat these as math books, and love them for it.
Like much of mathematics, it will make you a better programmer, but only accidentally.
This is a series more for engineers who want to delve into the depths of "computer science" that takes them to the magical world of mathematics. That way, its not worth if you are looking for direct real world application, like the CLRS or Sedgeick books provide.
That said, I think its worth having this for the humbling experience it provides. This experience is very much needed for every engineer, and specifically software engineers, because after a while of reapplying same old tropes to churn out solutions to problems, there creeps a feeling that we are at the top of our game, and "experts", when in reality most of us are only "experts" in a specific insipid framework or technology which are as permanent as the latest fashion trends. This series of books makes us feel worthless and keenly introspect. It makes us see that there is a vast array of knowledge that we severely lack. It builds personality and keeps us grounded. It separates true musicians from college band guitarists who shred the neck to cheering crowds.
I have Combinatorial Algorithms Part 1 Volume 4A and it makes me cry every time I peruse it.
If CLRS is an example of sonething that has direct real world application, then the other must be truly arcane and esoteric. My hatred for CLRS mostly stems from a completely terrible algorithms class I took which used it, ignored anything that might be useful, and focused 110% on proof-writing. For all the random crap that is covered in excruciating detail, there are some weird gaps where some things that are actually used in industry and have long been, don't get passing mention...
But Corman himself is also kind of a pretentious tool, so that doesn't help - he was certainly an abysmal freshman advisor.
I spent three years reading the first three and (attempting, at least) every exercise. I enjoyed it, but I don’t know if it made me a better programmer. I definitely did learn a lot, although maybe not that much that was really practical.
I have never found anything professionally useful in “Seminumerical Algorithms”, but Knuth’s writing is clear and concise that I go back reread the parts which I just found interesting.
(In particular I love why numbers with lower initial digits are more ‘common’ than others, and tests for randomness.)
It genuinely pains me to say this, as I've got a copy on my bookshelf, but no.
Reverence for it is religious/romantic in nature, and neither based on practical nor pedagogical means.
If you want to build practical skills and experience with algorithms, choose a problem domain involving them and dive straight into it.
If you want to expand your mind/skills, choose a programming language from a paradigm and do everything in it for a few years. Then once you've got it all figured out, choose a deliberately different paradigm and do the same thing. Repeat for 3-5 rounds.
Sure, there's nothing technically or factually wrong with the book. I had to look up some algorithms in it once for a reference and implemented it from that source, and Knuth writes clearly enough, but if someone wanted to learn english, or wanted to know how to improve their english, i similarly wouldn't direct them to read an old dictionary.
Edit 1: If i had to recommend books, aside from the observation that there's relatively few that i'd actually recommend with the goal of making one a better programmer explicitly, I've learnt more from lisp books and/or trying to apply languages to project euler or personal projects than I've ever picked up from TAOCP...
Edit 2: oh, and trying to improve and implement all the things that people tell you not to bother trying to improve and/or implement...
Think of the series like a detailed, focused encyclopedia. You can learn an incredible amount from reading and understanding it, but reading the whole thing is probably not especially efficient, and it will likely take a long time.
That said, you didn’t really set any goals; you just asked if it’s “worth the investment”. That’s very abstract, and to that I would say “yes”, but also “it’s very likely that you’re not at a point in your life that reading the series, especially cover to cover, is the best use of your time”.
> “it’s very likely that you’re not at a point in your life that reading the series, especially cover to cover, is the best use of your time”.
Genuinely curious, when would it be the best use of my time? Is there any scenario where "Read TAOCP and gain X superpowers" would hold true? I have been wanting to get into this for some time now because I like the idea of intellectual pursuit but I also hoped that some benefits, however incidental, would carry over into my career.
To know whether you'd enjoy it (which is the only measure of “worth it” I can think of in relation to this book), you could start with one of the recent sections on combinatorial algorithms (Volume 4A and part of Volume 4B have been published so far): http://www.cs.utsa.edu/~wagner/knuth/
For example, 7.2.1.2 on generating all permutations (http://www.cs.utsa.edu/~wagner/knuth/fasc2b.pdf) has a lot of cool mathematics, some low-language programs, some alphametic puzzles, .... Or 7.1.4 on Binary Decision Diagrams (http://www.cs.utsa.edu/~wagner/knuth/fasc1b.pdf) is on a little-known (IMO) data-structure presented in Knuth's unique way, which can solve many counting problems.
Stepping back a little, there are two ways you could read this book:
• Try to engage with the mathematics, try each exercise for at least half an hour before giving up and looking at the solution, etc. (Doing it this way can definitely be a massive time investment.) If you'd like to do this, and find the mathematics difficult, you should read Concrete Mathematics which Knuth co-wrote with Ron Graham and Oren Patashnik, which is a highly enjoyable book in itself.
> By the way, if you’ve ever thought about reading TAOCP, give it a try. A lot of people will tell you that it’s a reference work, and it’s not meant to be read straight through, but that isn’t true. The author has a clear point of view and a narrative and an idiosyncratic style, and the only thing that inhibits readability is the difficulty of the math. There’s an easy solution to that though: read until you get to math you don’t understand, then skip it and find the next section you can understand. Reading this way, I skip at least 80% of the book, but the remaining 20% is great!
You could even do the latter first, then the former. It's really up to you, and diving in and trying a chapter or two is the best way you can judge whether you'll find it an enriching experience. (BTW: Knuth mentions in some interview that he wanted to call the books “mathematical analysis of algorithms”... if you're thinking that the books are trying to define the art of computer programming and will make you a better programmer at the kind of programming tasks you're likely to encounter in a professional career, then it's probably not the best use of your time... which is why I mention reading it for yourself and seeing if you like it; IMO the books are really enjoyable and full of clarity and grace and humour and depth.)
Depends on your goals. For example, when writing a real-world application today, the standard library provides you with a "good enough" sorting and searching algorithms and dictionary structures. The whole 600+ pages of volume 3 are dedicated just to these topics. You aren't going to be a better "everyday" programmer after studying that book. (Though I'm not aware of any language that has radix sort in its stdlib :-))
However, they are a well of ideas as well as extensive treaties on "esoteric" topics (e.g., testing sequences for randomness, derivation of FFT, sorting networks, external sorting...)
Knuth has written that he plans to remove the section on external sorting from the next edition of vol 3 (if _that_ ever happens)… though think about the cache effects on modern CPUs: could we sort faster by thinking of the cache as "internal memory" and of RAM as a "tape or disk drive" and applying a decades old external sorting algorithm?
Similarly, most sorting routines rely only on less-than, but often your comparison routine returns -1, 0 or 1. I implemented string sorting in C++ using 3-way comparison (instead of just less-than) and it was actually faster!
The exercises and their answers are source of gems as well, even if you just look up the answer to use the result.
Also, I never read them cover-to cover. When I'm bored, I take one volume, pick a chapter/section and read it and skim through exercises, at the same time thinking of what that could be applied to.
The issue with thinking of RAM as a "tape or disk drive" is that a tape has linearly increasing access times, but RAM takes the same time to access any part, so this algorithm wouldn't be ideal. Even most of our external storage these days (flash, etc) is random-access. True O(n) access media are becoming more and more scarce, so I'm not surprised Knuth is removing it.
This book is not to accomplish a quick skills upgrade to boost your career. This book is meant to take you on a very long and extensive journey into fundamentals. Probably you will not make any more money from reading it in the short term, probably you will not even be a better programmer instantly. But with time, it will change the way you think and it will bring you a deep understanding of our subject's base.
TAOCP is no howto, no tutorial. For the subjects it covers, it is a meditational bible.
Either way, don't miss the parallel-psychology analog: Write your own.
Be it ever so humble, it will be yours and it will grow in leverage the more you test, update, organize, and refine it.
A good way to start is to write down things you've already learned about computer programming. Then get ready to start organizing and updating this information over time.
Pretty soon you'll find that you start on new projects faster and save lots of time and energy.
I keep mine in clear view on the shelf in the hope that its collected wisdom will radiate outward and suffuse into my code. Not happened yet, but perhaps it has a useful psychological effect as a shrine to algorithms; whenever I am tempted by a quick, cheap hack I see the books and am steered back to the righteous path.
Actually I usually just do the cheap hack anyway but it is reassuring to know that it is there.
Rather start with Concrete Mathematics (also by Knuth and friends). If you don’t like that book, you will probably hate TAOCP. It also covers the requisite mathematics that you’ll need for working through TAOCP.
I like Concrete Mathematics, and jumped into it while going through Chapter 1 of TAOCP. However, I wouldn't consider an opinion of CM when considering TAOCP. The first part of Chapter 1 of TAOCP is very math heavy. Concrete Mathematics was developed from a course meant to expand on that content and prepare students better for it. While the style is very similar between the two books (Knuth was an author of both, of course), the content is very different. TAOCP is probably more accessible to programmers who have an aversion or difficulty with the math in Concrete Mathematics and shouldn't turn away just because a math book is challenging or not their interest.
It's not really a massive time investment. Consider people you see every morning doing a sudoku or crossword on a patio at a coffee shop or on a subway. That's basically what TAOCP is to me, a past-time/hobby, leisurely going through the psets and trying to do things like patch an ancient algorithm or solve some basic number theory puzzles (Vol 2). Djb often refers and quotes the volumes in talks which is one reason why I started reading them, to understand how to hand roll and optimize assembly libraries https://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf
There is also the benefits of learning direct programming of hardware, for example MIXAL, used in the first 3 volumes (replaced by MMIXAL) is meant to be run without an operating system. Maybe you want to play around with direct control of a RISC-V chip or fpga or understand the quantum abstract machine Rigetti built and it's instruction language.
For me its worth reading just for entertainment. Personally I skim a lot of the maths, and I think most of the exercises are for people with maths degrees. I don't have a hope of doing them. Regardless, its a good read - very well written, fascinating if you have an interest in algorithms. And beautifully presented too.
Unless you're a computer scientist studying algorithms, probably not. But when I defended my Master's thesis in CS way back in 1983 - yes, I'm that old - my fiancee[1] gave me the three-volume set of Knuth as a graduation gift. I had used his algorithm for arbitrary precision arithmetic in my thesis project. I had figured out addition and subtraction trivially, and had gotten multiplication on my own, but division eluded me. I credit Knuth in part for my being able to graduate with my M.S. Also: my thesis advisor had found a mistake in an earlier edition and was credited in the notes in the back of one of the volumes. So: your mileage may vary. [1] Still married, 35 years at last count.
I suggest to work through some introductory chapters to learn MIX and then glance over the books to be aware what topics they have. Probably read some interesting chapters just to enjoy. So when you need to implement something that's covered, you can quickly read the information.
Reading it from start to end IMO is overkill, unless you want to be algorithm encyclopedia.
Though most developers can just skip it, because they won't need to implement any complex algorithm ever.
There are enough answers about why these kinds of books are not directly relevant to modern software engineering, but are good for putting said discipline in the context of computer science.
That being said, I think you CAN put all that you learn in those books into practice, even if it won't be in your job - try out competitive programming. Try TopCoder or CodeForces, or Google's Code Jam. I think Hackerrank is similar and heavily used for weeding out programmers during the early stages of interviews. As this article states: https://glenmccallum.com/2019/05/14/senior-developers-reject... this sort of "testing" could actually become even more common place for job applications because of how handy it is for recruiters.
But yeah, if you enjoy being competitive in general, consuming books like CLRS or TAOCP and then joining competitions will actually give you a good feel for a programming world very different from the usual, day to day development you'll do at your job.
Well I don't own it but have access to a copy from my office library.
I must say that reading it cover to cover is time taking but if you use it as a reference to lookup algorithms and stuff, it is really enlightening.
It's worth it to have it on your shelf just as a cheap way to remind yourself that you are in fact a real intellectual. Don't read it though, it's barely understandable even with a heavy math background.
This wouldn't be mentioned by Bill Gates “If you think you’re a really good programmer… read Art of Computer Programming… You should definitely send me a résumé if you can read the whole thing.”
Currently most of engineers have to do application development where you rarely come across of any need to use this knowledge.
But for being computer engineer and if you think all these coding challenges, the concept of these books is still relevant. Books on competitive coding take excerpt from that book only.
It is a daunting task to complete but it will show its worth over time.
> This wouldn't be mentioned by Bill Gates “If you think
> you’re a really good programmer… read Art of Computer
> Programming… You should definitely send me a résumé if you
> can read the whole thing.”
I don't think Bill meant it seriously....he never responded when I sent him my resume 20 years ago :-)
[+] [-] bbischof|6 years ago|reply
But.
If you want to learn _other_ things, these books are incredible. Maybe you want to learn deeply the combinatorial ideas behind data structures, or general techniques for squeezing small optimizations out of low level code. Or even maybe you want to see simple data structures used every-which-way to efficiently solve a huge range of mathematical problems. None of this is normal programming per se. I treat these as math books, and love them for it.
Like much of mathematics, it will make you a better programmer, but only accidentally.
[+] [-] vinayms|6 years ago|reply
That said, I think its worth having this for the humbling experience it provides. This experience is very much needed for every engineer, and specifically software engineers, because after a while of reapplying same old tropes to churn out solutions to problems, there creeps a feeling that we are at the top of our game, and "experts", when in reality most of us are only "experts" in a specific insipid framework or technology which are as permanent as the latest fashion trends. This series of books makes us feel worthless and keenly introspect. It makes us see that there is a vast array of knowledge that we severely lack. It builds personality and keeps us grounded. It separates true musicians from college band guitarists who shred the neck to cheering crowds.
I have Combinatorial Algorithms Part 1 Volume 4A and it makes me cry every time I peruse it.
[+] [-] thrower123|6 years ago|reply
But Corman himself is also kind of a pretentious tool, so that doesn't help - he was certainly an abysmal freshman advisor.
[+] [-] jpitz|6 years ago|reply
They may be extremely dense and hard to read, but I feel like you are hanging a lot of emotional baggage on a set of reference books.
[+] [-] musicale|6 years ago|reply
Is that a recommendation or anti-recommendation? Or both?
[+] [-] commandlinefan|6 years ago|reply
[+] [-] svat|6 years ago|reply
- Reflections on a year of reading Knuth https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic...
- Reflections on another year of reading Knuth https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic...
- Reflections on Three Years of Reading Knuth https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic...
[+] [-] JackFr|6 years ago|reply
(In particular I love why numbers with lower initial digits are more ‘common’ than others, and tests for randomness.)
[+] [-] thethirdone|6 years ago|reply
> Quicksort is O(log n)
Should be
> Quicksort is O(n log n)
[+] [-] ACow_Adonis|6 years ago|reply
Reverence for it is religious/romantic in nature, and neither based on practical nor pedagogical means.
If you want to build practical skills and experience with algorithms, choose a problem domain involving them and dive straight into it.
If you want to expand your mind/skills, choose a programming language from a paradigm and do everything in it for a few years. Then once you've got it all figured out, choose a deliberately different paradigm and do the same thing. Repeat for 3-5 rounds.
Sure, there's nothing technically or factually wrong with the book. I had to look up some algorithms in it once for a reference and implemented it from that source, and Knuth writes clearly enough, but if someone wanted to learn english, or wanted to know how to improve their english, i similarly wouldn't direct them to read an old dictionary.
Edit 1: If i had to recommend books, aside from the observation that there's relatively few that i'd actually recommend with the goal of making one a better programmer explicitly, I've learnt more from lisp books and/or trying to apply languages to project euler or personal projects than I've ever picked up from TAOCP...
Edit 2: oh, and trying to improve and implement all the things that people tell you not to bother trying to improve and/or implement...
[+] [-] c256|6 years ago|reply
That said, you didn’t really set any goals; you just asked if it’s “worth the investment”. That’s very abstract, and to that I would say “yes”, but also “it’s very likely that you’re not at a point in your life that reading the series, especially cover to cover, is the best use of your time”.
Hope that helps.
[+] [-] ashwnacharya2|6 years ago|reply
Genuinely curious, when would it be the best use of my time? Is there any scenario where "Read TAOCP and gain X superpowers" would hold true? I have been wanting to get into this for some time now because I like the idea of intellectual pursuit but I also hoped that some benefits, however incidental, would carry over into my career.
[+] [-] svat|6 years ago|reply
For example, 7.2.1.2 on generating all permutations (http://www.cs.utsa.edu/~wagner/knuth/fasc2b.pdf) has a lot of cool mathematics, some low-language programs, some alphametic puzzles, .... Or 7.1.4 on Binary Decision Diagrams (http://www.cs.utsa.edu/~wagner/knuth/fasc1b.pdf) is on a little-known (IMO) data-structure presented in Knuth's unique way, which can solve many counting problems.
Stepping back a little, there are two ways you could read this book:
• Try to engage with the mathematics, try each exercise for at least half an hour before giving up and looking at the solution, etc. (Doing it this way can definitely be a massive time investment.) If you'd like to do this, and find the mathematics difficult, you should read Concrete Mathematics which Knuth co-wrote with Ron Graham and Oren Patashnik, which is a highly enjoyable book in itself.
• As mentioned in a recent post (https://nickdrozd.github.io/2019/05/17/knuth-check.html):
> By the way, if you’ve ever thought about reading TAOCP, give it a try. A lot of people will tell you that it’s a reference work, and it’s not meant to be read straight through, but that isn’t true. The author has a clear point of view and a narrative and an idiosyncratic style, and the only thing that inhibits readability is the difficulty of the math. There’s an easy solution to that though: read until you get to math you don’t understand, then skip it and find the next section you can understand. Reading this way, I skip at least 80% of the book, but the remaining 20% is great!
You could even do the latter first, then the former. It's really up to you, and diving in and trying a chapter or two is the best way you can judge whether you'll find it an enriching experience. (BTW: Knuth mentions in some interview that he wanted to call the books “mathematical analysis of algorithms”... if you're thinking that the books are trying to define the art of computer programming and will make you a better programmer at the kind of programming tasks you're likely to encounter in a professional career, then it's probably not the best use of your time... which is why I mention reading it for yourself and seeing if you like it; IMO the books are really enjoyable and full of clarity and grace and humour and depth.)
[+] [-] zvrba|6 years ago|reply
However, they are a well of ideas as well as extensive treaties on "esoteric" topics (e.g., testing sequences for randomness, derivation of FFT, sorting networks, external sorting...)
Knuth has written that he plans to remove the section on external sorting from the next edition of vol 3 (if _that_ ever happens)… though think about the cache effects on modern CPUs: could we sort faster by thinking of the cache as "internal memory" and of RAM as a "tape or disk drive" and applying a decades old external sorting algorithm?
Similarly, most sorting routines rely only on less-than, but often your comparison routine returns -1, 0 or 1. I implemented string sorting in C++ using 3-way comparison (instead of just less-than) and it was actually faster!
The exercises and their answers are source of gems as well, even if you just look up the answer to use the result.
Also, I never read them cover-to cover. When I'm bored, I take one volume, pick a chapter/section and read it and skim through exercises, at the same time thinking of what that could be applied to.
[+] [-] KingMob|6 years ago|reply
[+] [-] geff82|6 years ago|reply
TAOCP is no howto, no tutorial. For the subjects it covers, it is a meditational bible.
[+] [-] themodelplumber|6 years ago|reply
Be it ever so humble, it will be yours and it will grow in leverage the more you test, update, organize, and refine it.
A good way to start is to write down things you've already learned about computer programming. Then get ready to start organizing and updating this information over time.
Pretty soon you'll find that you start on new projects faster and save lots of time and energy.
[+] [-] nineteen999|6 years ago|reply
[+] [-] diegolo|6 years ago|reply
[+] [-] silverdemon|6 years ago|reply
Actually I usually just do the cheap hack anyway but it is reassuring to know that it is there.
[+] [-] laichzeit0|6 years ago|reply
[+] [-] Jtsummers|6 years ago|reply
[+] [-] hackermailman|6 years ago|reply
There is also the benefits of learning direct programming of hardware, for example MIXAL, used in the first 3 volumes (replaced by MMIXAL) is meant to be run without an operating system. Maybe you want to play around with direct control of a RISC-V chip or fpga or understand the quantum abstract machine Rigetti built and it's instruction language.
[+] [-] bloat|6 years ago|reply
[+] [-] coverclock|6 years ago|reply
[+] [-] vbezhenar|6 years ago|reply
Reading it from start to end IMO is overkill, unless you want to be algorithm encyclopedia.
Though most developers can just skip it, because they won't need to implement any complex algorithm ever.
[+] [-] xenocratus|6 years ago|reply
That being said, I think you CAN put all that you learn in those books into practice, even if it won't be in your job - try out competitive programming. Try TopCoder or CodeForces, or Google's Code Jam. I think Hackerrank is similar and heavily used for weeding out programmers during the early stages of interviews. As this article states: https://glenmccallum.com/2019/05/14/senior-developers-reject... this sort of "testing" could actually become even more common place for job applications because of how handy it is for recruiters.
But yeah, if you enjoy being competitive in general, consuming books like CLRS or TAOCP and then joining competitions will actually give you a good feel for a programming world very different from the usual, day to day development you'll do at your job.
[+] [-] gcells|6 years ago|reply
[+] [-] timwaagh|6 years ago|reply
[+] [-] quickthrower2|6 years ago|reply
[+] [-] sneak|6 years ago|reply
In the monetary sense? Depends highly on your particular specialization.
In the intellectual sense? I own a copy, have read selected chapters. Perhaps. Are you planning on doing all the exercises, as if in a class?
[+] [-] abby_3017|6 years ago|reply
This wouldn't be mentioned by Bill Gates “If you think you’re a really good programmer… read Art of Computer Programming… You should definitely send me a résumé if you can read the whole thing.”
Currently most of engineers have to do application development where you rarely come across of any need to use this knowledge. But for being computer engineer and if you think all these coding challenges, the concept of these books is still relevant. Books on competitive coding take excerpt from that book only.
It is a daunting task to complete but it will show its worth over time.
[+] [-] jason_slack|6 years ago|reply
I don't think Bill meant it seriously....he never responded when I sent him my resume 20 years ago :-)