My favorite Knuth quote: "Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration. I try to learn certain areas of computer science exhaustively; then I try to digest that knowledge into a form that is accessible to people who don't have time for such study."
Not that email or any of the million other apps don't produce constant interruptions - but in terms of what I want my role to be and what I find pleasure in.
A world of high stimulation and constant social interaction? A million things shouting at me and I have to sort through them and grab the most important one and hop from thing to thing fast? That's what I like.
Focusing on one topic for hours, days, even weeks to really understanding every facet of it? I think it'd kill me. I know people theoretically need these big blocks of uninterrupted time to get work done, but I'd much rather have a bunch of meetings where we can figure out the path forward.
I have mad respect for him... but I never could have been an academic.
I like to take in all those digested kernels of knowledge, I like to digest as many of them as I can; I know I lose the details, but integrating them all into something bigger has always been my dream; it's just how I live my life.
Stanford CS has at least a couple of people who seem to be really decent humans as well as great computer scientists. Besides Don Knuth, John Ousterhout is another person who comes to mind. Both of them have a humility combined with humor that I find quite admirable.
My favorite Knuth story, attributed to Alan Kay (if you're around, would love confirmation):
When I was at Stanford with the AI project [in the late 1960s] one of the things we used to do every Thanksgiving is have a computer programming contest with people on research projects in the Bay area. The prize I think was a turkey.
[John] McCarthy used to make up the problems. The one year that Knuth entered this, he won both the fastest time getting the program running and he also won the fastest execution of the algorithm. He did it on the worst system with remote batch called the Wilbur system. And he basically beat the shit out of everyone.
And they asked him, "How could you possibly do this?" And he answered, "When I learned to program, you were lucky if you got five minutes with the machine a day. If you wanted to get the program going, it just had to be written right. So people just learned to program like it was carving stone. You sort of have to sidle up to it. That's how I learned to program."
I’ve posted this comment on HN before, but this is my best Knuth story:
In the 70's I had a co-worker, perhaps the best programmer in the department, that had gone to school with Knuth. He told me that one day while in college Knuth was using one of the available key-punch machines to punch his program on cards. My friend was ready to punch his program so he stood nearby to wait for Knuth to finish. Knuth, working on a big program, offered to Keypunch my friends program before finishing his own because my friend's program was shorter and Knuth could keypunch quite fast.
While watching over Knuth's shoulder, my friend noticed Knuth speeding up and slowing down at irregular intervals. Later he asked him about that and Knuth replied that he was fixing the bugs in my friend's Fortran as he punched it out.
> He did it on the worst system with remote batch called the Wilbur system.
I think you mean WYLBUR.
I had the "opportunity" to work with WYLBUR once in 1993, and I remember it to this day. "the worst system with remote batch" dramatically understates how bad it was. Hearing this raises Knuth even higher in my estimation.
The tl;dr is that Knuth wrote an elaborate implementation of a program to solve a particular problem, and Doug McIlroy replaced it entirely with a six step shell pipeline. (Knuth's program was written using his literate programming tools, could be typeset in TeX, and involved some precise work with data structures and algorithms.)
I love this story as an example both of Knuth's genius and perspective, but also as a way to show what his level of dedication can achieve. It's an amazing intellectual accomplishment.
I also love this story as a demonstration what those of us without that skill and dedication can achieve using the advancements built on the work of Knuth and others.
Knuth has said that the story as told above is apocryphal. I quote from here (http://www.informit.com/articles/article.aspx?p=1193856):
> Donald: The story you heard is typical of legends that are based on only a small kernel of truth. Here’s what actually happened: John McCarthy decided in 1971 to have a Memorial Day Programming Race. All of the contestants except me worked at his AI Lab up in the hills above Stanford, using the WAITS time-sharing system; I was down on the main campus, where the only computer available to me was a mainframe for which I had to punch cards and submit them for processing in batch mode. I used Wirth’s ALGOL W system (the predecessor of Pascal). My program didn’t work the first time, but fortunately I could use Ed Satterthwaite’s excellent offline debugging system for ALGOL W, so I needed only two runs. Meanwhile, the folks using WAITS couldn’t get enough machine cycles because their machine was so overloaded. (I think that the second-place finisher, using that "modern" approach, came in about an hour after I had submitted the winning entry with old-fangled methods.) It wasn’t a fair contest.
> As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."
Most people who learned to program on batch-oriented systems developed those habits. When you had to wait hours or even overnight for your compile to run, you spent a lot of time at your desk checking the code for syntax errors, and running the logic in your head.
I once had to write a program for a class didn't have access to a compiler. I wrote it all in notepad and double checked everything more thoroughly than I ever would have done otherwise. When I got the computer lab the compiler found one or two type-o's and the program ran perfectly the first time. The experience changed how I write code. It's worth trying out at least one time.
What a wonderful and genuine human being! It helps explain why his books are so incredibly humane while remaining deep on a very broad topic. Thanks for sharing. ️
One of the first times I was ever asked about the title of my books was in 1966, during the last previous ACM national meeting held in Southern California. This was before any of the books were published, and I recall having lunch with a friend at the convention hotel. He knew how conceited I was, already at that time, so he asked if I was going to call my books "An Introduction to Don Knuth." I replied that, on the contrary, I was naming the books after him. His name: Art Evans. (The Art of Computer Programming, in person.)
"I was sitting in Steve's office when Lynn Takahashi, Steve's assistant, announced Knuth's arrival. Steve bounced out of his chair, bounded over to the door and extended a welcoming hand.
"It's a pleasure to meet you, Professor Knuth," Steve said. "I've read all of your books."
No way; I worked for him from 1978-1986, and never heard him say a naughty word. Also, I tagged along when he went to Apple to meet Jobs, who wanted to show him a late prototype of the Mac, and don’t recall hearing any such thing at the time (and it would have shocked me).
FWIW Knuth himself doesn't think this happened nor does he sound like the kind of person (to me) who would try to cut someone down like that: https://m.youtube.com/watch?v=zJOS0sV2a24&t=26m
When Java was new, James Gosling and I gave a talk about it to a group that included Don. When we gave the talk I was in the process of leaving Sun for a startup. A couple of years later Don mentioned to me that the talk was the first time we met and, at the time, he couldn't figure out if I was really smart or really stupid for leaving Sun right after Java shipped. In 2010 we both agreed that in the long term, it was the right choice for me, but neither he nor I could have predicted that at the time.
“I am worried that algorithms are getting too prominent in the world,” he added. “It started out that computer scientists were worried nobody was listening to us. Now I’m worried that too many people are listening.”
Ok, so this journalist gets to spend WHOLE day with Knuth and all we get is some bits of history and quotes from couple of Google guys. Massively lost opportunity to provide window in this legends daily life, thoughts and habits...
Donald Knuth shaved the ultimate yak. He wanted to write a book with mathematics in it so he implemented what for decades has been industry-standard mathematical typesetting system: TeX.
That's remarkable. I just wish he'd finish the book.
He wrote TeX82 (modern TeX) in WEB, a literate programming language he invented that could be tangled into the Pascal machine code or weaved into literate documentation in ... TeX itself.
In fact he wrote WEB itself in WEB and by hand compiled the output of the tangle program, which he ran on the tangle source to get the same output.
He also wrote the program METAFONT to produce the typefaces, which he used to design the Computer Modern font.
And then he used that to write instruction books for TeX and METAFONT as well as new editions of The Art of Computer Science and others.
TeX is still in regular use in the academic community (ported via Web2C) over 35 years later, despite the changes in technology in that time (with hacks to deal with modern fonts, outputting PDFs instead of DVI, including images and the like).
He taught the yak to shave itself, and it's still clean shaven 35 years later.
Knuth has enormous patience for someone so productive.
My son used to refer to Knuth as "that amazing harpsichord player" and didn't care about the computer science "stuff" I tried to tell him was really important. And he had plenty of time to sit down with a kid and go through the score of Fantasia Apocalyptica and laugh together at the musical jokes in the score.
OTOH I either have to consciously decide not to stress about all the things I want to get done or else...stress and push hard. Yet the things I feel driven to accomplish are quite minor compared to Knuth's to do list!
Knuth is just ONE section away from writing about my (current) topic of study: Constraint Programming.
His 4th volume of TAOCP is so big that its beginning for him to take decades to write sections. Backtracking search + 3-SAT solvers is really interesting (and very closely related to Constraint Programming), but I'm deeply curious to know what his take on Constraint Programming will be.
Its really odd to be "waiting" for the next chapter of TAOCP. Its a serious body of work that has spanned decades of research and writing... I really hope he finishes the next section sooner.
Maybe I'll go and buy the fascicle for 3SAT and read it while waiting.
In his recent Dancing Links talk (a couple of weeks ago), he mentioned a generalization of exact cover that also covers constraint programming. You may want to look into that (XCC) in the meantime, in Knuth's fascicle on Dancing Links (https://cs.stanford.edu/~knuth/fasc5c.ps.gz).
I know the feeling of waiting for new chapters of TAOCP; I'm waiting for Chapter 8 on recursion because his idea of it is so different from everyone else's.
Once met Yoda himself on the 59th street platform of the Metra next to the University of Chicago. He needed to get to the airport within 45 minutes. I told him he needed to call a cab. Now I get to tell people I once helped him optimize an algorithm. ;-P
I had the pleasure to attend his 80th birthday conference (http://knuth80.elfbrink.se) in Piteå, Sweden where he also premiered his piece Fantasia Apocalyptica for church organ.
We attended the conference as students but still got to see the conference, listen to the music and be at the birthday dinner free of charge. When I talked to Knuth I was amazed by his humility. Every presentation was about how grand he was, but he still was humble and friendly.
They say you shouldn't meet your heroes, but they're obviously not talking about Knuth!
>“Here at Google, sometimes we just throw stuff together,” Dr. Norvig said, during a meeting of the Google Trips team, in Mountain View, Calif. “But other times, if you’re serving billions of users, it’s important to do that efficiently. A 10-per-cent improvement in efficiency can work out to billions of dollars, and in order to get that last level of efficiency, you have to understand what’s going on all the way down.”
I'm interested in this low level systems opimization. What type of jobs hire for this work? What areas in graduate school should one focus in? Is this type of skill set still employable?
It's not necessarily low level optimization. You get better optimization results if you're willing to change things at all levels of the software stack (including user expectations); so it's really about understanding what's going on all the way down and all the way up.
As with all things, the way to get experience and knowledge is to try things. Take a look at some software that you use that does something slow, and try to make it faster; it's easier if you have the source available. Lots of applications these days are slow to start, and that's easy to time, and probably easy to bucket things into things that don't need to be done or things that can be done later. If you interact with any long data processing loops, those are often good choices for optimization as well.
This skill set is most employable for companies that have scale; when you have ten servers, being able to turn off one of them because of efficiency gains doesn't help that much; but it's nice to turn off ten percent of your 10,000 node cluster. It also leads to nice claims on resumes "made efficiency gain in X, leading to Y% cost savings on servers", and since you're likely to spend a bunch of time on a single issue, you'll be able to have a good discussion when it comes to 'discuss some of your interesting projects' in interviews.
The only downside is you'll be super frustrated about everyone else's software being slow. You'll likely gravitate towards classic games because they don't have load times; I hope you like pixel art. :D
There's a position like that in my company, open to new hires (only bachelor's needed). The goal is to optimize algorithms and data analysis models that run on our HPC cluster. It basically boils down to changing compiler settings and seeing what runs the fastest.
For example, you could read the C++ spec forwards and backwards for a year and know your data structure big O notation just as well, but they will never tell you that inserting into the middle of a vector is faster than inserting into the middle of a linked list in a great many common cases.
Sure, ideally you'd measure, but you need to have some idea of what options to even compare with your measurements.
Working in any of the big cloud services (AWS, Azure, GCP) will involve this sort of work. Some services probably have more interesting scaling problems than others. (e.g. I bet AWS DynamoDB is more interesting than AWS CloudTrail).
Hmpf, considering that the author spent a whole day with Knuth there is disappointingly little interviewing going on. Other people (e.g. Norvig) are quoted more extensively.
True Jeff Dean story time (this was directly related to me): when Jeff spoke at Stanford, it was so crowded, Knuth had to sit on the floor (I think somebody eventually noticed and offered him a seat).
[+] [-] maxtollenaar|7 years ago|reply
source: https://www-cs-faculty.stanford.edu/~knuth/email.html
[+] [-] freewilly1040|7 years ago|reply
[+] [-] JimboOmega|7 years ago|reply
Not that email or any of the million other apps don't produce constant interruptions - but in terms of what I want my role to be and what I find pleasure in.
A world of high stimulation and constant social interaction? A million things shouting at me and I have to sort through them and grab the most important one and hop from thing to thing fast? That's what I like.
Focusing on one topic for hours, days, even weeks to really understanding every facet of it? I think it'd kill me. I know people theoretically need these big blocks of uninterrupted time to get work done, but I'd much rather have a bunch of meetings where we can figure out the path forward.
I have mad respect for him... but I never could have been an academic.
I like to take in all those digested kernels of knowledge, I like to digest as many of them as I can; I know I lose the details, but integrating them all into something bigger has always been my dream; it's just how I live my life.
[+] [-] samstave|7 years ago|reply
Reminds me of a "hermit-like wizard in his tower in solitary uninterrupted study of their craft."
[+] [-] hodgesrm|7 years ago|reply
[+] [-] pjmorris|7 years ago|reply
When I was at Stanford with the AI project [in the late 1960s] one of the things we used to do every Thanksgiving is have a computer programming contest with people on research projects in the Bay area. The prize I think was a turkey.
[John] McCarthy used to make up the problems. The one year that Knuth entered this, he won both the fastest time getting the program running and he also won the fastest execution of the algorithm. He did it on the worst system with remote batch called the Wilbur system. And he basically beat the shit out of everyone.
And they asked him, "How could you possibly do this?" And he answered, "When I learned to program, you were lucky if you got five minutes with the machine a day. If you wanted to get the program going, it just had to be written right. So people just learned to program like it was carving stone. You sort of have to sidle up to it. That's how I learned to program."
[0] http://www.softpanorama.org/People/Knuth/index.shtml
[+] [-] todd8|7 years ago|reply
In the 70's I had a co-worker, perhaps the best programmer in the department, that had gone to school with Knuth. He told me that one day while in college Knuth was using one of the available key-punch machines to punch his program on cards. My friend was ready to punch his program so he stood nearby to wait for Knuth to finish. Knuth, working on a big program, offered to Keypunch my friends program before finishing his own because my friend's program was shorter and Knuth could keypunch quite fast.
While watching over Knuth's shoulder, my friend noticed Knuth speeding up and slowing down at irregular intervals. Later he asked him about that and Knuth replied that he was fixing the bugs in my friend's Fortran as he punched it out.
[+] [-] JackFr|7 years ago|reply
I think you mean WYLBUR.
I had the "opportunity" to work with WYLBUR once in 1993, and I remember it to this day. "the worst system with remote batch" dramatically understates how bad it was. Hearing this raises Knuth even higher in my estimation.
https://en.wikipedia.org/wiki/ORVYL_and_WYLBUR
[+] [-] mschaef|7 years ago|reply
http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...
The tl;dr is that Knuth wrote an elaborate implementation of a program to solve a particular problem, and Doug McIlroy replaced it entirely with a six step shell pipeline. (Knuth's program was written using his literate programming tools, could be typeset in TeX, and involved some precise work with data structures and algorithms.)
I love this story as an example both of Knuth's genius and perspective, but also as a way to show what his level of dedication can achieve. It's an amazing intellectual accomplishment.
I also love this story as a demonstration what those of us without that skill and dedication can achieve using the advancements built on the work of Knuth and others.
[+] [-] svat|7 years ago|reply
And about the story you shared, Knuth has elaborated in an interview (with typical humility, he plays it down): http://www.informit.com/articles/article.aspx?p=1193856
[+] [-] stenecdote|7 years ago|reply
> As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."
[+] [-] ams6110|7 years ago|reply
[+] [-] seanmceligot|7 years ago|reply
[+] [-] throwaway427|7 years ago|reply
[+] [-] widforss|7 years ago|reply
https://youtu.be/74BfHoE66rc?t=42m17s
[+] [-] inimino|7 years ago|reply
[+] [-] martythemaniak|7 years ago|reply
[+] [-] ravishi|7 years ago|reply
[+] [-] enriquto|7 years ago|reply
One of the first times I was ever asked about the title of my books was in 1966, during the last previous ACM national meeting held in Southern California. This was before any of the books were published, and I recall having lunch with a friend at the convention hotel. He knew how conceited I was, already at that time, so he asked if I was going to call my books "An Introduction to Don Knuth." I replied that, on the contrary, I was naming the books after him. His name: Art Evans. (The Art of Computer Programming, in person.)
[+] [-] theonething|7 years ago|reply
Some of his writings on the subject:
https://www-cs-faculty.stanford.edu/~knuth/things.html
https://www-cs-faculty.stanford.edu/~knuth/316.html
[+] [-] pmoriarty|7 years ago|reply
"It's a pleasure to meet you, Professor Knuth," Steve said. "I've read all of your books."
"You're full of shit," Knuth responded.
http://www.folklore.org/StoryView.py?project=Macintosh&story...
[+] [-] drfuchs|7 years ago|reply
[+] [-] Permit|7 years ago|reply
[+] [-] utopcell|7 years ago|reply
[+] [-] wglb|7 years ago|reply
[+] [-] ChuckMcM|7 years ago|reply
[+] [-] cschep|7 years ago|reply
cheers!
[+] [-] sytelus|7 years ago|reply
[+] [-] sparrish|7 years ago|reply
3:16 Bible Texts Illuminated
https://smile.amazon.com/3-16-Bible-Texts-Illuminated/dp/089...
[+] [-] pbiggar|7 years ago|reply
https://www.amazon.com/Things-Computer-Scientist-Rarely-Lect...
[+] [-] claar|7 years ago|reply
[+] [-] jordigh|7 years ago|reply
That's remarkable. I just wish he'd finish the book.
[+] [-] fantispug|7 years ago|reply
He wrote TeX82 (modern TeX) in WEB, a literate programming language he invented that could be tangled into the Pascal machine code or weaved into literate documentation in ... TeX itself.
In fact he wrote WEB itself in WEB and by hand compiled the output of the tangle program, which he ran on the tangle source to get the same output.
He also wrote the program METAFONT to produce the typefaces, which he used to design the Computer Modern font.
And then he used that to write instruction books for TeX and METAFONT as well as new editions of The Art of Computer Science and others.
TeX is still in regular use in the academic community (ported via Web2C) over 35 years later, despite the changes in technology in that time (with hacks to deal with modern fonts, outputting PDFs instead of DVI, including images and the like).
He taught the yak to shave itself, and it's still clean shaven 35 years later.
[+] [-] gumby|7 years ago|reply
My son used to refer to Knuth as "that amazing harpsichord player" and didn't care about the computer science "stuff" I tried to tell him was really important. And he had plenty of time to sit down with a kid and go through the score of Fantasia Apocalyptica and laugh together at the musical jokes in the score.
OTOH I either have to consciously decide not to stress about all the things I want to get done or else...stress and push hard. Yet the things I feel driven to accomplish are quite minor compared to Knuth's to do list!
[+] [-] dragontamer|7 years ago|reply
His 4th volume of TAOCP is so big that its beginning for him to take decades to write sections. Backtracking search + 3-SAT solvers is really interesting (and very closely related to Constraint Programming), but I'm deeply curious to know what his take on Constraint Programming will be.
Its really odd to be "waiting" for the next chapter of TAOCP. Its a serious body of work that has spanned decades of research and writing... I really hope he finishes the next section sooner.
Maybe I'll go and buy the fascicle for 3SAT and read it while waiting.
[+] [-] svat|7 years ago|reply
I know the feeling of waiting for new chapters of TAOCP; I'm waiting for Chapter 8 on recursion because his idea of it is so different from everyone else's.
[+] [-] riemannzeta|7 years ago|reply
[+] [-] pengstrom|7 years ago|reply
We attended the conference as students but still got to see the conference, listen to the music and be at the birthday dinner free of charge. When I talked to Knuth I was amazed by his humility. Every presentation was about how grand he was, but he still was humble and friendly.
They say you shouldn't meet your heroes, but they're obviously not talking about Knuth!
[+] [-] owenversteeg|7 years ago|reply
I love that quote.
[+] [-] melling|7 years ago|reply
https://youtu.be/dhh8Ao4yweQ
He was ready for Moneyball and sports data science decades before everyone else.
[+] [-] username33382|7 years ago|reply
I'm interested in this low level systems opimization. What type of jobs hire for this work? What areas in graduate school should one focus in? Is this type of skill set still employable?
[+] [-] toast0|7 years ago|reply
As with all things, the way to get experience and knowledge is to try things. Take a look at some software that you use that does something slow, and try to make it faster; it's easier if you have the source available. Lots of applications these days are slow to start, and that's easy to time, and probably easy to bucket things into things that don't need to be done or things that can be done later. If you interact with any long data processing loops, those are often good choices for optimization as well.
This skill set is most employable for companies that have scale; when you have ten servers, being able to turn off one of them because of efficiency gains doesn't help that much; but it's nice to turn off ten percent of your 10,000 node cluster. It also leads to nice claims on resumes "made efficiency gain in X, leading to Y% cost savings on servers", and since you're likely to spend a bunch of time on a single issue, you'll be able to have a good discussion when it comes to 'discuss some of your interesting projects' in interviews.
The only downside is you'll be super frustrated about everyone else's software being slow. You'll likely gravitate towards classic games because they don't have load times; I hope you like pixel art. :D
[+] [-] nwsm|7 years ago|reply
[+] [-] EliRivers|7 years ago|reply
For example, you could read the C++ spec forwards and backwards for a year and know your data structure big O notation just as well, but they will never tell you that inserting into the middle of a vector is faster than inserting into the middle of a linked list in a great many common cases.
Sure, ideally you'd measure, but you need to have some idea of what options to even compare with your measurements.
[+] [-] gamegoblin|7 years ago|reply
[+] [-] chrisweekly|7 years ago|reply
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] kieckerjan|7 years ago|reply
[+] [-] gowld|7 years ago|reply
[+] [-] dekhn|7 years ago|reply