Thanks for mentioning my books! I wrote my first Common Lisp book about the same time Peter wrote his book and I met him at the first Lisp Users meetup in San Diego. His Python short Python programs and notebooks are also fantastic. Off topic, but: I wonder if the new Python++ language Mojo that sits on top of MLIR will end up as the top AI language. Maybe?
This might be my favorite programming book of all time.
It's the book that got me hooked on Common Lisp (from Scheme) too. Focusing less on the elegance of the language definition and more on writing elegant programs.
(English isn't as elegantly defined as Lojban and but that didn't hold back Tolkien.)
Lojban is elegant from I guess a logical/mathematical point of view, but I don't think from a human point of view. Iirc "in the land of invented languages", the author pointed out many issues with the language in that even it's biggest fans can't really speak it fluently...so what's the point? Esperanto is probably a better bar here, which I just started reading the Hobbit in lol. I will say that Lojbahn is super cool though. The attitudinals are particularly interesting.
Me too! I came looking for this sort of meta-comment. I've re-read the paper book a few times now.
What makes it my favorite is how clear Norvig's writing is. It's easy to follow (both when reading it in English, and when following its execution if you're a programmer), and it introduces important ideas so effortlessly that, years later, it will give you a chuckle.
Anyone interested in clearly communicating about technical topics, and with a knowledge of Lisp's nature and some idea of what programming in 1991 looked like, might be tickled to read Chapter 1; even its first few paragraphs are refreshing.
I think everyone would be better off surveying the AI classics before diving head first into ML. It's the nuclear option for problem solving; and sometimes you can get away with simpler approaches that are easier to implement, reason about and debug.
Currently, statistical/data-driven approaches work best, and that's what you will be expected to use whether you are building your own products, or working for an employer. Most people don't care about the GOFAI approaches anymore, seeing them as outmoded in all respects.
However, if you are curious and want to understand more of the history of approaches we have tried, and learn some really interesting algorithms along the way, I think studying the old school problems and their solutions can be both intellectually stimulating, and potentially increase your depth of understanding. After all, it's only once you've tried to solve a problem and failed miserably that you start to appreciate the depth of its complexity.
That depth of appreciation is sorely lacking in today's new cohorts, who are basically blinded by the incredibly convincing outputs of our cream-of-the-crop LLMs.
I'm a huge fan of classical AI, and adore PAIP, but this isn't really true if your goal is anything other than a deep understanding of AI in the most general sense.
While it would be great if everyone interested in the topic was well versed in the fundamentals, the truth is if you want to do anything from building something cool over the weekend to getting an actual job doing AI work, you're much better off starting not only with ML, but specifically with current SotA neural networks.
If you really want to get started in AI I highly recommend building even a trivial implementation of Stable Diffusion on your own. Not just because it's cool, but because at its heart it is an excellent demonstration of how current differentiable programming works. Diffusion models involve chaining together 3 separate models into an entire system that learns to solve a complex task. Once you understand this deeply, you can now solve a very broad range of tricky problems and are really approaching what we think of when we think of AI.
Differentiable programming is really the current pathway to any sort of AI solution to a problem.
I say this as the token "have you tried logistic regression?" guy in my org.
> I think everyone would be better off surveying the AI classics before diving head first into ML
this is untrue. ml algorithms have nothing to do with gofai algorithms. if there is something one "would be better off surveying before diving head first into ML" it would be mathematical analysis, statistics, and probability
There's nothing intrinsically nuclear about ML and it includes a bunch of simple approaches too.
The advice to spend your limited time and attention on outdated approaches seems counterproductive. The things in this book aren't just old - they ended up being a dead end in research. So if it's 2023 and you have 20 hours to learn something new, you can do much better than this book.
It’s always quite ironic to see hipsters today praise GOFAI systems and belittle the deep learning ´hype’, given that they were massively overhyped at the time and delivered next to nothing outside of some niches.
Even funnier to see how someone is always quick to explain that ´NN are not real AI’ when GOFAI was literally all about parsing, basic logic and search trees.
I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former?
Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods.
The company I work for makes a killing on applying a 1980s-era ML technique to a really tough modern business problem -- the resulting product is probably the best in the world at what it's doing.
Old techniques have several things going for them, with one of the more important ones for us being explainability. A random person off the street could hypothetically, with an hour or two of training, diagnose problems just by looking at the structure of the model. That's very helpful for adapting to market needs.
Generality is another big plus. Since the model encodes intuitive ideas there's a lot of room for using it in innovative ways.
Older techniques also tend to produce better results with less data, because big data wasn't as much of a thing back then.
Unless you have a crazy amount of resources, I think it's far better to be bleeding edge in as few things as possible. Solving a new business problem? Perhaps don't spend too much time on also solving all the childhood diseases of a new technology.
This book is a good intro to Lisp itself, which is worth knowing in any case- and symbolic AI is probably one of the best domains for it!
As for GOFAI in the age of DL/LLM, yes, you should know it, for a couple of reasons. A lot of these techniques aren't really considered "AI" anymore, they're just regular CS algorithms everybody should know: graph search, backtracking, optimization, parsing, etc. The other is that a lot of newer DL/LLM is actually going back to these old problems, but bringing all the new techniques to deal with limitations of the classical algorithms.
> Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods.
Same for me.
> I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former?
I think it still matters. Plenty of examples in the tech industry where "old" tech/paradigms became the new "hype". They say it's all a cycle.
Think of this as a "learn to program better" book, even if you don't care about GOFAI. It has a lot of very nice code, and commentary without blather. I've said this about as long as I knew of it, so this isn't just revisionism.
It depends on what you were expecting from GOFAI. The future state is likely to be a combination of approaches where each makes sense rather than a single one. We're still in the honeymoon phase with deep learning and disillusionment with its ultimate limitations is likely still to come.
With modern and trendy music, should we listen to old and classical music. Should we forget the history? I think not. My favourite genres are Jazz and classical.
Sorry if you don't get the metaphor, but it's like so.
If it's for work or for building something useful, the book is a waste of time. If it's for your personal interest, you can read and learn anything you want.
Personally I avoid books like this one (similar to how I avoid very esoteric languages) because I want to spend my time on things are interesting and useful instead of only interesting.
Does anyone have any other free Common Lisp book recommendations?
I decided to give CL a try after reading about REPL-driven development, especially CL’s interactive condition/debugging experience. I’m almost done going through Practical Common Lisp. It’s been a fun experience so far!
Every once in a while I find even the language standard insufficiently clear (e.g. how to change the exponent character when printing double precision floats), then I dig again into Common Lisp the Language (2nd ed.) by Guy Steele available at https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/clm.html .
Everyone says SICP is this profound thing but I just couldn't get into it or LISP at all. Am I really missing out on anything after a good CS education and practical experience?
I bought this book in paper time ago, put it aside (I bought half a dozen at the same time) and when I finally got to read it, I found out that I had brought the wrong version. Too late to return it...
A question: is any of the currently hot software written also in Lisp? I mean the LLMs, SD, etc.
If you are interested in machine learning, check out Gabor Melis's library: https://github.com/melisgl/mgl. It's not an area I'm super familiar with, so I can't speak to it's feature set, but I believe he used it to win a machine learning competition some years ago.
I don't think anyone's written a transformer or diffusion model with it, could be a fun challenge.
[+] [-] discreteevent|2 years ago|reply
https://markwatson.com/#books
Mark Watson on HN:
https://news.ycombinator.com/user?id=mark_l_watson
Edit: I should have said that you can also pay for his books:
https://leanpub.com/u/markwatson
[+] [-] mark_l_watson|2 years ago|reply
[+] [-] rahimnathwani|2 years ago|reply
[+] [-] kagevf|2 years ago|reply
“Good Old-Fashioned AI”
I had to look it up . . .
[+] [-] lukego|2 years ago|reply
It's the book that got me hooked on Common Lisp (from Scheme) too. Focusing less on the elegance of the language definition and more on writing elegant programs.
(English isn't as elegantly defined as Lojban and but that didn't hold back Tolkien.)
[+] [-] 7thaccount|2 years ago|reply
[+] [-] mr_luc|2 years ago|reply
What makes it my favorite is how clear Norvig's writing is. It's easy to follow (both when reading it in English, and when following its execution if you're a programmer), and it introduces important ideas so effortlessly that, years later, it will give you a chuckle.
Anyone interested in clearly communicating about technical topics, and with a knowledge of Lisp's nature and some idea of what programming in 1991 looked like, might be tickled to read Chapter 1; even its first few paragraphs are refreshing.
[+] [-] codr7|2 years ago|reply
[+] [-] Sirenos|2 years ago|reply
Currently, statistical/data-driven approaches work best, and that's what you will be expected to use whether you are building your own products, or working for an employer. Most people don't care about the GOFAI approaches anymore, seeing them as outmoded in all respects.
However, if you are curious and want to understand more of the history of approaches we have tried, and learn some really interesting algorithms along the way, I think studying the old school problems and their solutions can be both intellectually stimulating, and potentially increase your depth of understanding. After all, it's only once you've tried to solve a problem and failed miserably that you start to appreciate the depth of its complexity.
That depth of appreciation is sorely lacking in today's new cohorts, who are basically blinded by the incredibly convincing outputs of our cream-of-the-crop LLMs.
[+] [-] time_to_smile|2 years ago|reply
While it would be great if everyone interested in the topic was well versed in the fundamentals, the truth is if you want to do anything from building something cool over the weekend to getting an actual job doing AI work, you're much better off starting not only with ML, but specifically with current SotA neural networks.
If you really want to get started in AI I highly recommend building even a trivial implementation of Stable Diffusion on your own. Not just because it's cool, but because at its heart it is an excellent demonstration of how current differentiable programming works. Diffusion models involve chaining together 3 separate models into an entire system that learns to solve a complex task. Once you understand this deeply, you can now solve a very broad range of tricky problems and are really approaching what we think of when we think of AI.
Differentiable programming is really the current pathway to any sort of AI solution to a problem.
I say this as the token "have you tried logistic regression?" guy in my org.
[+] [-] medo-bear|2 years ago|reply
this is untrue. ml algorithms have nothing to do with gofai algorithms. if there is something one "would be better off surveying before diving head first into ML" it would be mathematical analysis, statistics, and probability
[+] [-] jstx1|2 years ago|reply
The advice to spend your limited time and attention on outdated approaches seems counterproductive. The things in this book aren't just old - they ended up being a dead end in research. So if it's 2023 and you have 20 hours to learn something new, you can do much better than this book.
[+] [-] hcks|2 years ago|reply
Even funnier to see how someone is always quick to explain that ´NN are not real AI’ when GOFAI was literally all about parsing, basic logic and search trees.
[+] [-] ducktective|2 years ago|reply
Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods.
[+] [-] kqr|2 years ago|reply
Old techniques have several things going for them, with one of the more important ones for us being explainability. A random person off the street could hypothetically, with an hour or two of training, diagnose problems just by looking at the structure of the model. That's very helpful for adapting to market needs.
Generality is another big plus. Since the model encodes intuitive ideas there's a lot of room for using it in innovative ways.
Older techniques also tend to produce better results with less data, because big data wasn't as much of a thing back then.
Unless you have a crazy amount of resources, I think it's far better to be bleeding edge in as few things as possible. Solving a new business problem? Perhaps don't spend too much time on also solving all the childhood diseases of a new technology.
[+] [-] dimatura|2 years ago|reply
As for GOFAI in the age of DL/LLM, yes, you should know it, for a couple of reasons. A lot of these techniques aren't really considered "AI" anymore, they're just regular CS algorithms everybody should know: graph search, backtracking, optimization, parsing, etc. The other is that a lot of newer DL/LLM is actually going back to these old problems, but bringing all the new techniques to deal with limitations of the classical algorithms.
[+] [-] danwee|2 years ago|reply
Same for me.
> I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former?
I think it still matters. Plenty of examples in the tech industry where "old" tech/paradigms became the new "hype". They say it's all a cycle.
[+] [-] abecedarius|2 years ago|reply
[+] [-] Muller20|2 years ago|reply
[+] [-] blihp|2 years ago|reply
[+] [-] kazinator|2 years ago|reply
Being able to trace how an answer is derived is also worth something.
[+] [-] oumua_don17|2 years ago|reply
https://en.wikipedia.org/wiki/GOFAI
[+] [-] hiepph|2 years ago|reply
Sorry if you don't get the metaphor, but it's like so.
[+] [-] bionhoward|2 years ago|reply
[+] [-] jstx1|2 years ago|reply
Personally I avoid books like this one (similar to how I avoid very esoteric languages) because I want to spend my time on things are interesting and useful instead of only interesting.
[+] [-] jsmorph|2 years ago|reply
[0] https://norvig.github.io/paip-lisp/#/chapter5?id=_52-pattern...
[1] https://github.com/norvig/paip-lisp/blob/main/lisp/patmatch....
[2] https://github.com/Comcast/sheens#pattern-matching
[+] [-] schemescape|2 years ago|reply
I decided to give CL a try after reading about REPL-driven development, especially CL’s interactive condition/debugging experience. I’m almost done going through Practical Common Lisp. It’s been a fun experience so far!
Edit: thanks, everyone!
[+] [-] guenthert|2 years ago|reply
[+] [-] atemerev|2 years ago|reply
https://lispcookbook.github.io/cl-cookbook/ -- also great and up to date
https://awesome-cl.com/ -- for anything else.
[+] [-] kgwgk|2 years ago|reply
https://www.lurklurk.org/onlisp/onlisp.html
[+] [-] kaveh808|2 years ago|reply
https://www.youtube.com/playlist?list=PLTA6M4yZF0MzsMlNL0N67...
[+] [-] tmtvl|2 years ago|reply
[+] [-] perihelions|2 years ago|reply
https://maxima.sourceforge.io/documentation.html
https://en.wikipedia.org/wiki/Maxima_(software)
[+] [-] pfdietz|2 years ago|reply
[+] [-] jefffoster|2 years ago|reply
Read it to learn how to program, learn a bit about AI as an added benefit.
[+] [-] ssrc|2 years ago|reply
* SICP, by Abelson and Sussman
* Lisp (3rd ed), by Winston and Horn
* The Art of Prolog, by Sterling and Shapiro
It's like the same book written by different authors, and all of them are good.
[+] [-] jimmaswell|2 years ago|reply
[+] [-] dimatura|2 years ago|reply
[+] [-] narag|2 years ago|reply
A question: is any of the currently hot software written also in Lisp? I mean the LLMs, SD, etc.
[+] [-] treeblah|2 years ago|reply
[+] [-] vindarel|2 years ago|reply
[+] [-] vzcx|2 years ago|reply
I don't think anyone's written a transformer or diffusion model with it, could be a fun challenge.
[+] [-] necroforest|2 years ago|reply
[+] [-] lincpa|2 years ago|reply
[deleted]
[+] [-] bighoki2885000|2 years ago|reply
[deleted]
[+] [-] pelasaco|2 years ago|reply