Richard Sutton and Andrew Barto Win 2024 Turing Award

[zackkatz]

Very cool to see this! It turns out my wife and I bought Andy Barto’s (and his wife’s) house.

During the process, there was a bidding war. They said “make your prime offer” so, knowing he was a mathematician, we made an offer that was a prime number :-)

So neat to see him be recognized for his work.

[mark_l_watson]

Nice! Well deserved. They make both editions of their RL textbook available as a free to read PDF. I have been a paid AI practitioner since 1982, and I must admit that RL is one subject I personally struggle mastering, and the Sutton/Barto book, the Cousera series on RL taught by Professors White and White, etc. personally helped me: recommended!

EDIT: the example programs for their book are available in Common Lisp and Python. http://incompleteideas.net/book/the-book-2nd.html

[ofirpress]

Good time to re-read The Bitter Lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

[cxr]

Canonical URL: <http://www.incompleteideas.net/IncIdeas/BitterLesson.html>

[khaledh]

Indeed a bitter lesson. I once enjoyed encoding human knowledge into a computer because it gives me understanding of what's going on. Now everything is becoming a big black box that is hard to reason about. /sigh/

Also, Moore's law has become a self-fulfilling prophecy. Now more than ever, AI is putting a lot of demand on computational power, to the point which drives chip makers to create specialized hardware for it. It's becoming a flywheel.

[kleiba]

This depends a little bit on what the goal of AI research is. If it is (and it might well be) to build machines that excel at tasks previously thought to be exclusively reserved to, or needing to involve, the human mind, then these bitter lessons are indeed worthwhile.

But if you do AI research with the idea that by teaching machines how to do X, we might also be able to gain insight in how people do X, then ever more complex statistical setups will be of limited information.

Note that I'm not taking either point of view here. I just want to point out that perhaps a more nuanced approach might be called for here.

[jdright]

> In computer vision, there has been a similar pattern. Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features. But today all this is discarded.Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.

I was there, at that moment where pattern matching for vision started to die. That was not completely lost though, learning from that time is still useful on other places today.

[Buttons840]

Oof. Imagine the bitter lesson classical NLP practitioners learned. That paper is as true today as ever.

[DavidPiper]

This describes Go AIs as a brute force strategy with no heuristics, which is false as far as I know. Go AIs don't search the entire sample space, they search based on their training data of previous human games.

[crabbone]

I remember the article, and remember how badly it missed the point... The goal of writing a chess program that could beat a world champion wasn't to beat the world champion... the goal was to gain understanding into how anyone can play chess well. The victory in that match would've been equivalent to eg. drugging Kasparov prior to the match, or putting a gun to his head and telling him to lose: even cheaper and more effective.

[perks_12]

The Bitter Lesson seems to be generally accepted knowledge in the field. Wouldn't that make DeepSeek R1 even more of a breakthrough?

[blufish]

nice read and insightful

[porridgeraisin]

Their book "Introduction to Reinforcement Learning" is one of the most accessible texts in the AI/ML field, highly recommend reading it.

[barrenko]

I've tried descending down the RL branch, always seem way out of my depth with those formulas and star-this, star-that.

[incognito124]

What is your background? Unfortunately I did not find it very accessible.

[jxjnskkzxxhx]

That book is a joy. Strong recommend.

[zelphirkalt]

You mean "Reinforcement Learning: An Introduction"? Or did they write another one?

[darkoob12]

They should have given it to some physicists to make it even.

[vonneumannstan]

Good time to remind everyone that Sutton is a human successionist and doesn't care if humans all die. He is not to be trusted nor celebrated: https://www.youtube.com/watch?v=NgHFMolXs3U

[textlapse]

The ACM award is for their professional academic achievements - this fetishism to dig into another person’s personal life and find the most weird thing they said as the thing that paints over all of their life’s achievements as evil must stop.

It’s silly and dangerous. Because you don’t like thing A and they said/did thing A all of their lofty accomplishments get nullified by anyone. And worst of all internet gives your opinion the same weight as someone else (or the rest of us) who knows a lot about thing B that could change the world. From a strictly professional capacity.

This works me up because this is what’s dividing up people right now at a much larger scale.

I wish you well.

[317070]

Have you ever met Sutton? He is the most heart-warming, caring and passionate hippy I have ever met. He does not want all humans to die. The talk you link also doesn't support your claim. Perhaps I missed it, in that case, do leave a timestamp.

In the talk, he says it will lead to an era of prosperity for humanity, however without humanity being in sole control of their destiny. His conclusion slide (at 12:33) literally has the bullet point "the best hope for a long-term future for humanity". That is opposite to you saying he "doesn't care if humans all die".

If I plan for my succession, I don't hope nor expect my daughter will murder me. I'm hoping for a long retirement in good health after which I will quietly pass in my sleep, knowing I left her as well as I could in a symbiotic relationship with the universe.

[zoogeny]

> doesn't care if humans all die

That seems to be a harsh and misleading framing of his position. My own reading is that he believes it is inevitable that humans will be replaced by transhumans. That seems more like wild sci-fi utopianism than ill-will. It doesn't seem like a reason to avoid celebrating his academic achievements.

[smokel]

It is interesting that you bring this to the attention, but I don't see why we should not trust or celebrate someone if they have views that you don't agree with.

Edit: especially since I think your implied claim that Sutton would actively want everyone to die seems very much unfounded.

[cowsandmilk]

> doesn't care if humans all die

His last slide literally says “best hope for a long-term future for humanity”. That’s literally the opposite of what you’re claiming.

[visarga]

I think he is trying to take the positive side of what is probably an inetability.

[nycticorax]

This is so silly. Do you imagine temporal difference learning is some kind of human successionist plot?

[Version467]

Very disappointing. I do not understand how people earnestly defend the successionist view as a good future, but I thought he might at least give some interesting arguments.

This talk isn't that. There are no substantive arguments for why we should embrace this future and his representation of the opposite side isn't in good faith either, instead he chose to present straw-man versions of them.

He concludes with "A successful succession offers [...] the best hope for a long-term future for humanity. How this can possibly be true when ai succession necessarily includes replacement eludes me. He does mention transhumanism on a slide, but it seems extremely unlikely that he's actually talking about that and the whole succession spiel is just unfortunate wording.

[ks2048]

At least his Twitter profile no longer has the bitcoin-meme-red-eyes thing.

[rhema]

I used their RL book for a course I taught. It's beautifully written and freely available (http://incompleteideas.net/book/the-book-2nd.html)! I kept getting distracted by the beautiful writing that I would miss the actual content.

[cxie]

Huge congratulations to Andrew Barto and Richard Sutton on the well-deserved Turing Award! as a student, their textbook Reinforcement Learning: An Introduction was my gateway into the field. I still remembered that how Chapter 6 on ‘Temporal Difference Learning’ fundamentally reshaped the way I thought about sequential decision-making.

a timeless classic that I still highly recommend reading today!

[textlapse]

This is a long time coming. To see through an idea from start to finish and make this span an entire field instead of a sub chapter in a dynamic programming book.

I wish a lot more games actually ended up using RL - the place where all of this started in the first place - would be really cool!

[jimbohn]

Well deserved, RL will only gain more importance as time goes on thanks to its (and neural nets) flexibility. The bitter lesson won't feel so bitter as we scale.

[j7ake]

Amazing that Sutton (American) chooses to live in Edmonton, AB rather than USA.

Shows he has integrity and is not a careerist focused on prestige and money above all else.

[armSixtyFour]

https://nationalpost.com/news/canada/ai-guru-rich-sutton-dee...

He gave up his US citizenship years ago but he explains some of the reasons why he left. I'll also say that the AI research coming out of Canada is pretty great as well so I think it makes sense to do research there.

[tbrockman]

As someone who grew up in Edmonton, attended the U of A, and had the good fortune of receiving an incredible CS education at a discount price, I'm incredibly grateful for his (and the other amazing professors there) immense sacrifice.

Great people and cheap cost of living, but man do I not miss the city turning into brown sludge every winter.

[jp57]

He's been there since he left Bell Labs, in the mid 2000's, I think. The U of A is, or was, rich with Alberta oil sands money and willing to use it to fund "curiosity-driven research", which is pretty nice if you're willing to live where the temperatures go down to -40 in the winter.

[Philpax]

Keen is a fully remote outfit, so he can work wherever. It's pretty likely that his reputation would open that door for him no matter where he goes.

[optimalsolver]

So 2025 really is the year of agents.

[jamesblonde]

Built a lot of my PhD on their work 20 years ago. It really stood the test of time.

[rvz]

Absolutely well deserved.

[wegfawefgawefg]

These guys are great but unfortunately the ai sutton and barto book is really bad. You would do better with Grokking Machine Learning by trask, and then a couple months of implementing ml papers.

[Buttons840]

I second this suggestion. Read Grokking Deep Reinforcement Learning before reading Sutton. Well, the Sutton book is free, so take a peak, but if the formulas scare you then read Grokking Deep Reinforcement Learning.

[317070]

These books are about different topics? Sutton and Barto is about Reinforcement learning, and the other book you mention by Trask is on Deep Learning?

[carabiner]

Wonder if he's still working in AGI with Carmack.

[pklee]

Very well deserved !! Amazing contributions !!

[PartiallyTyped]

This made my day! Well deserved!

[nextworddev]

RL may prove to be the most important tech going fwd due to test time compute

[byyoung3]

they deserve it. definitely recommend their book