How Claude Shannon helped kick-start machine learning

[nicholast]

He was also a jazz musician (the clarinet), a somewhat accomplished juggler, a devoted unicycle enthusiast, and left behind a basement full of contraptions he was building in various states of finish - like the electronic mouse navigating a maze, a chess playing machine, and all other kinds of curiosities. His papers are coherent and still relevant to this day and follow the birth of each of these fields like information theory and artificial intelligence. Who knows what else he might have been working on at Bell labs that we may not be privy too.

[JoeDaDude]

Not explicitly mentioned in TFA is that Shannon was the first to apply the Minimax algorithm [1] for computer chess. The Minimax algorithm, later streamlined to include alpha-beta pruning, has been a key component in AI game playing machines ever since.

[1]. https://en.wikipedia.org/wiki/Minimax [2]. https://stanford.edu/~cpiech/cs221/apps/deepBlue.html

I've posted this before, a compiled list of the machines and gadgets Claude Shannon built to experiment with simple AI ideas to play games [3]. Apologies for the repeat:

[3]. https://boardgamegeek.com/geeklist/143233/claude-shannon-man...

[sasvari]

Interesting episode from Tim Harford's Cautionary Tales podcast about Claude Shannon and his various other interests:

https://www.pushkin.fm/episode/fritterin-away-genius/

[jacobriis]

"His papers are coherent"

High praise for one of the greatest minds of his generation

[vernie]

Damn, I was happier not knowing about the juggling and unicycle stuff.

[pseudolus]

A film "The Bit Player" featuring "contemporary interviews, archival film, animation and dialogue from interviews with Shannon" was released in 2019. [0][1].

[0] https://www.youtube.com/watch?v=D7WmMSAxq8s

[1] https://www.amazon.com/Bit-Player-John-Hutton/dp/B08D2N2MPH/...

[eternalban]

Robert Price's 1982 interview ("oral history") with Shannon:

https://ethw.org/Oral-History:Claude_E._Shannon

[albert_e]

Thanks.

Unfortunately I am prime member but apparently I cannot watch this outside US

[lvl100]

In my mind Claude Shannon is the greatest mind in modern history (move over Einstein!). He’s vastly under-appreciated outside of CS circles.

[bob1029]

Information theory continues to be the one thing that sets my brain on fire to this day. Nothing else in science/engineering/math is as self-reinforcing in its concepts to me. Perhaps this is simply because I deal with the consequences of information theory every single day when I have to make decisions like "should this be a 32 or 64 bit integer" or "Should I compress individual items or batches of those items"

You don't even have to be able to remember the equations to get this stuff right many times. There is definitely an intuition that you can develop where you look at a test file's size and instantly realize it could not contain the x264 stream of data you were expecting it to.

[marginalia_nu]

As a physicst I associate him primarily with Shannon-Nyquist, but his name has that Gaussian quality of cropping up everywhere.

[maxthegeek1]

He's up there, but von neumann still beats him out IMO.

[cjwebb]

He was clearly a smart person.

I do wonder though, if the reason we won’t see his likes again is because he was truly a one-off, or that his particular environment enabled him to show his excellence.

I’ve been reading “The Idea Factory”, about Bell Labs, recently.

[voidhorse]

Environment definitely plays a big role. It’s a balance of environment and individual capabilities. When assessing intellectual contributions, I think people tend to underestimate environmental and structural factors (were you working at a place like bell labs, did you go to an elite school, etc.) and overestimate the “innate” abilities or gifts of the individual.

If you look at intellectual history, almost every genius worked in an environment in which they were surrounded by equally brilliant minds, or had ample correspondence with other thinkers of their time (of course there are exceptions).

[Barrin92]

If we're talking about Machine Learning in particular Turing deserves a mention for his 1948 paper The Unorganized Machine. It's insane to me how this anticipated neural nets and genetic algorithms at a time when computers barely existed.

https://weightagnostic.github.io/papers/turing1948.pdf

[Kalanos]

algorithms are just a condensed logical representation of information. that's why i associate him w the field

[Claude_Shannon]

You have any questions to me?

It is amazing how much of what we perceive as new had actually deep roots in past thinking.

[angrais]

What's it feel like to be dead?

[hegzploit]

Jon vom Neumann also comes to mind when thinking about brilliant people from the past century.

[pseudolus]

There's a new bio of von Neumann "The Man from the Future: The Visionary Life of John von Neumann" coming out in February in the US. [0] If you're impatient you can order a copy from Amazon's UK site where it's been available since October.

[0] https://www.amazon.com/Man-Future-Visionary-Life-Neumann/dp/...

[pixelpoet]

> Nobel Laureate Hans Bethe could not comprehend von Neumann's incredible intellect: "I have sometimes wondered whether a brain like von Neumann's does not indicate a species superior to that of man."

There are plenty of smart humans, but John von Neumann was something else entirely. Like the author of the article concludes about Shannon, I wonder if we will ever see another "human" mind like that again.

[bluetwo]

I was chatting about him with a friend and pointing out all the things in the room influenced by him. Of course he led a pretty normal life, so no one is going to make a "Beautiful Mind" type movie about him. Other than playing his record player too loud in his office and having wild martini happy hours with his wife and friends, I can't find anything controversial about him.