[erostrate]

List of "famous" ML people not to waste time on:

- Gary Marcus

- Juergen Schmidhuber

- Pedro Domingos

- Max Tegmark

- Eliezer Yudkowsky

Some context for people unfamiliar with ML research: the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that:

- He doesn't deserve the credit he claims, in most if not all cases.

- There's a few cases where his papers should have been cited and weren't. That's fairly common.

- People do not get much credit for formulating an abstract idea in a paper or implementing it on a toy problem. Credit belongs to whoever actually makes it work.

- Credit assignment in ML is not perfect but roughly works.

[YeGoblynQueenne]

>> Credit belongs to whoever actually makes it work.

That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.

[erostrate]

> credit belongs to the people who deserve credit for the work they've done that was useful to others

Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets.

For example, just taking a random paper that Schmidhuber claims should give him credit for GANs, https://people.idsia.ch/~juergen/FKI-126-90ocr.pdf hopefully you can easily see that a lot of work would be needed to turn this into a realistic image generation model. And that is, even if you admit that the idea is strongly related to GANs, which I'm not convinced of but won't spend time on.

> Credit belongs to whoever actually makes it work. >> That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice?

It is accepted practice in the ML community. If it weren't, Schmidhuber wouldn't be complaining.

[lowbloodsugar]

Basically ideas are a dime a dozen. Sure, your idea might be a good one, but how do we spot your grain of sand is special when it looks the same as the rest of the desert? Essentially having an idea isn't useful to others. Demonstrating that your idea has legs is useful to others.

I don't have to deal with citing papers, but I once had to deal with people pitching me ideas, wanting me to sign an NDA, in exchange for 50% of the revenue after I did all the actual work. Just out of curiosity, I signed one once. It was a fart app, IIRC. They thought a fart app needed an NDA, and that I'd then go do all the work and give them 50% because they "had the idea". It was so laughably sad.

If you think these ideas are valuable, I have a beautiful clock for you. It is right twice a day. You'll have the same problem: you won't know when it's right. You'll need someone else's work to tell that.

[jahsome]

The composer deserves no credit for this symphony! Only the orchestra members who "do the work" should be acknowledged!

/s

[logicchains]

>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases.

That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.

[refulgentis]

Frankly, I do, and it comes across as quibbling about categories and trying to define unnecessarily general taxonomic categories, as a reaction to positive reactions to other people.

ex. in the article: "Goodfellow eventually admitted that my PM is adversarial...but emphasized that it's not generative. However, [it] is both adversarial and generative (its generator contains probabilistic units)...It is actually a generalized version of GANs."

When you're at "actually, probabilities means generative, and actually you know what, even my initial claim was too specific: turns out its a generalized version of GANs", all in service of arguing a paper should have been cited in another paper, years after the other paper has been published, there's not much room for sympathy.

[erostrate]

Agreed, Ian Goodfellow wasn't the first to come up with the idea of jointly training a generator and discriminator. But he was the first to make it work for image generation with modern neural networks. For that Ian Goodfellow deserved to get most of the credit, and he did.

[rcbdev]

While I generally agree with you in these specific cases, I feel like this is a shortsighted form of argument nonetheless. Coming up with an idea begets a scalable implementation of it and it's not very wise to ascribe absolute value solely on one side of the equation.

[TimPC]

I think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make it not immediately obvious whether it perfectly fits the broad definition or not) that are actually promising.

[screye]

The sad part is that Schmidhuber is not a grifter by any means. If the turning prize for deep learning could go to 5 people instead of 3, he would very likely be on that list.

His lab is excellent and was easily Europe's best deep learning lab for decades before it blew up.

Some of his complaints are valid too. European labs often get ignored, and he has been sidelined despite being one of the most important people in deep learning himself.

But man doesn't know when an argument runs out of gas. His claims get grander with every passing year.

He would've just been the 'get off my lawn' grandpa of deep learning, but he somehow comes across as even more insufferable than that.

I wonder if 2023 schmidhuber was created because the polite one from a decade ago was ignored. A sort of evil phase, if you will.

I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.

[throwup238]

> I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.

It's a terrible look. We're in the middle of one of the biggest gold rushes in tech history and he's wasting time complaining when he claims to be one of its pioneers? That effort is much better invested in building stuff but I suspect he's fallen into the classic PI trap of writing grants all the time and leaving the real work to the rest of the faculty, atrophying his skills too much to do anything now that the industry is moving so quikcly.

[senderista]

Like the Carl Hewitt of ML?

[P-NP]

What a troll thing to write. User erostrate taking down the often so-called "father of modern AI" whose work is on erostrate's smartphone :-)

[erostrate]

Are you saying Schmidhuber should also get credit for smartphones?

[GlenTheMachine]

Why Tegmark?

[erostrate]

He talks a lot about the singularity but the biggest singularity I can see is in his ratio of "talks about AI" divided by "actual AI contributions".

[mahe_]

> Credit belongs to whoever actually makes it work.

This is just plain wrong. No working version without the idea.

[ben_w]

Ideas are dime-a-dozen.

I independently arrived at[0] something close to Max Tegmark's idea of the Mathematical Universe, almost nobody noticed and fewer still cared because I published it as a LiveJournal blog post whereas he fleshed it out into a whole book.

I didn't get credit because I didn't do the hard work that deserves credit, I had the flash of inspiration and stopped after a few paragraphs of mediocre student philosophy.

[0] and possibly predated, but I lost track of the date format when shifting from LJ to WP: https://kitsunesoftware.wordpress.com/2018/08/26/mathematica...