erdigious | 3 years ago | on: Tension Inside Google over a Fired AI Researcher’s Conduct
erdigious's comments
erdigious | 3 years ago | on: Tension Inside Google over a Fired AI Researcher’s Conduct
It continues to boggle my mind that this guy chose to get fired over this paper, which is the dumbest paper I have read in a long time.
As an example, his paper reports *median* congestion! I literally laughed out loud when I saw that. I don't think a single other chip design paper in the world uses that metric, and for good reason.
For those of you who don't know chip design, congestion is a measure of how many wires will need to go in the same place when you do routing for the chip. When wires overlap, they have to go above / below each other, and your chip only has so many layers. If congestion is too high, it can simply be impossible to make the chip.
Congestion is therefore all about the extreme values. Why in the world would you report median congestion, then? Median ignores extreme values, which are the only ones you care about!
I even checked the paper he cites as being prior work that uses median congestion, to see who else did something so stupid, and... it doesn't! It uses average, which makes way more sense. I mean, you really care about the max, but at least average is sensitive to extremes.
I could go on, but the TLDR is, his paper is hot garbage, and I cannot say I am surprised Google didn't want its name associated with it.
erdigious | 3 years ago | on: Tension Inside Google over a Fired AI Researcher’s Conduct
Some tweets by someone who replicated their results:
https://twitter.com/sguada/status/1521680121412808704
https://twitter.com/sguada/status/1521587406385807361
Tweets by other people who know the researchers:
https://twitter.com/_beenkim/status/1521626239596785664
https://twitter.com/ch402/status/1521631660155965441
https://twitter.com/JacobSteinhardt/status/15215993379432898...
(There was a bit of a twitter-storm when a related NYT article came out earlier)
erdigious | 3 years ago | on: Tension Inside Google over a Fired AI Researcher’s Conduct
Seems pretty clear to me, tbh:
1) Google terminated this guy "with cause" and then said so publicly. Google definitely does not do that lightly.
2) The TPU team used this method to help make TPUs. This is a flagship product, with billions of dollars on the line. There's just no way Google would sacrifice the quality of TPU to help a couple AI researchers look good.
Sometimes people are just jerks. PG's essay on haters seems relevant here: http://www.paulgraham.com/fh.html
page 1
Also, they revived some ancient 1998 IBM chip, and report on that, for no clear reason. They call it a 2004 benchmark, but what actually happened is that in 2004 someone made some synthetic variants and put them into a dataset. This is not some widely used benchmark in the field. Given their highly questionable choice of metrics, I would not be surprised if there was some serious cherry-picking on the benchmark as well.