Apple to Start Publishing AI Research

[amelius]

Of course! What researcher would want to work for a company that prohibits scientific publications?

I've always been amazed by their attitude.

Look at Microsoft Research, and their enormous scientific output over the years. IBM and Google look bleak by comparison, and Apple is not even on the chart.

[Cyph0n]

If you consider the number of papers published over the lifetime of IBM, and the fact that IBM Research does work in all levels of computing (materials science -> software), Microsoft's research output looks minuscule in comparison. But if you're considering only CS research, then you might be right.

Here is a nice snapshot of the major contributions that came out of IBM Research over the past 60 years: http://www.research.ibm.com/featured/history/. Edit: the invention of copper interconnects is enough for me to be frank (http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/copperc...).

On a similar note, Intel is another company that is very active in research and has published a significant amount of peer-reviewed publications. Samsung Research is yet another; they have an amazing presence at circuit design conferences, for instance.

[KKKKkkkk1]

Unfortunately, my prior reply here was flagged, but I'd like to try again. Here's a quote from Craig Federighi, senior VP of software engineering, less than half a year ago:

“Our practices tend to reinforce a natural selection bias — those who are interested in working as a team to deliver a great product versus those whose primary motivation is publishing,” says Federighi.

https://backchannel.com/an-exclusive-look-at-how-ai-and-mach...

In my opinion, this gives you a view of the thinking that goes inside Apple. Publishing papers correlates negatively with being a team player and delivering great products, and here Apple is on record as saying that they do not want that sort of people working there.

[deelowe]

Not sure how this stacks up against MS, but Google's research is posted here: https://research.google.com/pubs/papers.html

[hiddencost]

# of papers published is a poor metric (google's published like 40% as many papers, over a shorter period?)

MSR has done cool work and important work, but google's publications have been often been major paradigm shifts. In a lot of ways they're currently way out ahead of everyone else.

[bitmapbrother]

>Look at Microsoft Research, and their enormous scientific output over the years. IBM and Google look bleak by comparison, and Apple is not even on the chart.

Do you have any metrics to back up this dubious claim? I find it very hard to believe Microsoft is even in the same ballpark as IBM considering all of the research centers IBM has throughout the world and the number of years they've been in business.

[coldtea]

>Of course! What researcher would want to work for a company that prohibits scientific publications?

Depends on the compensation and several other factors. Lots of military/intelligence related research for example is not published either...

[ced]

Was it worth it for Microsoft?

[zackkatz]

First paper: "Snarky Reply and Joke Generation by a Virtual Assistant Using Deep Learning"

[mtgx]

My hope is that Apple will continue research on privacy-protecting and privacy-enhancing machine learning, because out of all the big tech companies using machine learning, they may just be the only ones to do that. Some random research for privacy technologies may come out of Google, too, but they are much less likely to actually use them at scale, especially if they conflict with ad revenue.

[numbas]

Actually Microsoft has been publishing a lot on privacy enhancing Machine Learning too.

For instance:

https://www.microsoft.com/en-us/research/publication/crypton...

https://www.microsoft.com/en-us/research/publication/private...

https://youtu.be/xsaXMUelOEA

As for Apple: seeing is believing.

[Ar-Curunir]

MSR researchers essentially invented differential privacy, so...

[hota_mazi]

> Researchers say among the reasons Apple has failed to keep pace is its unwillingness to allow its AI engineers to publish scientific papers, stymieing its ability to feed off wider advances in the field.

I don't follow. How would preventing employees from writing papers would stop them from reading papers?

[sytelus]

Modern AI research heavily depends on building work of others as well as having others build on your work. It's this virtuous cycle that is bringing all the advances we are seeing. Let's say someone at Apple had invented recurrent networks but they never published it. That would mean they would miss out on literally hundreds of researchers working on this, enhancing techniques in very critical ways and writing thousands of papers over the period of just few years that would feed in to this cycle. Many of these papers would contribute absolutely important enhancements to productionize things.

Much of the AI research currently starts with some seed idea like GAN which looks cool in small experiments but have tons of blind spots that needs to iron out. Unlike typical product efforts where you can probably work your way out through brute force engineering, advances in most AI related areas requires massive amount of collaboration, mathematical acrobatics, trial-and-error and cross-pollination across fields consuming multi-person-years before yielding fruits. In theory, Apple can still keep any silver bullets they find in the field secret but in practice such silver bullets are rare and advances are very incremental spanning over many years and many people.

[leereeves]

I think they mean that Apple isn't offering researchers the incentives they desire, and may have trouble recruiting them.

[argonaut]

Would you rather work for a company that has their engineers actively contribute to open source, or a company where all of your code is private?

[jedberg]

If the could publish their research other researchers would take their work in a new direction that they didn't think of, which they could then bring back.

Basically they can't provide that seed insight that motivates others.

[maciejgryka]

I'm just at NIPS and Apple does indeed have a decent presence, which is a good sign (even if there are no actual publications yet). Super interesting to see how this will play out compared to their past. There's lots of criticism that we can throw at big tech companies, but the fact they many of them are so open about their research and thus are forcing others to do the same is pretty cool.

[iverjo]

Jokingly, this was the first apple AI research I saw online, about a month ago: https://arxiv.org/abs/1610.08120v1

[deepnotderp]

Just goes to show you the market power of ml researchers, even apple has to bend the knee.

[pinouchon]

I completely agree. But still I find it surprising that apple managed to hire Salakhutdinov. He is a huge name in Deep Learning (with a focus on math, unsupervised learning and autoencoders)

[home_boi]

The free market wins.

All the AI/ML engineers were going to FB/Google/etc. (and on some rare occasions M$)

[_1]

> and on some rare occasions M$

Are we still doing that?

[k_lander]

So glad that this is happening. It is really inspiring to see the knowledge sharing spirit become the expected default in the community. This is only going to be great for progress in the field!

[dovdovdov]

Based on my Siri experience, I imagine a chimp pulling strings in a control room. :)

[Animats]

Has Apple published anything yet?

[brudgers]

Conference slides have been tweeted.

[deepnotderp]

Yay! I never thought I'd ever see this though.

[turingbook]

Actually, Apple is one of sponsors of this year's NIPS conference: https://nips.cc/Conferences/2016/Sponsors

[serge2k]

> Amazon.com Inc.’s Alexa

So Alexa is really the biggest thing in the market so far right? How many papers does Amazon publish?

[SEJeff]

Not sure how many papers they publish, but I do really use this paper[1] from James Hamilton (literally the Architect of AWS) as the defacto way of building large distributed services:

[1] http://perspectives.mvdirona.com/2014/07/challenges-in-desig...

[echosmith]

Amazon's Alexa group has published 10 papers in the last 2 years.

https://www.amazon.jobs/en-gb/landing_pages/alexa-machine-le...

Dunno about the rest of the company.

[melling]

Story broke 6 hours ago:

http://www.businessinsider.com/apple-is-finally-going-to-sta...

Mac site picks it up 5 hours ago and it gets reported on HN: https://9to5mac.com/2016/12/06/apple-ai-researchers-can-publ...

Bloomberg writes the same story an hour ago and it finally gets huge traction on HN. My guess is that many people ignore the "new" page and it's all a matter of luck that 3 or 4 people get it to the front page where a story takes off.

[JamilD]

BusinessInsider and Mac blogs tend to be less reputable sources than Bloomberg, whose articles regularly make it to the HN front page. Might have contributed to this particular submission's success.