felippee's comments

Sure it is not technically a loan. But it carries the same sentiment change when it blows up. People get extremely cautious, to the point of skipping some really good ideas. And not just those VC's that made the bets but everyone else too. Fear spreads just as effectively as hype.

> Yes. Sensationalist.

Yes, perhaps. But I'm entitled to my opinion just as you are entitled to yours. And time will tell who was right.

> So your explicit reason for omitting Waymo, as I understand it, is that it didn't support your argument?

You see, when you make any argument, you always omit the infinite number of things that don't support it and focus on the few things that do. The fact that something does not support my argument, does not mean it contradicts it.

You might also note that this is not a scientific paper, but an opinion. Yes, nothing more than an opinion. May I be wrong? Sure. And yet this opinion appears to shared by quite a few people, and makes a bunch of other people feel insecure. Perhaps there is something to it? We will see.

But in the worst case it will make some people think a bit and make an argument either for or against it. I may learn today a good argument against it, that will make me think about it more and perhaps I will change my opinion, or I'll be able to defend it.

So far you have not provided such an argument, but I wholeheartedly encourage you to do so.

Hi, it appears that "sensationalist garbage" triggered quite a bit of a discussion. This is typically indicative that the topic is "sensitive". Perhaps because many people feel the winter coming as well. Maybe, maybe not, time will tell.

And FYI, Tesla is in the business of making self driving car. If you read the article, you might learn that Tesla is actually the first company to sell that option to customers. You can go to their website right now and check that out.

Uber, like it or not is one of the big players of this game. I agree they may have somewhat toxic culture, but I guarantee you there are plenty of really smart people there who know exactly the state of the art. And their failure is therefore indicative of that state of the art.

I also omitted Cruise automation and a bunch of other companies, perhaps because they have more responsible backup drivers that so far avoided fatal crashes. But I analyze the California DMV disengagement reports in another post if you care to look. And by no means any of these cars is safe for deployment yet.

OK, the definition of scalable is crucial here and it causes lots of trouble (this is also response to several other posts so forgive me if I don't address your points exactly).

Let me try once again: an algorithm is scalable if it can process bigger instances by adding more compute power.

E.g. I take a small perceptron and train it on pentium 100, and then take a perceptron with 10x parameters on Core I7 and get better output by some monotonic function of increase in instance size (it is typically a sub linear function but it is OK as long as it is not logarithmic).

DL does not have that property. It requires modifying the algorithm, modifying the task at hand and so on. And it is not that it requires some tiny tweaking. It requires quite a bit of tweaking. I mean if you need a scientific paper to make a bigger instance of your algorithm this algorithm is not scalable.

What many people here are talking about is whether an instance of the algorithm can be created (by a great human effort) in a very specific domain to saturate a given large compute resource. And yes, in that sense deep learning can show some success in very limited domains. Domains where there happens to be a boatload of data, particularly labeled data.

But you see there is a subtle difference here, similar in some sense to difference between Amdahl's law and Gustafson's law (though not literal).

The way many people (including investors) understand deep learning is that: you build a model A, show it a bunch of pictures and it understands something out of them. Then you buy 10x more GPU's, build model B that is 10x bigger, show it those same pictures and it understands 10x more from them. Look I, and many people here understand this is totally naive. But believe me, I talked to many people with big $ that have exactly that level of understanding.

Hey, a small advice for the future: never build your belief entirely on a youtube video of a demo. In fact, never build your belief based on a demo, period.

This is notorious with current technology: you can demonstrate anything. A few years ago Tesla demonstrated a driverless car. And what? Nothing. Absolutely nothing.

I'm willing to believe stuff I can test myself at home. If it works there, it likely actually works (though possibly needs more testing). But demo booths and youtube - never.

> but it’s moving down the right path

Time will tell. I think DL is amazing, but is no the right path towards solving problems such as autonomy. I think if you enter this field today, you should definitely take a look at other methods than DL. I actually spent a few years reading neuroscience. It was painful, and I certainly can't tell I learned how the brain works, but I'm pretty certain it has nothing to do with DL.

I certainly encourage everybody to consult the source material! Man, this is a blog, opinion by default not perfect.

But when I hear the keyword "major advances" I'm highly suspicious. I had seen already so many such "major advances" that never went beyond a circle of self citing clique.

I'll happily read your next post where you will include all of those. In fact amount of VC money spent in that field would only support my claim. And the number of papers is irrelevant. There were thousands of papers about Hopfield network in the 90's and where are all of them now? You see, all the things you point out is the surface. What really matters is that self driving cars crash and kill people, and no one has any idea how to fix it.

> The difference between the current AI renaissance and the past pre-winter AI ecosystems is the level of economic gain realized by the technology

I would argue this is well discounted by level of investment made against the future. I don't think the winter depends on the amount that somebody makes today on AI, rather on how much people are expecting to make in the future. If these don't match, there will be a winter. My take is that there is a huge bet against the future. And if DL ends up bringing just as much profit as it does today, interest will die very, very quickly.

Thanks for that, that is essentially my point. Agree it is not very rigorous, but it gets the idea across. By scalable we'd typically think "you throw more gpu's at it and it works better by some measure". Deep learning does that only in extremely specific domains, e.g. games and self play as in alpha go. For majority of other applications it is architecture bound or data bound. You can't throw more layers, more basic DL primitives and expect better results. You need more data, and more phd students to tweak the architecture. That is not scalable.

Author here: I'm using deep learning daily so I have a bit of an idea on what I'm talking about.

1) Not my point. Hype is doing very well. But narrative begins to crack, actually indicative of a burst... 2) DL does not scale very well. It does scale better than other ML algorithm because those did not scale at all. If you want to know what scales very well, look at CFD (computational fluid dynamics). DL in nowhere near that ease in scaling. 3) self driving is the poster child of current "AI-revolution". And it is where by far most money is allocated. So if that falls, rest of DL does not matter. 4) Not that this matters, does it?

Yeah, the problem is deep learning sucked a bunch of money - essentially took a loan against the future in the form of VC investments. And if that loan does not get payed, for the next few years you may not afford to explore all that other stuff.

Hi, author here:

Well first off: letters to investors are among the most biased pieces of writing in existence.

Second: I'm not saying connectionism did not succeed in many areas! I'm a connectionist by heart! I love connectionism! But that being said there is disconnect between the expectations and reality. And it is huge. And it is particularly visible in autonomous driving. And it is not limited to media or CEO's, but it made its way into top researchers. And that is a dangerous sign, which historically preceded a winter event...

Author here, yeah, it is the autumn. But I guess not many people would recognize the meaning, winter on the other hand is not ambiguous...

Yeah depends on what you mean by close. I see the cheapest 1080ti at newegg right now for $909. Bought mine a year ago (founders edition) for $699. It is getting better but we are still far away from sanity. IMHO this (year old) card should sell today for ~$500 if things were to be normal.

The number you pull out is not telling. There are billions of miles driven in the US alone and people get killed on average after 100 mil miles driven. That is where the bar is, and I can tell you, that is a high bar. Even including all the idiots, teenagers, DUI's etc. it is still one fatality on 100 mln miles. So sure, it is a tragedy that those 40.000 people die, but that is not a simple problem to fix. In fact obesity and sugar caused disease is probably a much simpler problem to solve, that could likely save way more people.

Tesla was not a self driving car, please stop confusing this.

This case is in fact the first autonomous test vehicle caused fatality.

There were at least several Tesla autopilot related fatalities and injuries, but I would seriously not put those into self driving bag.

Though I agree with the spirit of what you are saying, I would add that it is perhaps not the weakness of neural networks per se, but weakness of the current architectures and training (supervised, hence verbalized) paradigms. I think we could do much more to improve things, if we stopped pushing for extra % on benchmarks and instead rethink the problems we'd like to solve and approach them from a new angle.

It is 0.99 deaths/100 mil in California, but fair enough. I'm not sure where you got your "well over 10mil miles" data. Last time I checked, Waymo clocked 3mil, Uber 2mil, Cruise <0.5mil, and the rest are small potatoes, so it looks more like maybe 6-7 million miles to date at best (though if you can provide a reference that would be great). But that aside, you are missing one more important point:

Current autonomous cars still have a backup driver. So what we are measuring here is a compound safety level of autonomous tech + attentive, professional, sober human. The real data of interest is the safety level of autonomous vehicle alone. We don't have that data. We can proxy it, by looking at the numbers of disengagements and their severity, and that data currently does not look particularly good. But nonetheless it is just proxy data. Once a larger scale tests without backup drivers are concluded, we will get a better picture. Until then, I advise to withhold from any statements such as "autonomous vehicles are much safer than humans", because they are simply not supported by any data.

As for you final statement, I bet there are many undeveloped countries or particular cities with huge number of deaths per mile. But exceeding their death rate on US roads is not anything to be celebrated.

I'm sorry to disappoint you, but again the data does not support your claim. The cumulative number of miles driven by autonomous test fleet to date is nowhere near 100 million miles, which is where one should expect the first fatality from an average human. So no, your claim is empty.