[clickok]

You have misunderstood Sutton's argument. Quoting Rich:

  One thing that should be learned from the bitter lesson is 
  the great power of general purpose methods, of methods that 
  continue to scale with increased computation even as the
  available computation becomes very great. 

The point isn't that improvements in our algorithms is unnecessary or unhelpful, rather that the algorithms we should focus on will be capable of scaling with arbitrary amounts of compute/data. Such as, for example, neural networks, where we see an almost constant rate of improvement (for the appropriate architecture) as more resources are added.

[KKKKkkkk1]

I think this argument is neither here nor there. For computer vision problems, we use convnets, which are models inspired by a biological model of vision. By doing that we are embedding our preconceived notions of what vision is into our models instead of throwing compute and data at the problem. Earlier attempts using multi-layer perceptrons have been massive failures. Is this consistent with Sutton's analysis or contrary to it?

[clickok]

It's consistent with it.

Rich used to be very bullish on neural nets, then somewhat dismissive of them (due to the fragility/inadequacy of FCNs), and then increasingly enthusiastic as the renewed interest demonstrated that those problems could be overcome-- e.g., through better initialization, training, and (as you note) different architecture choices. His main concern was whether a method could keep working as more resources became available, as otherwise you would tautologically end up with something short of true artificial intelligence.

The important thing is that the technique can scale with increasing data or compute without hitting a hard or soft limit.