[cmatthias]

This is true but irrelevant to the parent's question -- in the article, it's made clear that Google's requests are happening over HTTP, which is the most obvious reason why robots.txt should be respected.

[yamtaddle]

It's relevant because it attacks the premise of their objection.