[rparet]

It was mentioned before in the thread, but I'll just go ahead and leave this here again:

http://en.wikipedia.org/wiki/Planning_fallacy

Trying to improve developer estimates is just another case of doing the wrong thing, righter.

[jacques_chester]

Every time there's a discussion on estimation the Planning Fallacy is brought up as a kind of trump card.

But it's not. First, psychologists have done research on how to ameliorate it. See the "unpacking effect" paper I linked elsewhere. Kahneman and others have also pushed using "reference classes". In the estimation literature this is called "estimation by analogy" and it is a well-established technique.

Which reveals the next thing about the Planning Fallacy; it comes from the psychological literature. There are at least two other bodies of literature which need to be dealt with before we give up entirely. The first is ordinary project estimation literature; most of the advanced stuff is by people from the Operations Research field. The second is work done by Statisticians working on what they call forecasting which is essentially parametric estimation of timeseries data.

These 3 bodies of literature seem to have evolved largely in isolation. I haven't see OR papers talking about the psych research, I haven't seen the psychologists citing the International Journal of Forecasting and so on.