top | item 13027068

(no title)

samps | 9 years ago

Needless to say, I disagree. It can be straight-up misleading to report means without including a more nuanced view of the distribution. You don't need to use a bunch of fancy statistics, but you do need to consider whether your results could have arisen by random chance. That's not a distraction; it's accurately reporting what you found.

Here's one frightening example of spurious performance results in CS: https://www.cis.upenn.edu/~cis501/papers/producing-wrong-dat...

discuss

order

amelius|9 years ago

> It can be straight-up misleading

It is only misleading if the reader doesn't understand statistics. There is, imho, nothing wrong with putting all your focus on the subject matter, and skipping the statistics while being frank about it.

Also, if you need statistics to show that your method is better than other methods, then perhaps your method is not really that much better.

throw_away_777|9 years ago

If your analysis involves data, you can't skip the statistics. If there is no analysis of data than go ahead and skip the stats all you want. You need to try and determine the uncertainty in your results, both systematic and statistical.