top | item 13486263

(no title)

tma-1 | 9 years ago

This might be a dumb question, but why not use the classification accuracy score as a measure of uncertainty?

discuss

order

laingc|9 years ago

It's not a dumb question at all. The classification accuracy and "uncertainty" are different, but the explanation depends on what you mean by uncertainty.

Let me try and give one intuitive explanation; if others would like to chime in with something better, by all means.

Let's suppose that you are classifying objects in images - say bananas and oranges, but it could be tumors or anything that you like.

So we train a classifier to predict this, and we find that of 100 classifications on a hold-out set, we get 73 of them correct. You might, quite reasonably, interpret this to mean that if we randomly select a new image of either an orange or a banana, we will have a 0.73 probability of classifying it correctly. (There are actually some subtleties in this interpretation which I'm ignoring, but they aren't so important for the point I want to make.)

Suppose, however, that we draw out an image that we want to feed into our classifier, and we look at it for a moment. Suppose further that this image contains an object that is long, thing, curved and yellow. We'd expect our classifier to classify it as a banana, and sure enough, it does. Now we draw out another image, except this one has an object that is long, but bent almost completely in a circle, and is more orange than yellow. Now, we might still expect our classifier to classify this as a banana, but should the classifier really be as certain about this prediction as it was about the previous one? Intuitively, I would say not. However, the overall classification accuracy remains unchanged, and so we can't say anything in particular about the certainty of this prediction.

So uncertainty isn't just the proportion of your results that you classify correctly.

Furthermore, it also isn't exactly equivalent to the class probability produced by your classifier, though I don't think this is the best forum for me to get into the details on that.

nerfhammer|9 years ago

That's how accurate a given classifier is as a whole vs. a measure of confidence for a particular prediction.