[conover]

This is the classic precision versus recall discussion. The discussion centers around how you think about the cost of a false positive versus false negative.

Some people think it's fine for the process to have low precision but high recall. Low precision is that of the number of conversations the process flagged as a positive, some unacceptable (to you) percentage turned out to be a not/false "positive". High recall is that of all the actually positive conversations, the process flagged an acceptable (to you) percentage of them as positive (i.e. only "missed" a few/false negative).

What does it cost to wrongly identify conversation a positive when it's really not a positive (false positive)?

What does it cost to wrongly identify a conversation as a negative when it's really a positive (false negative)?

You decide.