[magusdei]

Hmm, this still doesn't quite answer my question at the level of concreteness I was looking for. But thank you for clarifying.

The thing is, you can only define predictive accuracy relative to some experimental design. Otherwise you can always claim that there is some unknown, unperformed experiment where the predictions of the model and your actual behaviour would diverge to a greater degree than is permissible by your accuracy threshold, no matter how many successful experiments have already been done in constrained conditions.

Imagine a task where you have to classify images as being of dogs or non-dogs. We can already train a model that can almost perfectly predict the choices you would make during the runs of such an experiment. But we obviously wouldn't call such a model a "model of your brain"!

My question is this: what would be a sufficient experimental design or empirical criterion to decide that some program is a model of you? The loosest criterion I could imagine would be something like "can successfully deceive your loved ones into believing they are you in a single text chat of unbounded duration with some extremely high success rate." Recent advances in NLP lead me to believe that we'll be able to reach at least this level of fidelity quite soon.

[mannykannot]

To be fair, qsort is not insisting on seeing an algorithm that is metaphysically identical with a human mind, nor claiming that such an algorithm would be a p-zombie, devoid of subjective experiences, which are both positions that you might find from dualist philosophers.