[ramidarigaz]

I think there is a good option for choosing your priors in the Earth/Sun problem you mentioned. Just put them at 50/50. That's basically stating that you have no preference for one theory over the other, and it allows for any evidence you gather to be maximally effective in computing the posterior.

[ikeboy]

That quickly leads to inconsistencies.

What's the probability of the earth going around some object bigger than it?

What's the probability of the earth going around something bigger than it that isn't the sun?

The answer to all 3 cannot all be 50/50.

[taejo]

That's true, but if we assume the probabilities of "Earth orbits Sun" and "Sun orbits Earth" we still get the correct conditional probabilities under the condition "either the Sun orbits the Earth or the Earth orbits the Sun", and the ratio between the two conditional probabilities is the same as the ratio between the absolute probabilities. So we can still decide which hypothesis is better, and by how much.