[atoav]

You could say how you produced the random numbers and which random numbers came out. What you can't do however, is then prove that you actually did that and only that and not for example re-rolled the dice till the results looked right to you.

But you have the same issue with a PRNG, just boiled down to the seed value. Disprove that I didn't chose seed 42 by random, you can't.

The only way to protect against this is registering the study with a third party beforehand and letting them dice out the numbers as soon as the hashed data is there.

By registering your plan beforehand, someone can check if you altered it to cherrypick or p-hack your results after the fact. By letting them chose the random numbers you protect against you rerolling the numbers when inconvenient. By letting them only see the hashed data you protect against them rerolling/altering based on their interest and against you swapping out data point indicies after the fact, since they can rehash the actual values and check if it matches their selection.

For many experimental designs this could be overkill, but registering studies is becoming more common in medical studies. I doubt they go as far as I described with the random numbers.

[fc417fc802]

> The only way

Not so. Assuming I don't need a whole bunch of runs in the final paper I can use the zero seed for the published work. I don't think anyone will contest that.

[topaz0]

We're talking about randomizing experiment participants into treatment and control group... What you're saying is equivalent to "I can use the same order of assignments for every experiment I do"...