[spinf97]

> Ending the experiment too early

> Running experiments until you get a hit

But if I'm running an experiment how do I know how many time to run it.

[remus]

Before you start your experiment, you calculate how many samples you need based on the estimated effect size you're looking for and how small you want your confidence interval to be.

Small effect with high confidence => more samples

Big effect with low confidence=> less samples

[analog31]

In the physical sciences you can often estimate the noise level in a null measurement -- or even measure it. You often do this just to get your setup working before doing something like wasting a precious specimen on a "this time for real" measurement.