[andrewcooke]

you want mode, not median. there's no reason to think that the background is close to the median colour. that's kinda the whole point of the article...

(median works well if you have symmetrically distributed noise, which is true when denoising astronomy photos, for example, but not here).

[tomkarlo]

I was thinking the same thing, but for sets of images where the occluding items are a relatively small percentage of the image area (and are moving around enough between frames), taking the median pixel value is effectively the same thing as the mode, but faster. (e.g. if you take 10 frames, the 1-2 frames where a pixel is occluded will almost certainly be an outlier to the 8-9 frames where the pixel is almost exactly the same, and the median will take the value in the middle of those 8-9 frames.

(Another problem with mode is that you'd have to posterize the image to ensure that shifting light, noise and camera movement don't cause the values of background pixels to vary slightly. Mode is much more brittle in that respect.)

If I recall correctly, Lightroom / Photoshop has a handy "median" filter, but no mode filter, which is why the popular method uses median.