[eonw]

seems like a hard nut to crack, considering there are constant changes to google algos that could screw with test results mid-test, and many other moving parts. I think what you are doing is great, have built similar tools in the past and always ran into issues due to so many moving parts, but im not a statistician.

Also what do you consider success? what if one title/meta combo gets a better rank but fewer clicks(ie; the bots like it but the users dont)? which one would you keep? I know many people like to brag about how many rankings they have but as an SEO for over a decade, i think traffic count is still king. in that id rather rank 11th for an extremely high volume term, then rank 2nd for a low volume one.

another thought... a blackhat site gets banned halfway through one of your tests and you move up based on no actions you took. thus you would get a false positive, no?