[thegginthesky]

Great package! I'll test it out on my free time.

I'm wondering if you'd like to accept a contribution for Bayesian AB Testing, based on this whitepaper[0] and developed in Numpy.

If so, we can chat at my email gbenatt92 at zohomail dot com, or I can open a draft PR to discuss the code and paper.

[0]https://vwo.com/downloads/VWO_SmartStats_technical_whitepape...

[e10v_me]

Thank you! I hope it will be useful for you.

Regarding your question, first, I'd like to understand what problem you want to solve, and whether this approach will be useful for other users of tea-tasting.

[thegginthesky]

No problem! I have most of the code in very small functions that I'd be willing to contribute.

At my company we have very time sensitive AB tests that we have to run with very few data points (at most 20 conversions per week, after 1000 or so failures).

We found out that using Bayesian A/B testing was excellent for our needs as it could be run with fewer data points than regular AB for the sort of conversion changes we aim for. It gives a probability of group B converting better than A, and we can run checks to see if we should stop the test.

Regular ABs would take too long and the significance of the test wouldnt make much sense because after a few weeks we would be comparing apples to oranges.