Skip to content
  • There are no suggestions because the search field is empty.

How "odds to win" are calculated?

Odds to Win (OTW) represents the probability that a specific variation is performing better than the Control. It measures the area under the probability distribution curve that sits to the right of zero (the point of no difference).

OTW has no minimum sample size threshold. You could have an OTW of 100% after only 10 sessions, but that doesn't mean the result is reliable yet, it just means that, based only on those 10 sessions, the variant is ahead. We recommend combining this with other metrics including confidence interval and statistical significance when analysing your results.

Calculation Methods by Metric

We use different models depending on the data type and based on the type of goal metrics:

  • Binomial Bayesian: Used for binomial distribution goal metrics e.g. "Yes/No" events like Conversion Rate (converted vs. not converted).

  • Gaussian Bayesian: Used for goal metrics that do not follow binomial distribution e.g. continuous numbers like Average Order Value. This accounts for the mean and the standard deviation of order values.

  • Combined Model: For some goal metrics, we combine the above two to provide an accurate estimate.

These tailored testing approaches enable us to accurately assess the statistical significance of the results, taking into consideration the unique characteristics of each metric.​