A/B Testing
      Back to home
      1. Visually Help Center
      2. A/B Testing

      How do I know when to stop a test once statistical significance is reached?

      When the test reaches statistical significance, you will see the following message on the experience report page: 

      Screenshot 2024-03-22 at 21.11.18

      At this point, you can simply change the allocation of the winner to 100%.

      How "odds to win" are calculated?

      We use different statistical methods based on the main KPI of the experience.

      For goal metrics that have a binomial distribution, like Conversion Rate, we utilize the binomial Bayesian test.
      ​

      However, for goal metrics that do not follow a binomial distribution, such as AOV (Average Order Value) and PSV (Per-Session Value), we utilize the Gaussian Bayesian test. These tailored testing approaches enable us to accurately assess the statistical significance of the results, taking into consideration the unique characteristics of each metric.