Cookie Cats Game AB Testing

Setting the stage

To determine the better user experience, we conducted an A/B test, splitting our audience into two groups: Group A (gate30) and Group B (gate40).
Formulating Hypotheses

The hypothesis is that retention is higher when the gate is at level 30 than when it is at level 40.
Data Exploration

We begin by exploring our data, ensuring it's clean and understanding its basic characteristics through descriptive statistics and visualizations
Visualizing Data Distributions

Histograms and boxplots reveal the distribution and spread of our metric for both control and test groups, helping us visually inspect any potential differences.
Conducting the T-Test

Using a t-test, we compare the means of the two groups. The t-statistic and p-value inform us about the statistical significance of our results.
Results Interpretation

The p-value indicates whether we reject or fail to reject the null hypothesis. In this case:
Test Type: Non-Parametric
AB Hypothesis: Fail to Reject H0
p-value: 0.0509
Drawing Conclusions

If we want to keep retention high we should not move the gate from level 30 to level 40.
Next Steps

Modifying to include number of game rounds played or how much in-game purchases are made by the two AB-groups.