Player engagement on a mobile game
Executive Summary of Findings Here is a summary of our three key analytical questions and their statistically backed answers:
- 1-Day Retention: Did moving the gate to level 40 affect whether players returned after one day? Result: No significant impact. (p-value = 0.75) Conclusion: The change didn't make a difference to immediate, short-term player churn.
- 7-Day Retention: Did moving the gate to level 40 affect whether players returned after one week? Result: Yes, a statistically significant negative impact. (p-value = 0.002) Conclusion: The treatment (gate at 40) caused fewer players to stick with the game for a full week compared to the control (gate at 30).
- Player Engagement: Did moving the gate to level 40 affect how many rounds players played? Result: No significant impact. (p-value = 0.0509) Conclusion: We found no statistical evidence that moving the gate changed the overall engagement (total rounds played) of the player base.
Behavioral Interpretation Our analysis paints a very clear picture. The only statistically significant effect we found was a negative one. • Moving the gate later (to level 40) gave players no benefits. It did not improve 1-day retention and did not increase their engagement. • Instead, moving the gate later actively hurt long-term retention. • This suggests the earlier "friction point" at level 30 is not a bug; it's a feature of the core engagement loop. It might act as a "soft commitment" check, filtering out uncommitted players and solidifying the habit for those who pass it. By delaying this check, the game may be failing to "hook" players effectively, leading to them dropping off before the 7-day mark. Final, Actionable Recommendation Recommendation: Stick with the original game design and keep the gate at level 30.