Power Analysis in Marketing: A Hands-On Introduction

Statistical power analysis is a crucial but often misunderstood aspect of A/B testing in marketing. Power is defined as the probability of correctly detecting a true effect if one exists, and it is the complement of a Type II error. The process involves defining null and alternative hypotheses, setting a significance level (α), and calculating a test statistic to compare against a critical value. To compute power, one must assume a true effect size, which can be determined by what is considered a meaningful business impact or by analyzing historical data.