A/B Testing Quant Interview Guide
A/B testing quant interview guide for randomization, metric choice, power, p-values, sample size, and experiment pitfalls.
Candidates seeing experiment design, metrics, and inference prompts.
Start with the metric
An A/B test needs a clear metric, treatment definition, randomization unit, and decision rule. The test is only as useful as the design.
Randomization protects comparison
Random assignment helps make groups comparable, but implementation details and interference can still break the interpretation.
Concrete example
If testing a new onboarding flow, define whether success is signup, activation, retention, or revenue before computing statistics.
Power and sample size matter
A test with too little data may miss a real effect. A huge test can make tiny, irrelevant effects look statistically significant.
Common mistakes
Candidates often jump to p-values before defining the metric and randomization. Design comes before inference, because bad design invalidates clean math.
Practice the pattern
Use the LeetQuidity curriculum and calibration to turn this topic into a focused practice plan.