Why Causal Inference is Hard
Most "X caused Y" takes are just good lighting on a coincidence.
Three gremlins:
Confounding โ A third thing nudges both X and Y. Heavy users see more prompts AND convert more.
Selection bias โ Your sample isn't the population. Only engaged users see your experiment.
Interference โ Users collide. Price test for drivers changes rider behavior.
Design beats modeling. If users interact, don't A/Bโuse switchbacks. Split by time, not random rows. Pre-commit your rules before you ship.