When should you run a marketing experiment?
An experiment is preferred when the question concerns the effect of an action and assignment can be controlled without disproportionate risk. Otherwise, an observational design must state its assumptions.
Direct answer
Choose a design proportionate to the decision and risk.
An experiment is preferred when the question concerns the effect of an action and assignment can be controlled without disproportionate risk. Otherwise, an observational design must state its assumptions.
Decision value > experiment cost + riskESTIMAND
Estimation profile
- Unit of analysis
- Assigned unit × group × periodControlled scientific terminology
eligible unit × assignment - Exact estimand
Design candidate for a causal estimand- Model or deliverable
Decision value > experiment cost + risk
01—08
Verifiable analysis framework
- 01Question
When should you run a marketing experiment?
- 02Estimand
Design candidate for a causal estimand · eligible unit × assignment
- 03Data
eligible_population · treatment · outcome · assignment_unit
- 04Model or deliverable
Decision value > experiment cost + risk
- 05Declared calculation
Decision value > experiment cost + risk → Design candidate for a causal estimand
- 06Uncertainty checks
Ethics and risk · Contamination · Operational feasibility
- 07Method-specific validation
Check data, estimate stability and interpretation limits. Ethics and risk · Contamination · Operational feasibility
- 08Limitations
Demand strong causal claims when no credible variation exists.
Required variables
eligible_populationtreatmentoutcomeassignment_unit
Checks · Controlled scientific terminology
- Ethics and risk
- Contamination
- Operational feasibility
Choose a design proportionate to the decision and risk.
Demand strong causal claims when no credible variation exists.
Scientific sources
1 source
Method connections