Research & Evidence

Search for a method

Search titles, questions, territories and MSC identifiers.

    ← All methods
    METHOD DOSSIERMSC-P-014Experimentation and causalityVerified scientific dossier

    How do you design a marketing geo experiment?

    A geo experiment compares test and control regions before and during an intervention. The design must limit contamination, pre-period imbalance and temporal dependence.

    Scientific editorial team: Marketing Science Center

    Direct answer

    Estimate an aggregate incremental effect in the studied regions and period.

    A geo experiment compares test and control regions before and during an intervention. The design must limit contamination, pre-period imbalance and temporal dependence.

    Hernán & Robins, Causal Inference: What IfVaver & Koehler, 2011

    01

    Operational summary

    Scientific question and scope

    How do you design a marketing geo experiment?

    Supported

    Estimate an aggregate incremental effect in the studied regions and period.

    Forbidden

    Attribute the effect to each individual or extrapolate unreservedly to other markets.

    02

    Three reading levels

    1. 01

      Decision-maker — Connect the result to a declared decision, useful threshold and error cost.

    2. 02

      Practitioner — Fix population, unit, horizon, available variables and analysis rule before calculation.

    3. 03

      Analyst — Reproduce the calculation, quantify uncertainty and document diagnostics, failures and sensitivities.

    03

    Concrete marketing situation

    A geo experiment compares test and control regions before and during an intervention. The design must limit contamination, pre-period imbalance and temporal dependence.

    04

    Scientific question and scope

    How do you design a marketing geo experiment?

    Incremental outcome in treated geographies

    Required data

    geo × period

    Population, unit of analysis, origin date and horizon must be declared in the deliverable. Without them, the estimand silently changes.

    05

    Why a simple analysis can fail

    • Attribute the effect to each individual or extrapolate unreservedly to other markets.
    • Were non-overlapping geographies assigned before intervention and analyzed by assignment?
    • Is the counterfactual construction independent of post-treatment model selection?
    • Does the uncertainty method respect the small number of geographies and temporal dependence?

    06

    Method intuition

    The method does not automatically turn an association into evidence. It links a declared question to an estimand, specification, compatible data and a bounded interpretation rule.

    Incremental outcome in treated geographies

    07

    Required data

    Scientific symbol dictionary

    Y_test_post
    Observed aggregate outcome in test geographies during the declared intervention period. Unit: declared aggregate outcome · Type: number · Role: input
    Y0_test_post
    Externally supplied counterfactual aggregate outcome for those test geographies under no treatment; not fitted by this branch. Unit: same aggregate outcome · Type: number · Role: input
    z
    Two-sided normal critical value for the declared confidence level. Unit: dimensionless · Type: positive number · Role: parameter
    supplied_SE
    Externally supplied standard error for the conditional contrast; not estimated by this branch. Unit: same aggregate outcome · Type: positive number · Role: input
    conditional_increment
    Observed aggregate outcome minus the supplied counterfactual, conditional on both supplied inputs. Unit: aggregate outcome · Type: number · Role: output
    CI
    Normal interval around the conditional contrast using the supplied standard error. Unit: aggregate outcome · Type: ordered pair · Role: output

    Exact sealed engine inputs

    treated_geo_post_outcome
    Value: 120 · Unit: index · Type: number · Data status: synthetic
    counterfactual_post_outcome
    Value: 110 · Unit: index · Type: number · Data status: synthetic
    standard_error
    Value: 4 · Unit: index · Type: number · Data status: synthetic
    z_critical
    Value: 1.95996398454 · Unit: dimensionless · Type: number · Data status: parameter

    08

    Formal model

    Formal model

    conditional_incremental_outcome = treated_geo_post_outcome − counterfactual_post_outcome; CI = conditional_incremental_outcome ± z_critical × standard_error

    Evidence claims: MSC-P014-C01 · MSC-P014-C02 · MSC-P014-C03 · MSC-P014-C04 · MSC-P014-C05 · MSC-P014-C06

    Scientific question and scope

    Incremental outcome in treated geographies

    09

    Declared calculation

    1. 01

      Validate the treated aggregate, supplied counterfactual, supplied positive standard error and fixed intervention period.

    2. 02

      Subtract the supplied no-treatment counterfactual from the treated post-period outcome and form the normal interval.

    3. 03

      Report a conditional illustration only; a randomized geo design, counterfactual fit and geo-level uncertainty remain outside this branch.

    10

    Numerical example or application case

    Synthetic illustration — teaching values, not observed — synthetic data or declared parameters; no real observations

    Sealed inputs

    • treated_geo_post_outcome=120 [index]
    • counterfactual_post_outcome=110 [index]
    • standard_error=4 [index]
    • z_critical=1.95996398454 [dimensionless]

    Reproducible results

    • conditional_increment=10
    • CI95=[2.160144; 17.839856]

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    11

    Validity assumptions

    • Were non-overlapping geographies assigned before intervention and analyzed by assignment?
    • Is the counterfactual construction independent of post-treatment model selection?
    • Does the uncertainty method respect the small number of geographies and temporal dependence?

    12

    Diagnostics and uncertainty

    Diagnostics and uncertainty

    Were non-overlapping geographies assigned before intervention and analyzed by assignment? · Is the counterfactual construction independent of post-treatment model selection? · Does the uncertainty method respect the small number of geographies and temporal dependence?

    Bounded diagnostic

    Conditional illustration with supplied counterfactual and SE; not a fitted geo experiment.

    Explicit uncertainty contract · quantified_conditional

    Method: Deterministic reference-engine calculation on the sealed input slice.

    Target: Uncertainty of the supplied-counterfactual incremental outcome.

    Engine evidence: metric.ci_lower= · metric.ci_upper=

    Interpretation: The interval uses a supplied standard error and counterfactual; it does not validate a fitted geo experiment.

    Stop when

    Attribute the effect to each individual or extrapolate unreservedly to other markets.

    13

    Result interpretation

    • Estimate an aggregate incremental effect in the studied regions and period.
    • The result is conditional on the declared population, horizon, specification and diagnostics. It must not be extended to another decision without new justification.

    14

    Supported and forbidden conclusions

    Supported

    Estimate an aggregate incremental effect in the studied regions and period.

    Forbidden

    Attribute the effect to each individual or extrapolate unreservedly to other markets.

    15

    Possible marketing decision

    1. 01

      Estimate an aggregate incremental effect in the studied regions and period.

    2. 02

      The result is conditional on the declared population, horizon, specification and diagnostics. It must not be extended to another decision without new justification.

    16

    When to use — when to stop

    Use when

    Estimate an aggregate incremental effect in the studied regions and period.

    geo × period

    Stop when

    Attribute the effect to each individual or extrapolate unreservedly to other markets.

    Methodological alternatives

    • Run a randomized paired-geo experiment with randomization-based inference.
    • Use a prespecified synthetic-control or BSTS design only when randomization is unavailable and label it observational.
    • Use placebo periods and leave-one-geo-out sensitivity to probe counterfactual stability.

    17

    Reproducibility contract

    Python and R are the executable references. SPSS and SAS remain secondary syntaxes until checked on the same data, specification and diagnostics.

    The engine selects only this page’s explicit slice and branch. Its hash, outputs, and diagnostics remain sealed in the atomic dossier; no generic fallback branch is allowed.

    Slice fingerprint: 975b12b0153042485fcfcd6db1e894c2386ab04c6491184f6365423348c2fb46

    Formal model

    conditional_incremental_outcome = treated_geo_post_outcome − counterfactual_post_outcome; CI = conditional_incremental_outcome ± z_critical × standard_error

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    18

    Expected final deliverable

    1. 01

      How do you design a marketing geo experiment? — geo × period

    2. 02

      Y_test_post · Y0_test_post · z · supplied_SE · conditional_increment · CI

    3. 03

      conditional_incremental_outcome = treated_geo_post_outcome − counterfactual_post_outcome; CI = conditional_incremental_outcome ± z_critical × standard_error

    4. 04

      Were non-overlapping geographies assigned before intervention and analyzed by assignment? · Is the counterfactual construction independent of post-treatment model selection? · Does the uncertainty method respect the small number of geographies and temporal dependence?

    5. 05

      Estimate an aggregate incremental effect in the studied regions and period. / Attribute the effect to each individual or extrapolate unreservedly to other markets.

    19

    Scientific sources and evidence status

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    1. MSC-P014-C01 — The causal target contrasts mean potential outcomes under two defined interventions.
    2. MSC-P014-C02 — Consistency links the observed outcome to the potential outcome under the assigned policy.
    3. MSC-P014-C03 — The causal estimand assumes one customer's assignment does not alter another customer's outcome.
    4. MSC-P014-C04 — Geo experiments randomize non-overlapping regions to treatment conditions.
    5. MSC-P014-C05 — Geo-targeting operationalizes the assigned regional intervention.
    6. MSC-P014-C06 — The design must follow an explicit process.
    1. Hernán & Robins, Causal Inference: What If
    2. Vaver & Koehler, 2011

    Method connections

    Parent territoryMeasurement and causality: how can a marketing effect be established?Compare withWhen should you run a marketing experiment?Compare withHow do you estimate an effect with difference-in-differences?

    Read next

    MSC-H-001Measurement and causality: how can a marketing effect be established?→MSC-P-010When should you run a marketing experiment?→MSC-P-015How do you estimate an effect with difference-in-differences?→MSC-P-013How do you measure campaign incrementality with a control group?→