Research & Evidence

Search for a method

Search titles, questions, territories and MSC identifiers.

    ← All methods
    METHOD DOSSIERMSC-P-015Experimentation and causalityVerified scientific dossier

    How do you estimate an effect with difference-in-differences?

    Difference-in-differences compares change in a treated group with change in an untreated group. Identification relies on credible parallel trends without treatment.

    Scientific editorial team: Marketing Science Center

    Direct answer

    Estimate an ATT for groups and periods covered by the design.

    Difference-in-differences compares change in a treated group with change in an untreated group. Identification relies on credible parallel trends without treatment.

    Hernán & Robins, Causal Inference: What IfArnold et al., 2020

    01

    Operational summary

    Scientific question and scope

    How do you estimate an effect with difference-in-differences?

    Supported

    Estimate an ATT for groups and periods covered by the design.

    Forbidden

    Prove future parallel trends from pre-period data alone.

    02

    Three reading levels

    1. 01

      Decision-maker — Connect the result to a declared decision, useful threshold and error cost.

    2. 02

      Practitioner — Fix population, unit, horizon, available variables and analysis rule before calculation.

    3. 03

      Analyst — Reproduce the calculation, quantify uncertainty and document diagnostics, failures and sensitivities.

    03

    Concrete marketing situation

    Difference-in-differences compares change in a treated group with change in an untreated group. Identification relies on credible parallel trends without treatment.

    04

    Scientific question and scope

    How do you estimate an effect with difference-in-differences?

    ATT in a two-group, two-period design

    Required data

    unit × group × period

    Population, unit of analysis, origin date and horizon must be declared in the deliverable. Without them, the estimand silently changes.

    05

    Why a simple analysis can fail

    • Prove future parallel trends from pre-period data alone.
    • Are pre-treatment trends credible for the chosen comparison group?
    • Could units anticipate treatment or change composition before adoption?
    • For staggered adoption, is group-time ATT used instead of an invalid two-way fixed-effects aggregation?

    06

    Method intuition

    The method does not automatically turn an association into evidence. It links a declared question to an estimand, specification, compatible data and a bounded interpretation rule.

    ATT in a two-group, two-period design

    07

    Required data

    Scientific symbol dictionary

    Y_T_post
    Mean outcome for treated-group units after intervention. Unit: outcome · Type: number · Role: input
    Y_T_pre
    Mean outcome for the same treated group in the declared pre-period. Unit: outcome · Type: number · Role: input
    Y_C_post
    Mean outcome for the declared comparison group in the post-period. Unit: outcome · Type: number · Role: input
    Y_C_pre
    Mean outcome for the same comparison group in the pre-period. Unit: outcome · Type: number · Role: input
    z
    Two-sided normal critical value for the declared confidence level. Unit: dimensionless · Type: positive number · Role: parameter
    cluster_SE
    Externally supplied cluster-robust standard error for the DiD contrast in this sealed illustration. Unit: outcome · Type: positive number · Role: input
    ATT_DiD
    Two-period difference-in-differences contrast, interpreted causally only under the declared identification assumptions. Unit: outcome · Type: number · Role: output
    CI
    Normal interval based on the supplied cluster-robust standard error. Unit: outcome · Type: ordered pair · Role: output

    Exact sealed engine inputs

    treated_pre
    Value: 100 · Unit: index · Type: number · Data status: synthetic
    treated_post
    Value: 120 · Unit: index · Type: number · Data status: synthetic
    control_pre
    Value: 90 · Unit: index · Type: number · Data status: synthetic
    control_post
    Value: 98 · Unit: index · Type: number · Data status: synthetic
    standard_error
    Value: 4 · Unit: index · Type: number · Data status: synthetic
    z_critical
    Value: 1.95996398454 · Unit: dimensionless · Type: number · Data status: parameter

    08

    Formal model

    Formal model

    att_did = (treated_post − treated_pre) − (control_post − control_pre); CI = att_did ± z_critical × standard_error

    Evidence claims: MSC-P015-C01 · MSC-P015-C02 · MSC-P015-C03 · MSC-P015-C04 · MSC-P015-C05 · MSC-P015-C06

    Scientific question and scope

    ATT in a two-group, two-period design

    09

    Declared calculation

    1. 01

      Lock treated and comparison groups, timing, outcome definition and the supplied cluster-robust uncertainty specification.

    2. 02

      Compute each group’s change, subtract the comparison change from the treated change, and form the declared interval.

    3. 03

      Interpret as ATT only if parallel trends, no anticipation, overlap and valid comparison timing are defensible.

    10

    Numerical example or application case

    Worked example — synthetic data or declared parameters; no real observations

    Sealed inputs

    • treated_pre=100 [index]
    • treated_post=120 [index]
    • control_pre=90 [index]
    • control_post=98 [index]
    • standard_error=4 [index]
    • z_critical=1.95996398454 [dimensionless]

    Reproducible results

    • ATT_DiD=12
    • CI95=[4.160144; 19.839856]

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    11

    Validity assumptions

    • Are pre-treatment trends credible for the chosen comparison group?
    • Could units anticipate treatment or change composition before adoption?
    • For staggered adoption, is group-time ATT used instead of an invalid two-way fixed-effects aggregation?

    12

    Diagnostics and uncertainty

    Diagnostics and uncertainty

    Are pre-treatment trends credible for the chosen comparison group? · Could units anticipate treatment or change composition before adoption? · For staggered adoption, is group-time ATT used instead of an invalid two-way fixed-effects aggregation?

    Bounded diagnostic

    Interpretation still requires parallel trends, no anticipation and an appropriate clustered variance design.

    Explicit uncertainty contract · quantified_conditional

    Method: Deterministic reference-engine calculation on the sealed input slice.

    Target: Uncertainty of the difference-in-differences ATT illustration.

    Engine evidence: metric.ci_lower= · metric.ci_upper=

    Interpretation: The interval is conditional on the supplied cluster-robust standard error and does not establish parallel trends.

    Stop when

    Prove future parallel trends from pre-period data alone.

    13

    Result interpretation

    • Estimate an ATT for groups and periods covered by the design.
    • The result is conditional on the declared population, horizon, specification and diagnostics. It must not be extended to another decision without new justification.

    14

    Supported and forbidden conclusions

    Supported

    Estimate an ATT for groups and periods covered by the design.

    Forbidden

    Prove future parallel trends from pre-period data alone.

    15

    Possible marketing decision

    1. 01

      Estimate an ATT for groups and periods covered by the design.

    2. 02

      The result is conditional on the declared population, horizon, specification and diagnostics. It must not be extended to another decision without new justification.

    16

    When to use — when to stop

    Use when

    Estimate an ATT for groups and periods covered by the design.

    unit × group × period

    Stop when

    Prove future parallel trends from pre-period data alone.

    Methodological alternatives

    • Estimate group-time ATT with not-yet-treated or never-treated controls for staggered adoption.
    • Use an event-study design to expose dynamic effects and pre-trend diagnostics.
    • Prefer randomized rollout when intervention timing can be assigned.

    17

    Reproducibility contract

    Python and R are the executable references. SPSS and SAS remain secondary syntaxes until checked on the same data, specification and diagnostics.

    The engine selects only this page’s explicit slice and branch. Its hash, outputs, and diagnostics remain sealed in the atomic dossier; no generic fallback branch is allowed.

    Slice fingerprint: 928e93319b64e017684dad7e878d79a85a38b2296a0925cc3597773489b3fc56

    Available asset · MSC-005

    Promotion incrementality

    Available asset · MSC-T02

    DiD calculator

    Formal model

    att_did = (treated_post − treated_pre) − (control_post − control_pre); CI = att_did ± z_critical × standard_error

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    18

    Expected final deliverable

    1. 01

      How do you estimate an effect with difference-in-differences? — unit × group × period

    2. 02

      Y_T_post · Y_T_pre · Y_C_post · Y_C_pre · z · cluster_SE · ATT_DiD · CI

    3. 03

      att_did = (treated_post − treated_pre) − (control_post − control_pre); CI = att_did ± z_critical × standard_error

    4. 04

      Are pre-treatment trends credible for the chosen comparison group? · Could units anticipate treatment or change composition before adoption? · For staggered adoption, is group-time ATT used instead of an invalid two-way fixed-effects aggregation?

    5. 05

      Estimate an ATT for groups and periods covered by the design. / Prove future parallel trends from pre-period data alone.

    19

    Scientific sources and evidence status

    Verified dossier: claims are linked to passage-level sources, the Python/R calculation is reproduced, limits are explicit, and independent scientific review is complete.

    1. MSC-P015-C01 — The causal target contrasts mean potential outcomes under two defined interventions.
    2. MSC-P015-C02 — Consistency links the observed outcome to the potential outcome under the assigned policy.
    3. MSC-P015-C03 — Fitting a regression does not itself establish a causal design.
    4. MSC-P015-C04 — Identification requires explicit parallel-trends and no-anticipation assumptions.
    5. MSC-P015-C05 — The framework distinguishes common and staggered adoption designs.
    6. MSC-P015-C06 — Covariate adjustment depends on the identification design.
    1. Hernán & Robins, Causal Inference: What If
    2. Arnold et al., 2020
    3. Callaway & Sant’Anna, 2021

    Dataset · Tool

    Dataset · MSC-005Promotion incrementality→Tool · MSC-T02DiD calculator→

    Method connections

    Parent territoryMeasurement and causality: how can a marketing effect be established?RequiresCorrelation or causality: what can an analysis actually support?Compare withHow do you design a marketing geo experiment?

    Read next

    MSC-H-001Measurement and causality: how can a marketing effect be established?→MSC-P-002Correlation or causality: what can an analysis actually support?→MSC-P-014How do you design a marketing geo experiment?→MSC-P-017How do you detect selection, contamination and attrition in an experiment?→