METHOD DOSSIERMSC-P-002Evidence foundations

Correlation or causality: what can an analysis actually support?

Correlation describes co-movement. A causal effect requires a credible counterfactual and identification assumptions that do not follow from the coefficient alone.

Direct answer

Separate an observed association from an identified causal effect, then show what adjustment changes.

Correlation describes co-movement. A causal effect requires a credible counterfactual and identification assumptions that do not follow from the coefficient alone.

Hernán & Robins, Causal Inference: What If Cinelli, Forney & Pearl, 2022

The answer in 30 seconds

1

A correlation or crude difference describes what is observed together. It defines neither the intervention nor the counterfactual outcome.

2

A causal effect compares mean outcomes under two well-defined interventions for a declared population and period.

3

In an observational study, causal interpretation depends on design, the causal graph and assumptions not proven by the coefficient.

Hernán & Robins, Causal Inference: What If

Three reading levels

1
Decision-maker: ask which decision would change and which counterfactual comparison supports it.
2
Practitioner: declare exposure, outcome, population, period and adjustment variables before analysis.
3
Analyst: separate estimand, identification and estimation; document positivity, sensitivity and external validity.

Concrete marketing situation

A team observes that retargeted customers buy more within 30 days. Yet retargeting mainly targets visitors with strong prior intent. The useful question is not only 'who buys more?' but 'what would the purchase rate of the same population have been under retargeting, then under no retargeting?'

Scientific question and estimand

Target population: eligible customers represented by a synthetic sample of 10,000 units. At time zero, A=1 is assignment to a fixed policy: one display impression, same creative, within 24 hours, capped at one impression over seven days. A=0 suppresses every retargeting impression for seven days. Y is purchase within 30 days and L is prior intent. Estimand: ATE = E[Y¹] − E[Y⁰], standardized to the sample's fixed empirical weights.

ATE = E[Y¹] − E[Y⁰]

Why the crude comparison can mislead

Confounding: prior intent affects both retargeting and purchase; L → A and L → Y create non-causal association.
Reverse causality: an early outcome signal may trigger exposure when timing is poorly defined.
Bad control: adjusting for a mediator or collider can block part of the effect or open a biasing path.

Cinelli, Forney & Pearl (2022)

Method intuition

The declared DAG is L → A, L → Y and A → Y. Variation in A comes from operational targeting, not random assignment. Standardization compares risks under A=1 and A=0 within each level of L, then weights strata by their target-population share. It closes the observed back-door path through L; it corrects no unmeasured confounder.

L→ A→ YL → Y

Greenland, Pearl & Robins (1999) Cinelli, Forney & Pearl (2022)

Required data

One row per prior-intent stratum and exposure level, with count n and purchases y.
Stable exposure definition, binary 30-day outcome, population, window and exclusions fixed before computing.
Confounders chosen through causal reasoning, not automatic variable selection.

Formal model and symbols

Association

RDcrude = E[Y|A=1] − E[Y|A=0]

A: policy assignment; Y: purchase within 30 days.

Standardization

E[Yᵃ] = Σₗ E[Y|A=a,L=l]P(L=l)

L: prior intent; a: declared intervention; ATE: difference of standardized risks.

Declared calculation, step by step

01
Compute crude risks by exposure.
02
Compute risk in each L × A cell.
03
Weight each risk by P(L) in the target population.
04
Subtract standardized risks and compute the standard error.

End-to-end numerical example

Synthetic dataset created for learning. It describes no real campaign.

End-to-end numerical example
L	A	n	Y=1	Risk
0	0	4,800	192	4%
0	1	1,200	72	6%
1	0	800	192	24%
1	1	3,200	896	28%

Download the synthetic CSV

Association

22.00% − 6.86% = 15.14 pp

ATE

14.80% − 12.00% = 2.80 pp

95% CI

[1.20; 4.40] pp

The gap between 15.14 and 2.80 points comes from prior-intent composition. The Wald interval handles sampling noise, not omitted confounding.

Validity assumptions

Consistency: observed assignment exactly matches the defined display policy, its comparator and time zero.
Conditional exchangeability: within L, no unmeasured determinant still affects both A and Y.
Positivity: each level of L contains exposed and unexposed customers.
No interference: one customer's exposure does not change another customer's outcome.

Hernán & Robins, Causal Inference: What If Cinelli, Forney & Pearl (2022)

Diagnostics and uncertainty

01

Check cell counts and risks, timing and treatment definition.

02

Check positivity: no empty cell; inspect extreme exposure probabilities with individual data.

03

The Wald CI assumes independent binomial cells and conditions on observed counts and weights. If weights are estimated, use bootstrap or an influence function. Report sensitivity to unmeasured confounding and DAG choices separately.

Interpreting both results

The crude association answers 'what difference is observed between received groups?' The standardized ATE answers 'what mean difference would arise under two policies for the declared population?', only if identification assumptions are credible. The interval quantifies sampling uncertainty under the model; it does not automatically cover confounding bias.

Supported and forbidden conclusions

Supported

Describe a crude association of 15.14 points in this dataset.
Under the declared assumptions, estimate a standardized ATE of 2.80 points for this population.

Forbidden

Say that the coefficient or significance proves retargeting impact.
Generalize to another population, period or treatment version without new justification.

Possible marketing decision

The manager may decide to run an A/B test when the stakes justify randomized variation, or provisionally use the standardized estimate with a caution margin and sensitivity analysis. The calculation informs the decision; it does not choose budget or deployment.

When to use and when not to use

01

Use standardization when treatment, outcome, population and a defensible pre-treatment adjustment set are available.

02

Do not use it as a causal shortcut when positivity fails, treatment is ambiguous or major confounders are missing.

03

Alternatives: A/B or geo experiments, difference-in-differences, instrumental variables, synthetic control and sensitivity analyses depending on design.

Reproducible implementations

Same CSV, same standardization, same expected outputs. Software executes the calculation; it does not validate causal assumptions.

Python 3.13

# Python 3.13, standard library only
import csv, math
rows = list(csv.DictReader(open("msc-p002-retargeting.csv", encoding="utf-8")))
for r in rows:
    r.update({k:int(r[k]) for k in ("prior_intent","retargeted","n","purchases")})
N = sum(r["n"] for r in rows)
weights = {l:sum(r["n"] for r in rows if r["prior_intent"]==l)/N for l in (0,1)}
risk = {(r["prior_intent"],r["retargeted"]):r["purchases"]/r["n"] for r in rows}
crude = {a:sum(r["purchases"] for r in rows if r["retargeted"]==a)/sum(r["n"] for r in rows if r["retargeted"]==a) for a in (0,1)}
standardized = {a:sum(weights[l]*risk[l,a] for l in (0,1)) for a in (0,1)}
ate = standardized[1]-standardized[0]
se = math.sqrt(sum(weights[l]**2*(risk[l,1]*(1-risk[l,1])/next(r["n"] for r in rows if r["prior_intent"]==l and r["retargeted"]==1)+risk[l,0]*(1-risk[l,0])/next(r["n"] for r in rows if r["prior_intent"]==l and r["retargeted"]==0)) for l in (0,1)))
print(crude[1]-crude[0], ate, (ate-1.959964*se, ate+1.959964*se))

R 4.5

# R 4.5
d <- read.csv("msc-p002-retargeting.csv")
d$risk <- d$purchases / d$n
w <- aggregate(n ~ prior_intent, d, sum); w$weight <- w$n / sum(w$n)
d <- merge(d, w[c("prior_intent","weight")], by="prior_intent")
crude <- aggregate(cbind(purchases,n) ~ retargeted, d, sum)
crude$risk <- crude$purchases / crude$n
std <- aggregate(I(weight*risk) ~ retargeted, d, sum)
ate <- std[std$retargeted==1,2] - std[std$retargeted==0,2]
cell <- transform(d, v=weight^2*risk*(1-risk)/n)
se <- sqrt(sum(cell$v))
c(crude_RD=diff(crude$risk), ATE=ate, lo=ate-qnorm(.975)*se, hi=ate+qnorm(.975)*se)

IBM SPSS Statistics 31

* IBM SPSS Statistics 31.
GET DATA /TYPE=TXT /FILE='msc-p002-retargeting.csv' /FIRSTCASE=2 /DELCASE=LINE
 /DELIMITERS=',' /VARIABLES=prior_intent F1.0 retargeted F1.0 n F8.0 purchases F8.0.
MATRIX.
GET x /VARIABLES=prior_intent retargeted n purchases.
COMPUTE risk=x(:,4)&/x(:,3).
COMPUTE N=CSUM(x(:,3)).
COMPUTE w={CSUM(x(1:2,3))/N;CSUM(x(1:2,3))/N;CSUM(x(3:4,3))/N;CSUM(x(3:4,3))/N}.
COMPUTE p1=CSUM((x(:,2)=1)&*x(:,4))/CSUM((x(:,2)=1)&*x(:,3)).
COMPUTE p0=CSUM((x(:,2)=0)&*x(:,4))/CSUM((x(:,2)=0)&*x(:,3)).
COMPUTE s1=CSUM((x(:,2)=1)&*w&*risk); s0=CSUM((x(:,2)=0)&*w&*risk).
COMPUTE se=SQRT(CSUM(w&*w&*risk&*(1-risk)&/x(:,3))).
PRINT {p1-p0;s1-s0;s1-s0-1.959964*se;s1-s0+1.959964*se}
 /TITLE='crude_RD ATE CI_low CI_high'.
END MATRIX.

SAS 9.4

/* SAS 9.4 */
proc import datafile='msc-p002-retargeting.csv' out=d dbms=csv replace; guessingrows=max; run;
proc sql;
 create table cell as select a.*, purchases/n as risk,
  (select sum(n) from d b where b.prior_intent=a.prior_intent)/(select sum(n) from d) as weight
 from d a;
 create table crude as select retargeted, sum(purchases)/sum(n) as risk from cell group by retargeted;
 create table std as select retargeted, sum(weight*risk) as risk,
  sum(weight*weight*risk*(1-risk)/n) as variance from cell group by retargeted;
quit;
data result; merge crude(where=(retargeted=0) rename=(risk=c0)) crude(where=(retargeted=1) rename=(risk=c1))
 std(where=(retargeted=0) rename=(risk=s0 variance=v0)) std(where=(retargeted=1) rename=(risk=s1 variance=v1));
 crude_RD=c1-c0; ATE=s1-s0; SE=sqrt(v0+v1); CI_low=ATE-1.959964*SE; CI_high=ATE+1.959964*SE; run;
proc print data=result; var crude_RD ATE CI_low CI_high; run;

Expected final deliverable

A decision dossier contains the causal question, versioned DAG, estimand, population and period, data dictionary, crude and adjusted calculation, interval, positivity diagnostics, non-testable assumptions, sensitivity, generalization limits and proposed human decision.

Scientific sources and evidence level

Hernán & Robins, Causal Inference: What If
Full methodological text: counterfactuals, consistency, exchangeability, positivity, standardization and interference.
Cinelli, Forney & Pearl (2022)
Methodological article on good and bad controls, back-door paths, mediators and colliders.
Greenland, Pearl & Robins (1999)
Formal foundation for causal graphs and path-based identification criteria.

These sources support the method. The dataset and numerical results are synthetic MSC creations; they are not external empirical validation.

Dataset · Tool

Dataset · MSC-P-002-RETARGETINGSynthetic retargeting and prior intent→

Method connections

Parent territoryMeasurement and causality: how can a marketing effect be established?