METHOD DOSSIERMSC-P-018Regression and econometrics

Predictive or causal regression: what are you trying to estimate?

The same regression family can serve two incompatible goals: predict future spend or estimate an intervention effect. The estimand, admissible variables, design and validation change.

Direct answer

Correctly separate a day-7 prediction from a total causal effect defined at day 0.

The same regression family can serve two incompatible goals: predict future spend or estimate an intervention effect. The estimand, admissible variables, design and validation change.

Shmueli, 2010 Hernán & Robins, Causal Inference: What If

The answer in 30 seconds

1

A predictive regression estimates a future value for a new unit and is judged on data not used for fitting.

2

A causal regression targets the effect of a defined intervention in a population and depends on an identification design, not model fit alone.

3

A post-treatment variable can improve prediction at day 7 while being inadmissible for the total causal effect decided at day 0.

Shmueli (2010) Arnold et al. (2020)

Three reading levels

1
Decision-maker: state whether the decision asks who will spend most or what sending the reminder would cause.
2
Practitioner: fix the forecast origin, intervention, comparator, outcome, population and variables available at each time.
3
Analyst: separate predictive loss, causal estimand, identification, estimation and uncertainty scope.

Concrete marketing situation

A retailer randomizes an email reminder at day 0 among 2,400 eligible customers. At day 7 it wants to predict 30-day spend with information then available. Separately, it wants the average total effect of sending the reminder. The day-7 engagement index is useful for the first question but occurs after assignment for the second.

Two questions, two estimands

Prediction: for a customer observed at day 7, estimate E[Y|X] and error on 600 holdout customers. Causality: among 2,400 randomized customers, estimate ATE = E[Y(1)−Y(0)], where T=1 assigns one reminder at day 0 and no other reminder through day 30, T=0 assigns none through day 30, and Y is 30-day spend in euros.

Prediction

μ(x) = E[Y | X=x]

Causality

ATE = E[Y(1) − Y(0)]

Hernán & Robins, Causal Inference: What If

Why the same regression can mislead

A low RMSE defines neither intervention nor counterfactual and therefore proves no causal effect.
Adding post-treatment engagement helps predict Y but blocks part of the T → M → Y path when reading the coefficient on T.
Selecting variables because they predict well does not replace causal reasoning about confounders, mediators and colliders.

Arnold et al. (2020) Ramspek et al. (2021) Cinelli, Forney & Pearl (2022)

Intuition and causal timeline

L and S precede T but do not cause it because T is randomized. The declared DAG is L,S → M; L,S → Y; T → M → Y; and T → Y. L is baseline intent, S prior spend and M day-7 engagement. At day 7, M is available predictive information. For the total effect of T decided at day 0, M is a mediator not to adjust for. The same column therefore changes status with the question and decision time.

T→ M→ YT → Y

Cinelli, Forney & Pearl (2022)

Required data and availability

customer_id; train/holdout split; baseline_intent_z and prior_spend_eur measured before day 0.
reminder_assigned is randomized assignment at day 0; engagement_index_7d is measured after assignment.
spend_30d_eur is the continuous outcome. The 1,800 train rows fit predictions; 600 holdout rows assess their internal transport.

Formal models and symbols

Day-7 prediction

Y = β₀ + β₁L + β₂S + β₃T + β₄M + ε

L: baseline_intent_z; S: prior_spend_eur; T: reminder_assigned; M: engagement_index_7d; Y: spend_30d_eur.

Total effect

Y = α₀ + τT + α₁L + α₂S + u

τ: mean difference adjusted for precision; M is excluded.

μ(x) is the conditional mean of Y for predictor vector x. β₀ and α₀ are intercepts; β₁ through β₄ and α₁ through α₂ are slopes; τ is the ATE; ε and u are the two model residuals.

Declared calculation, step by step

01
Generate or load 2,400 rows with seed 20260819 and preserve the fixed split.
02
Fit two predictions on train: without M, then with M; evaluate only on holdout.
03
Paired-resample the 600 holdout errors 2,000 times for the RMSE-improvement interval.
04
Estimate τ on all 2,400 units excluding M and compute the HC1 robust interval.
05
Separately fit the model with M to show why its T coefficient does not answer the total ATE.

End-to-end numerical example

Synthetic MSC dataset. It describes no real campaign.

End-to-end numerical example
Object	Specification	Result
RMSE	Prediction without M	6.1006 €
RMSE	Prediction with M	5.1734 €
ATE	Causal model without M	8.3667 €
βT \| M	Model adjusted for M	2.0657 €

Predictive gain

15.20%
95% CI [11.05%; 19.62%]

ATE

8.37 €
95% CI [7.87 €; 8.86 €]

Generator truth

8.80 €

Download the synthetic CSV Download the Python script

Validity assumptions

Prediction: stable definitions, day-7 availability, population and measurement process across train, holdout and future use.
Causality: correctly implemented randomization, consistency of both policies, no interference, observed outcome and non-differential follow-up.
The linear model estimates a mean difference and may improve precision; randomization, not regression form, identifies the ATE.

Hernán & Robins, Causal Inference: What If Arnold et al. (2020) Cinelli, Forney & Pearl (2022)

Diagnostics and uncertainty

01

Prediction: compare holdout RMSE, MAE and R²; check drift, out-of-support values and actual availability of M at forecast origin.

02

Causality: check assignment ratio, pre-treatment balance, attrition, contamination and HC1 robust standard errors.

03

The bootstrap interval for RMSE improvement describes this synthetic holdout. The HC1 ATE interval describes sampling noise under the declared design, not external validity.

Interpret without mixing objectives

The day-7 model including M reduces RMSE by 15.20%, 95% bootstrap CI [11.05%, 19.62%]. This is a predictive gain on this holdout. The causal model without M estimates the ATE at €8.37, HC1 CI [€7.87, €8.86], near the synthetic truth of €8.80. With M, the coefficient on T falls to €2.07 and no longer represents the total effect.

Supported and forbidden conclusions

Supported

Choose the model including M for a day-7 forecast in a comparable setting.
Report €8.37 as the estimated average total effect under the declared randomized protocol.

Forbidden

Present the RMSE reduction as proof that the reminder increases spend.
Read €2.07 as the total causal effect or generalize €8.37 to another population without new evidence.

Possible marketing decision

Keep two separate objects: a day-7 spend score that may use M, and an experimental total-effect estimate that excludes it. The first prioritizes an operational action at day 7; the second decides whether the reminder policy should be retained.

When to use each approach

01

Use predictive regression when the target is an individual future outcome and the forecast origin is explicit.

02

Use causal estimation when a decision changes T and a credible design identifies the counterfactual.

03

Do not use this linear model when the outcome, support or assignment mechanism materially differs without a new audit.

04

Alternatives: temporal validation and nonlinear models for prediction; randomized difference in means for the ATE; dedicated mediation analysis if the direct effect becomes the target.

Reproducible implementations

Python 3.13 and R 4.5 are references. SPSS 31 and SAS 9.4 are secondary syntaxes for the causal model; they certify neither standard-error equivalence nor assumptions.

Python 3.13

python msc-p018-reference.py --csv msc-p018-predictive-causal.csv

R 4.5

d <- read.csv("msc-p018-predictive-causal.csv")
train <- subset(d, split == "train"); hold <- subset(d, split == "holdout")
pre <- lm(spend_30d_eur ~ baseline_intent_z + prior_spend_eur + reminder_assigned, train)
post <- lm(spend_30d_eur ~ baseline_intent_z + prior_spend_eur + reminder_assigned + engagement_index_7d, train)
e0 <- hold$spend_30d_eur - predict(pre, hold)
e1 <- hold$spend_30d_eur - predict(post, hold)
rmse <- function(e) sqrt(mean(e^2))
c(pre_rmse=rmse(e0), post_rmse=rmse(e1), improvement=100*(rmse(e0)-rmse(e1))/rmse(e0))
set.seed(20260819)
b <- replicate(2000, { i <- sample(seq_along(e0), replace=TRUE); 100*(rmse(e0[i])-rmse(e1[i]))/rmse(e0[i]) })
quantile(b, c(.025, .975))
hc1 <- function(fit) {
  X <- model.matrix(fit); e <- residuals(fit); n <- nrow(X); p <- ncol(X)
  bread <- solve(crossprod(X)); meat <- crossprod(X, X * e^2)
  se <- sqrt(diag((n/(n-p)) * bread %*% meat %*% bread))
  cbind(estimate=coef(fit), se=se, lower=coef(fit)-1.959964*se, upper=coef(fit)+1.959964*se)
}
total <- lm(spend_30d_eur ~ reminder_assigned + baseline_intent_z + prior_spend_eur, d)
adjusted <- lm(spend_30d_eur ~ reminder_assigned + baseline_intent_z + prior_spend_eur + engagement_index_7d, d)
hc1(total)["reminder_assigned", ]
hc1(adjusted)["reminder_assigned", ]

IBM SPSS Statistics 31

GET DATA /TYPE=TXT /FILE='msc-p018-predictive-causal.csv'
 /DELCASE=LINE /DELIMITERS="," /ARRANGEMENT=DELIMITED /FIRSTCASE=2
 /VARIABLES=customer_id A5 split A7 baseline_intent_z F12.6 prior_spend_eur F12.6
 reminder_assigned F1.0 engagement_index_7d F12.6 spend_30d_eur F12.6.
REGRESSION /DEPENDENT spend_30d_eur
 /METHOD=ENTER reminder_assigned baseline_intent_z prior_spend_eur.

SAS 9.4

proc import datafile="msc-p018-predictive-causal.csv" out=p018 dbms=csv replace; guessingrows=max; run;
proc reg data=p018;
  model spend_30d_eur = reminder_assigned baseline_intent_z prior_spend_eur / hcc hccmethod=1;
run; quit;

Expected final deliverable

A dossier contains two separate records: prediction with day-7 origin, admissible variables, holdout metrics and drift; causality with day-0 protocol, DAG, ATE estimand, interval, threats and validity population. Outputs are never merged into one conclusion.

Scientific sources and evidence level

Shmueli (2010)
Conceptual foundation: goals, data, validation and criteria differ between explanation and prediction.
Hernán & Robins, Causal Inference: What If
Full methodological text: counterfactuals, ATE, randomization, consistency and interference.
Arnold et al. (2020)
Open-access methodological article: variable selection, evaluation and interpretation differ by question.
Cinelli, Forney & Pearl (2022)
Causal foundation: adjusting for a mediator can block the targeted total effect.
Ramspek et al. (2021)
Empirical review: it documents and bounds conflation frequency in a sample of 180 observational studies.

These texts support the method. The dataset, generating mechanism and results are synthetic MSC creations, not external empirical validation.

Dataset · Tool

Dataset · MSC-P-018-PREDICTIVE-CAUSALSynthetic randomized reminder, engagement and spend→

Method connections

Parent territoryMarketing response models: shape, delay and saturation RequiresCorrelation or causality: what can an analysis actually support?RequiresHow do you diagnose a marketing regression before interpreting it?Compare withHow do you validate a marketing forecast?