METHOD DOSSIERMSC-P-026Choice models

Logit vs Probit: how do you choose for purchase probability?

Compare both links on the same data using probabilities, marginal effects, calibration, Brier, log loss, AUC and tail regions. Raw coefficients live on different scales.

Direct answer

Choose a link using probabilities, holdout diagnostics, interpretation needs and decision thresholds.

Compare both links on the same data using probabilities, marginal effects, calibration, Brier, log loss, AUC and tail regions. Raw coefficients live on different scales.

Amemiya, 1975 Gündüz & Fokoué, 2015

The answer in 30 seconds

1

Logit and Probit both model a binary probability from the same linear predictor. The former uses the logistic function, the latter the cumulative normal function.

2

Their raw coefficients are not directly comparable because the latent scale changes. Compare predicted probabilities, marginal effects and performance on the same observations instead.

3

In the synthetic example, both links yield almost the same average marginal effect and very similar holdout scores. This does not prove universal interchangeability.

Amemiya (1975), Qualitative Response Models Gündüz & Fokoué (2015)

Three reading levels

1
Decision-maker: ask whether the link choice changes a probability, segment or decision at a relevant threshold.
2
Practitioner: keep data, variables, train-test split and metrics identical for both models.
3
Analyst: distinguish coefficient scale, probability-scale marginal effect, calibration, discrimination and proper score.

Concrete marketing situation

A CRM team wants to estimate 30-day purchase probability from prior engagement, displayed discount and mobile device. It is choosing between Logit and Probit. The useful question is not 'which coefficient is larger?' but 'do probabilities, marginal effects and out-of-sample diagnostics lead to a different decision?'

Scientific question and estimand

Target population: eligible customers represented by 1,200 synthetic observations. Unit: one customer. Outcome Y: purchase within 30 days. Predictive, non-causal comparison. Estimands: P(Y=1|X) and the average marginal effect of engagement, averaged over the 400 holdout covariate profiles. The first 800 rows are used for estimation.

P(Y=1|X=x) · AME engagement on holdout X

Amemiya (1975), Qualitative Response Models

Why a superficial comparison fails

Comparing 0.991 with the Probit coefficient 0.596 confounds effect and latent-scale normalization.
Choosing solely by training AIC ignores calibration and out-of-sample generalization.
Comparing AUC alone ignores calibration because AUC reflects ranking, not the exact probability level.

Gündüz & Fokoué (2015) Dimitriadis et al. (2023)

What actually changes between Logit and Probit

What actually changes between Logit and Probit
Dimension	Logit	Probit	Useful comparison
Link	log[p/(1−p)]	Φ⁻¹(p)	Predicted probability
β	Log-odds scale	Latent-normal scale	Do not compare raw magnitudes
AME	Probability-scale change averaged over declared X		Comparable on the probability scale

Gündüz & Fokoué (2015)

Required data

purchase equals 1 for a purchase within 30 days and 0 otherwise. One row represents one customer.
prior_engagement_z is drawn from N(0,1) and retained without post-draw restandardization. One unit belongs to this generating scale.
discount_pct takes 0, 5, 10, 15 or 20%. The model uses discount10 = discount_pct / 10, so one unit equals 10 discount points.
mobile equals 1 for a mobile device and 0 otherwise. split is train for 800 rows and holdout for 400 rows.

Formal model and symbols

AMEₖ = n⁻¹ Σᵢ f(ηᵢ)βₖ, f = Λ(ηᵢ)[1−Λ(ηᵢ)] or φ(ηᵢ)

Shared predictor

ηᵢ = β₀ + β₁ engagementᵢ + β₂ discount10ᵢ + β₃ mobileᵢ

i indexes a customer; n=400 fixed holdout profiles. β is estimated by maximum likelihood on 800 training rows.

Logit

pᵢ = Λ(ηᵢ) = 1 / (1 + exp(−ηᵢ))

Λ is the logistic CDF and f its density. A coefficient shifts log-odds, not probability by a constant amount.

Probit

pᵢ = Φ(ηᵢ)

Φ is the standard normal CDF and φ its density. The coefficient lives on a different latent scale.

For engagement, AME is the average local probability derivative per one unit of the generating scale, reported in percentage points. It is not the finite change from X to X+1.

Nelder & Wedderburn (1972) Hill, Griffiths & Lim (2018), chapter 16

Declared calculation, step by step

01
Freeze the sample: 800 training rows and 400 holdout rows.
02
Fit both links with the same outcome, predictors and intercept.
03
Compute holdout probabilities, then Brier, log loss, AUC and calibration-in-the-large.
04
Compute the engagement AME over the 400 fixed profiles and its 95% delta-method CI using the Fisher covariance estimated on training data.
05
Inspect two tail profiles and check whether a business threshold changes side.

Nelder & Wedderburn (1972) Hill, Griffiths & Lim (2018), chapter 16 Dimitriadis et al. (2023)

End-to-end numerical example

Synthetic dataset created for learning. It describes no real campaign.

Seed 20260819. The outcome is generated from a logistic teaching process, which favors neither a universal conclusion nor a real-world claim.

MSC-P-026 · holdout n=400 · purchase rate 38.75%
Metric	Logit	Probit
β engagement	0.991442	0.596133
AME engagement	19.8738 pp	19.8780 pp
95% CI AME	[17.1598; 22.5878] pp	[17.1767; 22.5794] pp
Brier	0.175319	0.175367
Log-loss	0.527456	0.527414
AUC	0.809401	0.809743
Calibration p̄−ȳ	+0.006011	+0.006850
p(engagement=−2)	0.084485	0.076382
p(engagement=+2)	0.829609	0.830134

Tail profiles fix discount_pct=10, discount10=1 and mobile=1; only engagement changes between values −2 and +2 on the generating scale.

Download the synthetic CSV

Reading: metric gaps are tiny on this one holdout; the tail profile at −2 differs by 0.81 percentage point. Sampling uncertainty is reported for the AME, not for every metric difference.

The AME interval conditions on the 400 observed holdout covariate profiles and uses the Fisher covariance from the 800-row training fit. It does not quantify uncertainty in the profile distribution or in metric gaps.

Validity assumptions

Specification: the same linear predictor contains the relevant variables and transformations for each link.
Independence for the displayed inference: no residual clustering by household, store or campaign. Otherwise adapt uncertainty.
Overlap: no perfect separation and no profiles outside the training support.
The true link form and transportability to another period cannot be established by model fit alone.

Diagnostics and uncertainty

Estimation

Both Fisher-scoring fits converge, in 6 and 8 iterations. This standard-library script does not compute separation or influence diagnostics; they remain required before operational use.

Probability quality

Brier and log loss summarize probability accuracy; calibration-in-the-large shows average overprediction of 0.60 and 0.69 percentage point.

Discrimination and uncertainty

AUC is near 0.81 for both links but does not assess calibration. Delta-method AME intervals overlap almost completely; this is not an equivalence test.

Dimitriadis et al. (2023)

Interpreting the results

The 0.991/0.596 ratio mainly reflects scale, not an effect nearly twice as large.
AME values of 19.87 and 19.88 points are numerically very close on the probability scale in this support.
Neither model dominates every metric: Logit has slightly lower Brier, Probit slightly better log loss and AUC.

Supported and forbidden conclusions

Supported

State that Logit and Probit probabilities and marginal effects are very close in this synthetic holdout.
Provisionally prefer Logit for odds-ratio interpretation, or retain Probit when a latent-normal representation is substantively justified.

Forbidden

Infer that a larger coefficient means a stronger marketing effect across links.
Present the predictive engagement association as the causal effect of an intervention.
Generalize the tiny difference in this example to every sample size, specification, rare class or tail region.

Possible marketing decision

For this dossier, the manager may retain Logit as the working model because point estimates are very close on this holdout and odds ratios ease communication. This finding is not an equivalence test. Probability comparisons and a reassessment rule remain necessary if population, event rarity or thresholds change.

Gündüz & Fokoué (2015)

When to use and when not to use

Use

Use Logit or Probit for a binary outcome when the conditional probability and its diagnostics are the target.

Do not use alone

Do not use them alone to identify a causal effect, handle longitudinal dependence or model time to event.

Alternatives

Alternatives: complementary log-log for justified asymmetry, mixed models or GEE for dependence, survival models for timing, causal methods for interventions.

Reproducible implementations

Download the Python reference script

Python 3.13.7 · standard library · reference

python msc-p026-reference.py --csv msc-p026-logit-probit.csv
# rows=1200 train=800 holdout=400
# Fits Logit and Probit by Fisher scoring.
# Reports AME delta-method CI, Brier, log-loss, AUC, calibration and declared tail profiles.

R 4.5.1 · stats/utils · reference

# R 4.5.1, base stats/utils only
d <- read.csv("msc-p026-logit-probit.csv")
train <- subset(d, split == "train")
holdout <- subset(d, split == "holdout")
formula <- purchase ~ prior_engagement_z + I(discount_pct/10) + mobile

auc <- function(y, p) mean(outer(p[y==1], p[y==0], ">") + .5*outer(p[y==1], p[y==0], "=="))
for (link in c("logit", "probit")) {
  fit <- glm(formula, data=train, family=binomial(link=link))
  eta <- predict(fit, newdata=holdout, type="link")
  p <- predict(fit, newdata=holdout, type="response")
  dmu <- if (link == "logit") p*(1-p) else dnorm(eta)
  ame <- mean(dmu*coef(fit)["prior_engagement_z"])
  ame_fun <- function(b) {
    e <- model.matrix(formula, holdout) %*% b
    mean((if (link=="logit") plogis(e)*(1-plogis(e)) else dnorm(e))*b[2])
  }
  eps <- 1e-5; b <- coef(fit)
  grad <- sapply(seq_along(b), function(j) {u<-v<-b;u[j]<-u[j]+eps;v[j]<-v[j]-eps;(ame_fun(u)-ame_fun(v))/(2*eps)})
  se <- sqrt(drop(t(grad) %*% vcov(fit) %*% grad))
  print(c(link=link, AME=ame, low=ame-1.959964*se, high=ame+1.959964*se,
    Brier=mean((p-holdout$purchase)^2), LogLoss=-mean(holdout$purchase*log(p)+(1-holdout$purchase)*log(1-p)),
    AUC=auc(holdout$purchase,p), Calibration=mean(p)-mean(holdout$purchase)))
}

IBM SPSS Statistics 31.0.0.0 · secondary

* IBM SPSS Statistics 31, secondary fit syntax.
GET DATA /TYPE=TXT /FILE='msc-p026-logit-probit.csv' /FIRSTCASE=2
 /DELIMITERS=',' /VARIABLES=customer_id A5 split A7
 prior_engagement_z F10.6 discount_pct F2.0 mobile F1.0 purchase F1.0.
COMPUTE discount_10=discount_pct/10.
TEMPORARY. SELECT IF split='train'.
GENLIN purchase (REFERENCE=FIRST) WITH prior_engagement_z discount_10 mobile
 /MODEL prior_engagement_z discount_10 mobile INTERCEPT=YES DISTRIBUTION=BINOMIAL LINK=LOGIT.
TEMPORARY. SELECT IF split='train'.
GENLIN purchase (REFERENCE=FIRST) WITH prior_engagement_z discount_10 mobile
 /MODEL prior_engagement_z discount_10 mobile INTERCEPT=YES DISTRIBUTION=BINOMIAL LINK=PROBIT.
* Use a saved model or OMS workflow to score the unchanged holdout.

SAS 9.4 TS1M8 · secondary

/* SAS 9.4 TS1M8, secondary fit syntax */
proc import datafile='msc-p026-logit-probit.csv' out=d dbms=csv replace; guessingrows=max; run;
data d; set d; discount_10=discount_pct/10; run;
proc logistic data=d(where=(split='train'));
 model purchase(event='1')=prior_engagement_z discount_10 mobile / link=logit;
run;
proc logistic data=d(where=(split='train'));
 model purchase(event='1')=prior_engagement_z discount_10 mobile / link=probit;
run;
/* Score the same holdout and compute Brier, log-loss, AUC and calibration separately. */

Python is the numerically verified reference for the published table. R mirrors the same models and metrics. SPSS and SAS are secondary fit snippets; their holdout scoring and reporting must be completed and checked in the declared local installation before use.

Expected final deliverable

01
Question, population, unit, horizon, binary outcome and dated predictors.
02
Train-holdout split, preprocessing and identical specifications.
03
Coefficients on their own scale, probabilities, marginal effects with CI, convergence and separation.
04
Brier, log loss, AUC, calibration, tail profiles and decision thresholds on the same holdout.
05
Supported conclusion, forbidden conclusion, human choice and reassessment rule.

Scientific sources and evidence level

Amemiya (1975), Qualitative Response ModelsFoundational article, full text verified
Defines binary qualitative-response models and the normal and logistic forms. It also states that estimates are often close up to a multiplicative factor. It does not establish universal model equivalence.
Gündüz & Fokoué (2015)Theoretical and computational comparison, full text verified
Documents central similarity, tail differences and predictive comparisons between binary links. Its results depend on its definitions and studied settings, so this page keeps the claim bounded.
Dimitriadis et al. (2023)Methodological article on probabilistic evaluation, full text verified
Separates calibration, discrimination and overall predictive performance and defines Brier and logarithmic scores. It does not decide which link is substantively appropriate for a marketing population.
Nelder & Wedderburn (1972)Generalized linear model foundation, full text verified
Establishes iterative weighted regression for maximum-likelihood estimation in generalized linear models, including binomial response models.
Hill, Griffiths & Lim (2018), chapter 16Econometrics reference chapter, full text verified
Defines continuous-variable marginal effects, their sample average and delta-method uncertainty. The displayed interval remains conditional on this page’s declared design.

Dataset · Tool

Dataset · MSC-P-026-LOGIT-PROBITSynthetic Logit and Probit comparison→

Method connections

Parent territoryCustomer and choice science: behavior, value and heterogeneity