FRENITESDE

Research & Evidence

Search for a method

Search titles, questions, territories and MSC identifiers.

40 results
  1. MSC-P-001How do you turn a marketing claim into a testable question?Decision Science↗
  2. MSC-P-002Correlation or causality: what can an analysis actually support?Marketing Measurement↗
  3. MSC-P-003How should uncertainty in a marketing result be expressed?Decision Science↗
  4. MSC-P-004Statistical significance or effect size: which result should be interpreted?Decision Science↗
  5. MSC-P-005How do you measure a marketing construct that is not directly observable?Market Research↗
  6. MSC-P-006How do you design and validate a measurement scale?Market Research↗
  7. MSC-P-007Alpha or omega: how should scale reliability be assessed?Market Research↗
  8. MSC-P-009PCA, EFA or CFA: which method should you choose?Market Research↗
  9. MSC-P-010When should you run a marketing experiment?Marketing Measurement↗
  10. MSC-P-011How do you design an A/B test that actually estimates an effect?Marketing Measurement↗
  11. MSC-P-012How many observations does an experiment need?Decision Science↗
  12. MSC-P-013How do you measure campaign incrementality with a control group?Marketing Measurement↗
  13. MSC-P-017How do you detect selection, contamination and attrition in an experiment?Marketing Measurement↗
  14. MSC-P-018Predictive or causal regression: what are you trying to estimate?Marketing Models↗
  15. MSC-P-019How do you diagnose a marketing regression before interpreting it?Marketing Models↗
  16. MSC-P-022How do you estimate price elasticity and its uncertainty?Pricing Science↗
  17. MSC-P-026Logit vs Probit: how do you choose for purchase probability?Customer Science↗
  18. MSC-P-029Which customers have the highest probability of churn?Customer Science↗
  19. MSC-P-027TAM, UTAUT or UTAUT2: which framework should be used to study technology acceptance?Market Research↗
  20. MSC-H-001Measurement and causality: how can a marketing effect be established?Marketing Measurement↗
  21. MSC-H-002Marketing response models: shape, delay and saturationMarketing Models↗
  22. MSC-H-003Pricing science: connecting price, demand and contributionPricing Science↗
  23. MSC-H-004Customer and choice science: behavior, value and heterogeneityCustomer Science↗
  24. MSC-H-005Measurement science: building valid indicatorsMarket Research↗
  25. MSC-H-006Statistical decision methods: choose, quantify, validateDecision Science↗
  26. MSC-P-008How do you validate a marketing measurement scale?Market Research↗
  27. MSC-P-014How do you design a marketing geo experiment?Marketing Measurement↗
  28. MSC-P-015How do you estimate an effect with difference-in-differences?Marketing Measurement↗
  29. MSC-P-020How do you address price endogeneity?Pricing Science↗
  30. MSC-P-021Fixed or random effects: which panel model should you choose?Marketing Models↗
  31. MSC-P-023How do you estimate a demand function?Pricing Science↗
  32. MSC-P-024How do you simulate a price-volume-margin scenario?Pricing Science↗
  33. MSC-P-028How do you estimate CLV with BG/NBD and Gamma-Gamma?Customer Science↗
  34. MSC-P-030How do you analyze retention with a survival model?Customer Science↗
  35. MSC-P-031How do you build a useful customer segmentation?Customer Science↗
  36. MSC-P-032How do you test segmentation stability?Customer Science↗
  37. MSC-P-033How do you validate a marketing forecast?Decision Science↗
  38. MSC-P-034How do you build a Monte Carlo simulation for a marketing decision?Decision Science↗
  39. MSC-P-035How do you model saturation and adstock?Marketing Models↗
  40. MSC-P-039Which statistical test should you choose?Decision Science↗
← All methods
METHOD DOSSIERMSC-P-004Evidence foundations

Statistical significance or effect size: which result should be interpreted?

A p-value measures compatibility with a null model. Effect size describes observed magnitude and should be paired with an interval and a practical relevance threshold.

Direct answer

Assess detection, magnitude and precision together.

A p-value measures compatibility with a null model. Effect size describes observed magnitude and should be paired with an interval and a practical relevance threshold.

Model or deliverablestandardized effect = difference / pooled SD

Wasserstein, Schirm & Lazar, 2019

ESTIMAND

Estimation profile

Unit of analysis
Declared unit or comparison × horizonControlled scientific terminologycomparison × outcome
Exact estimand
Effect size + interval + practical threshold
Model or deliverable
standardized effect = difference / pooled SD

01—08

Verifiable analysis framework

  1. 01
    Question

    Statistical significance or effect size: which result should be interpreted?

  2. 02
    Estimand

    Effect size + interval + practical threshold · comparison × outcome

  3. 03
    Data

    group · outcome · sample_size · variance

  4. 04
    Model or deliverable

    standardized effect = difference / pooled SD

  5. 05
    Declared calculation

    standardized effect = difference / pooled SD → Effect size + interval + practical threshold

  6. 06
    Uncertainty checks

    Predefined threshold · Multiplicity · Interval width

  7. 07
    Method-specific validation

    Check data, estimate stability and interpretation limits. Predefined threshold · Multiplicity · Interval width

  8. 08
    Limitations

    Conclude that an effect is important because p < 0.05.

Required variables

  • group
  • outcome
  • sample_size
  • variance

Checks · Controlled scientific terminology

  • Predefined threshold
  • Multiplicity
  • Interval width
Possible decision

Assess detection, magnitude and precision together.

Not supported

Conclude that an effect is important because p < 0.05.

Scientific sources

1 source

  1. Wasserstein, Schirm & Lazar, 2019

Method connections

Parent territoryStatistical decision methods: choose, quantify, validate

Read next

MSC-P-003How should uncertainty in a marketing result be expressed?→MSC-P-012How many observations does an experiment need?→MSC-P-013How do you measure campaign incrementality with a control group?→