Research & Evidence

Search for a method

Search titles, questions, territories and MSC identifiers.

    ← All methods
    MSC-H-006Evidence foundationsScientific territory

    Statistical decision methods: choose, quantify, validate

    This territory links variable type, sampling design, estimand and error cost. The test or model comes after the question, never before it.

    Scientific editorial team: Marketing Science Center

    Direct answer

    Choose a method consistent with the question and make decision uncertainty visible.

    This territory links variable type, sampling design, estimand and error cost. The test or model comes after the question, never before it.

    Model or deliverableQuestion → design → estimand → uncertainty → decision boundary

    Wasserstein, Schirm & Lazar, 2019Fay & Proschan, 2010

    ESTIMAND

    Estimation profile

    Unit of analysis
    Decision × population × horizon
    Exact estimand
    A defensible chain from question to evidence
    Model or deliverable
    Question → design → estimand → uncertainty → decision boundary

    01—09

    Territory pathway

    01 · MSC-P-001How do you turn a marketing claim into a testable question?→02 · MSC-P-003How should uncertainty in a marketing result be expressed?→03 · MSC-P-004Statistical significance or effect size: which result should be interpreted?→04 · MSC-P-012How many observations does an experiment need?→05 · MSC-P-033How do you validate a marketing forecast?→06 · MSC-P-034How do you build a Monte Carlo simulation for a marketing decision?→07 · MSC-P-039Which statistical test should you choose?→08 · MSC-P-048Can we stop an A/B test as soon as it becomes significant?→09 · MSC-P-040What should be done when dozens of variants or segments are tested at once?→

    01—08

    Verifiable analysis framework

    1. 01
      Question

      Statistical decision methods: choose, quantify, validate

    2. 02
      Estimand

      A defensible chain from question to evidence · Decision × population × horizon

    3. 03
      Data

      population · unit_of_analysis · decision_horizon · outcome · assumptions

    4. 04
      Model or deliverable

      Question → design → estimand → uncertainty → decision boundary

    5. 05
      Declared calculation

      Question → design → estimand → uncertainty → decision boundary → A defensible chain from question to evidence

    6. 06
      Uncertainty checks

      Unique decision intent · Explicit identification assumptions · Reproducible evidence trail

    7. 07
      Method-specific validation

      Check data, estimate stability and interpretation limits. Unique decision intent · Explicit identification assumptions · Reproducible evidence trail

    8. 08
      Limitations

      Guarantee a correct decision from statistical significance alone.

    Required variables

    • population
    • unit_of_analysis
    • decision_horizon
    • outcome
    • assumptions

    Checks

    • Unique decision intent
    • Explicit identification assumptions
    • Reproducible evidence trail
    Possible decision

    Choose a method consistent with the question and make decision uncertainty visible.

    Unsupported decision

    Guarantee a correct decision from statistical significance alone.

    Scientific sources

    2 sources

    1. Wasserstein, Schirm & Lazar, 2019
    2. Fay & Proschan, 2010

    Read next

    MSC-P-001How do you turn a marketing claim into a testable question?→MSC-P-003How should uncertainty in a marketing result be expressed?→MSC-P-004Statistical significance or effect size: which result should be interpreted?→MSC-P-012How many observations does an experiment need?→MSC-P-033How do you validate a marketing forecast?→MSC-P-034How do you build a Monte Carlo simulation for a marketing decision?→