marketing-science-center.com

Customer reviews and AI: reading a positioning map without overreading it

A study published in PLOS One analyzes how a language model codes 5,895 customer reviews of headphones, with a human validation on 400 reviews and a comparison against simple rules; this document details its scope and limits before any commercial application.

The essentials

A study published in the journal PLOS One on November 11, 2026 analyzes how a large language model codes online customer reviews to compare four headphone brands across several perception dimensions rather than a single overall score. The interest of this paper is methodological: it concerns how to measure, not a current market diagnosis. [C3]

The analyzed sample comprises 5,895 Amazon reviews, with a targeted cap of 1,500 reviews per brand: Sony, Bose and Beats reach this cap, while JBL counts 1,395. For reviews with an available date, the period runs from January 13, 2003 to September 28, 2018. No measurement of the current market for the four brands is produced by this study. [C1] [C2]

The comparison with the human benchmark covers 400 reviews, coded by two independent people blind to the model's outputs, with disagreements then adjudicated. Table 5 (p. 9) reports a weighted kappa of 0.68 for perceived quality and a kappa of 0.57 for emotional connection. These coefficients measure coding agreement with the human benchmark; they are neither accuracy percentages nor measures of effects on sales or return on investment. [C5] [C6]

The authors' results

For each review, the model receives the text, brand name, product title and star rating (p. 7). Table 4 (p. 8) lists seven extracted dimensions: brand image, perceived quality, perceived value, emotional connection, recommendation intent, primary attribute focus and dissatisfaction source. [C4] [C11]

Because the star rating is among the inputs given to the model, the strong convergence observed between its quality coding and that rating does not amount to an independent validation. The authors themselves claim no added value from the model for overall quality, as this dimension remains largely recoverable from the star rating alone. [C4] [C7]

The authors also compare two simple coding rules, the star rating and a polarity score, against the coding produced by the model, across the full corpus of 5,895 reviews. This comparison takes the model's own coding as the reference. It therefore does not demonstrate the model's superiority against the same independent human benchmark used for the 400-review validation. [C10]

Critical reading and limits

Similar review counts per brand facilitate comparison but do not ensure that positive, neutral and negative opinions occur in their population proportions. Brand-level results remain conditional on the analytic sample; different sampling or weighting could change the aggregate estimates (pp. 14–15). [C13]

A separate limitation concerns differences in product models, price ranges and review periods, which may influence observed patterns despite filtering. The study covers one category and one platform; the measures depend on the language model, prompt and coding scheme (p. 15). Our reading: this paper establishes neither causal effects nor transfer to other markets or the present period. [C8]

The authors describe emotional connection as an affective signal drawn from reviews, not a fully validated measure of brand attachment (p. 14). Our reading: retain this distinction and examine coding errors before making strategic interpretations. [C6] [C9]

Proposed application

Before considering a comparable use for a brand diagnosis, we propose, without this approach having been tested by the authors, to start from a precise question, build a recent corpus comparable across the studied brands, have a subsample coded by two independent people blind to the model's outputs, then compare a simple rule and the model against the same human benchmark before any commercial decision. This sequence is our proposal, not a recommendation proven by the study itself. [C5] [C10]

Sources and verification scope

Study reviewed: Li J, Cao H, "Mapping headphone brand positioning from e-commerce reviews with large language models", PLOS One 21(11):e0361055, published November 11, 2026, DOI https://doi.org/10.1371/journal.pone.0361055. This is a published open-access research article under a CC-BY 4.0 licence; the main text, 17 pages long, was verified. [C1]

Page references used in this reading: corpus and sample page 7; human validation page 8 and table 5 page 9; comparison with simple baselines page 10; emotional connection page 14; limitations page 15. [C1] [C5] [C6] [C7] [C9] [C8]

The authors announce an S1 appendix covering the prompt and reproducibility settings (p. 15), and derived data and analytical scripts in the availability statement (pp. 1–2), with a Zenodo repository: https://doi.org/10.5281/zenodo.23320929. For this reading, we acquired neither that appendix nor the repository files; they were neither reviewed nor executed. No replication was conducted and our verification does not confirm the model's exact parameters. [C12] [C14] [C15]