Wharton School researchers studied the operation of AI shopping systems and found noticeable instability in their choices. According to The Decoder, adding one external source, for example Wirecutter, changed the share of selected products by as much as 99 percentage points.
The result was also affected by changing the order of the same information. In other words, the same data presented in a different sequence could lead the system to a different choice.
This matters for consumers who want to task AI with finding a product or best price: the recommendation may depend not only on product characteristics but also on the structure of input data. However, the available material is only metadata and a summary from The Decoder, so methodological details and the scale of the study remain unclear.
editorial commentary
Why it matters
If the result is confirmed in full study and independent verifications, AI shopping services will need to show robustness of recommendations and sensitivity to sources. The nearest verifiable signal is publication of the methodology and replication of the experiment on other product categories. Substantial uncertainty is associated with the fact that only a metadata retelling of The Decoder is currently available.