“Customers who viewed this item also viewed” appears on product pages all across the internet. For e-retailers, this represents an opportunity to get more and different products before customers’ eyes. “These recommender systems are put in place to help niche items that usually have a smaller market share,” said Dokyun Lee, assistant professor of business analytics, Xerox Junior Faculty Chair AY 2017-2018. “The systems highlight these kinds of niche items so that they can be more easily found by users who would not normally search for them or see them in brick-and-mortar stores, and so the intent is to increase sales of those items.”
However, research that Lee coauthored with Kartik Hosanagar at the University of Pennsylvania’s Wharton School demonstrates that while implementing recommender systems does increase overall sales figures, it does not generally improve the relative sales for niche items.
In the paper, titled “How Do Recommender Systems Affect Sales Diversity? A Cross-Category Investigation Via Randomized Field Experiment,” Lee shows that the typical software used in recommender systems — collaborative filters — generally prioritizes considerably popular items. What this means is that individual diversity increases as consumers purchase more items they would not otherwise have seen, but that aggregate diversity decreases as the filters reinforce the success of already popular products.
For example, a customer on a product page for a blockbuster comedy film would be likely to have other major films recommended to them, even in other genres. That customer’s purchase diversity would increase when buying an action film based on such a filter, but less popular films — such as an indie comedy the consumer might also have considered — would not benefit.
“Traditional recommender systems carry the unintended consequence of increasing concentration bias,” Lee said. Instead, he notes that content-based recommender systems, in which items are tagged to prioritize similarity over popularity, actually can improve aggregate diversity.