New research from BerkeleyHaas looked at whether giving people incentives to write online reviews can help sellers. This research has given us a new perspective on how reviews affect sales and profits.

The researchers teamed up with Airbnb to carry out a large and costly experiment. They wanted to see if offering incentives for reviews had any impact on day-to-day marketing. What they discovered was quite surprising. While offering rewards did increase the number of reviews, these incentivized reviews tended to be more negative in nature. Oddly enough, these negative reviews didn’t affect how many nights were booked or the total revenue.

Incentivizing feedback

In the Airbnb experiment, participants were split into two groups: the treatment group and the control group. The treatment group received an email that offered them a $25 Airbnb coupon in exchange for a review if they hadn’t already posted one within the typical eight or nine days after their stay. The control group got a simple reminder email to leave a review without any financial incentive.

“We found that reviews induced by incentives were more likely to contain information about lower quality experiences,” the researchers explain. “We also found that when compared to non-incentivized reviews with the same star ratings, incentivized reviews corresponded to worse guest experiences. In other words, a seller may have received the same star rating from both study groups, but the incentivized guest was more likely to post a negative review, while upholding an inflated star rating.”

The program that encouraged reviews with rewards did change the number and kinds of reviews guests wrote. However, it didn’t have a clear positive impact on the platform and its sellers. In fact, it often had a negative effect. The number of nights booked for the properties didn’t change because of incentivized reviews. But the kind of bookings did change. There were more bookings, but they tended to be shorter, which means they were not as good of a fit.

As a result, the study showed that offering rewards for reviews didn’t have a clear positive or negative impact on revenue. In fact, incentivized reviews didn’t make transactions better and sometimes even made them worse. However, this approach didn’t lead to more complaints or fewer customers coming back for more business.

“Ultimately, we found that while reviews do play a meaningful role in the overall marketing efforts for online sellers, incentivizing those reviews may not deliver the results many would assume,” the authors conclude.