Are user tests trustworthy?
|Dpreview recently reffered to scientific research. That should be interesting! It's about the reliability of on-line consumer reviews (camera reviews, lens reviews, printer reviews). In the article:Deceptive Reviews: The Influential Tail"", which you can download as a pdf, Eric Anderson (Northwestern University) and Duncan Simester (MIT) describe their research. The title of the article refers to the misleading effect of a limited number of negative user reviews.|
They come to a few surprising conclusions. First of all, they show that negative reviews have much more influence than positive reviews. Also, they found that 5% of the reviews written by "users" who had not bought the product were significantly more negative than the other reviews. They make a plausible case that these negative reviews are written by users who love a brand, rather than by a competitive brand's fanboys. Also, they exclude a number of other possible explanations for a negative review. ("We are able to rule out several alternative explanations for the low rating effect. The effect cannot be attributed to differences, item reviewer differences, gift recipients, customers transactions, or items, misidentifying it unobserved differences in the timing of the reviews. ")
Since you don't know the writer of a consumer review of a camera or lens, it is difficult to evaluate the commentary. But if you want to go off a camera review or lens review by users, then pay the most attention to the negative reviews. Keep in mind that the score that such reviewers give are overly low, but pay close attention to the substantive arguments. They may include valuable points.