Amazon.com reviews are rigged

Amazon.com user reviews are a critical factor in increasing the sales for the giant retailer. The number of times, I have heard my wife say that she selected a particular product because of great user reviews, is far too high to count. These reviews can have significant credibility issues, though…In their great scholarly paper – Six degrees of reputation: The use and abuse of online review and recommendation systems – Shay David and Trevor Pinch point to some of the follies of user-generated reviews at Amazon.com:

Evidently, in many areas of cultural production user reviews are mushrooming as an alternative to traditional expert reviews. If there was any doubt, it has long ago been established that reviews and recommender systems play a determining role in consumer purchasing (an early review is available in Resnick and Varian, 1997) and recent qualitative research adds weight to the claim that these review systems have causal and positive effects on sales; to nobody’s surprise, books with more and better reviews are shown to sell better (Chevalier and Mayzlin, 2004). With people in the culture industries increasingly realizing this truism, many of the reviews are thus positively biased and it becomes very hard to distinguish the ‘objective’ quality of the reviews. In addition, due to the large variance in the quality of the reviews, and the varied agendas of the reviewers, user input too often becomes untrustworthy leaving the consumers with little ability to gauge an item’s actual quality. Do we live in a cultural Lake Wobegon where “all the books are above average?” (to paraphrase Keillor, 1985) Is there a way to review the reviewers, to guard the guards? As will be discussed in details below, emerging systems like the one employed on sites like Amazon.com (2005) suggest that there are ways to try to solve this bias problem by offering a tiered reputation management system which offers a set of checks and balances. But these new options also bring with them new problems as the participants adjust to what is at stake in this new economy of reputation.

They offer examples with links to amazon.com reviews:

This instance concerned one of Pinch’s own books Analog Days: The Invention and Impact of the Moog Synthesizer (Pinch and Trocco, 2002). This book that chronicles the invention and early days of the electronic music synthesizer was well received by reviewers both offline and online, and the Amazon.com editors quote a review from the Library Journal that reads as follows:

… In this well–researched, entertaining, and immensely readable book, Pinch (science & technology, Cornell Univ.) and Trocco (Lesley Univ., U.K. [sic]) chronicle the synthesizer’s early, heady years, from the mid–1960s through the mid–1970s … . Throughout, their prose is engagingly anecdotal and accessible, and readers are never asked to wade through dense, technological jargon. Yet there are enough details to enlighten those trying to understand this multidisciplinary field of music, acoustics, physics, and electronics. Highly recommended. [link]

original review

A similar (but distinctly different) book that had appeared earlier — Electronic Music Pioneers by Ben Kettlewell (Vallejo, Calif.: ProMusic Press, 2002) — received the following user review on Amazon.com on 15 April 2003:

This book is a must. Highly recommended., April 15, 2003 / Alex Tremain (Hollywood, CA USA)

… In this well–researched, entertaining, and immensely readable book, Kettlewell chronicles the synthesizer’s early, years, from the turn of the 20th century — through the mid–1990s … . Throughout, his prose is engagingly anecdotal and accessible, and readers are never asked to wade through dense, technological jargon. Yet there are enough details to enlighten those trying to understand this multidisciplinary field of music, acoustics, physics, and electronics. Highly recommended. [link]

Copied review

The ‘similarity’, of course, is striking. The second review is simply a verbatim copy of the first one, replacing only the name of the authors and the period the book covers.

In another case a user reviewed several Tom Hanks/Meg Ryan movies. The user posted the same review for the movies Sleepless in Seattle and You’ve Got Mail. He found that each of those films was “a film about human relations, hope and second chances, but most importantly about trust, love, and inner strength.” [link link]

copied review

rev4

As we know, especially with the demands for producing one blockbuster after another, Hollywood movies are sometimes strikingly similar, and yet posting the same review for two different films suggests that the reviewer is interested less in accurate representation of the movie’s content or qualities and more in the sort of reputation and identity that he or she can build as someone who posts numerous reviews.

The authors point perverse incentives for different actors to game the review mechanisms:

  • Self-plagiarize in order to write reviews quickly and to build up reviewer reputation (see examples with links above)
  • Write unusually positive reviews to butter the publisher and the author in hope of landing a job as a full-time reviewer
  • People with vested financial interest in the success of a product, take advantage of the cloak of anonymity and try to game the system by having family members etc. create positive reviews for their product and negative reviews for competing products
  • Write reviews to see your name associated with a popular product. This can work as an ego boost for adolescents or even some adults
  • Write reviews to promote other web sites or substitute products (example)

The authors find the problems mentioned above, based on their very limited analysis (due to the limitations of Amazon.com APIs), to effect about 1% of the reviews. I suspect, though, that a thorough analysis will reveal a significantly higher level of problem reviews.

So what can be done to deal with these issues? I believe that any system – both online and offline – has limitations that can be exploited by bad actors for personal benefit. Anybody remember the Armstrong Williams fiasco as an example of problems with offline systems? Still, as long as these systems are transparent and user expectations of how the systems work are properly managed, such systems can be valuable. Some of the specific things Amazon.com can do are:

  • Provide more statistics about various issues with its user generated product reviews so as to properly set user expectations. They should point to all the various kinds of problems so that users take all the reviews on their site, with a pinch of salt. This, of course will be hard because doing this might reduce their revenue and potentially their influence, but in the long term pay off in terms of higher customer satisfaction.
  • Be more receptive to user complaints and create a mechanism to penalize the people who try and game the system. At present they seem to be taking a completely hands off approach to policing the user reviews and as a result customers end up paying a price.
  • Build up a more sophisticated notion of reputation which is based on reputation of users in other communities. Such a notion should include more elements then just the number of reviews a person has entered
  • Build up a more sophisticated meta-moderation system like the one built by Slashdot.

This is not a simple problem but one Amazon.com should tackle to maintain long-term trust relationship with their users.

Advertisements

63 thoughts on “Amazon.com reviews are rigged

  1. Simply wish to say your article is as amazing. The clarity in your post is just excellent and i could assume
    you are an expert on this subject. Fine with your permission allow me to
    grab your feed to keep updated with forthcoming
    post. Thanks a million and please keep up the
    enjoyable work.

  2. Hey! Do you know if they make any plugins to help with SEO?
    I’m trying to get my blog to rank for some targeted keywords but I’m not
    seeing very good gains. If you know of any please share.
    Many thanks!

  3. Hi there this is kinda of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding know-how so I wanted to get guidance from someone with experience. Any help would be greatly appreciated!

  4. Please let me know if you’re looking for a writer for your weblog. You have some really great posts and I feel I would be a good asset. If you ever want to take some of the load off, I’d absolutely love to write some material for your blog in exchange for a link
    back to mine. Please blast me an e-mail if interested. Thanks!

  5. Hi all, here every person is sharing these know-how, thus it’s fastidious to read this weblog, and I used to go to see this blog every day.

  6. Katy Perry concert tickets going on sale today can be availed from various sources.
    With a career spanning 30 years, since Cesar Chavez asked me to start shooting concert
    tickets going on sale today and am a huge supporter of the environment.
    Once the seats are legitimate seats.

  7. And instead of feeling obligated to hang out
    with your family on Thanksgiving, you can just throw a Fourth of July
    barbecue with your friends. Obviously no one can tell you what nail polish to wear but if you are keen on looking
    hot this summer try to take the nail polish trends for summer 2012
    into consideration. Look for Essie at your favorite nail spa or salon or you can purchase
    the shades on-line direct from Essie.

  8. And instead of feeling obligated to hang out with your
    family on Thanksgiving, you can just throw a Fourth of July
    barbecue with your friends. Click here to see the entire collection from Kim, Khloe, Kris, Kylie and Kendall.
    The intuitive feeling I had about the rainbow trend must have been right.

  9. One other thing to consider can be that broker agents let investors to purchase legal papers which can be really based on underlying assets regarding each expense car or truck.
    Sometimes it is in reality preferable to business during the night time because of the toughness of selected markets.
    While different websites may well attempt to wow
    you with higher affiliate payouts as well as small restricts,
    just ioption will give you the choice to seriously make
    money from ioption.

  10. Be aware that HPV has no cure, but the question that arises here is these tips are good for the body.
    This amazing 5 night cruise departs from New York City and travels to the
    socialite while she was dating someone else.

    The level of acidity that is uncomfortable like the onset
    of the Teulada Moraira coast. Be sure to have assistance going
    around for whatever reason.

  11. Thanks for a marvelous posting! I certainly enjoyed reading it, you happen to be a great author.
    I will always bookmark your blog and will come back in the foreseeable future.
    I want to encourage yourself to continue your great job, have a nice weekend!

  12. If your minute gamble seems to lose, double ones stake yet again,
    and also proceed doing this till your coloring really does surface, when position your current online win through the string involving bets stands out as the very same as if you
    earned the starting up guess. For decades it was explained that oddsmakers are
    definitely more susceptible sooner from the 12 months as the groups usually are completely new, however When i will not
    invest in the idea, claimed Roxborough, now outdated.

  13. Wow that was odd. I just wrote an incredibly long comment but after I clicked submit my comment didn’t show up. Grrrr… well I’m not
    writing all that over again. Anyway, just wanted to say wonderful blog!

  14. These ‘server logs’ typically include your web request, Internet Protocol
    address, browser type, browser language, the date and time of your request
    and one or more cookies that may uniquely identify your browser.
    But it has a problem of duration of battery service.
    It helps you to keep a constant watch and also record the footage of the entire area.

  15. This is covered by a vast ocean, which most scientists believe is made
    of water. It’s free, and it contains 24 of my Answers, to the most top FAQs on, DIY solar panels, and knowing this information, will certainly help people, like you, who are planning, to go for, Home made solar energy. This photovoltaic effect would be developed and improved upon to lead to the photovoltaic solar panels in use today.

  16. Installation of CCTV camera is considered an integral aspect of any provision for security in any
    area whatsoever. The consolidation process among China’s radio and TV networks announced on 24 August 2010 envisages financing of more than 10 billion dollars, with investments coming from the government as well as broadcasters and TV companies. Simon Cowell stepped up his security detail, at an undisclosed price, after finding a tracking device under his car.

  17. Hello would you mind letting me know which
    web host you’re working with? I’ve loaded your blog in 3 different internet browsers and I
    must say this blog loads a lot quicker then most.
    Can you suggest a good hosting provider at a
    reasonable price? Thanks a lot, I appreciate it!

  18. All you have to stick to to stay away from rank in google with rocket video 3 review scams and other online review sites
    are having a greater effect than ever on the perception of shoppers.
    Be sure to get a quick overview of rank in google with rocket video 3
    review techniques, but the ability to ride the Sky Splash, a six-stories-tall inner tube ride that seats six people.
    Use a lot of people actually prefer to pay through online systems such as Paypal.

  19. Today, I went to the beachfront with my children. I found a sea shell and gave it to my 4 year
    old daughter and said “You can hear the ocean if you put this to your ear.” She placed
    the shell to her ear and screamed. There was a hermit crab inside
    and it pinched her ear. She never wants to go back! LoL I know this is entirely off
    topic but I had to tell someone!

  20. At family reunions, an old photo of great-grandma and great-grandpa printed
    on a pin and marked with the reunion year makes a special gift
    for everyone. From time to time we all find ourselves buying something we hadn’t intended to, or coming home with something we really didn’t want.
    Each color is individually fired at high temperatures to ensure no color bleeding and the longest durability.

  21. Usually the fear of not writing well enough, of receiving negative feedback, or fear of rejection
    causes people to dismiss this avenue for marketing.
    Tired of sounding like everyone else and want something new and
    fresh. Before you pass out due to the thought of cold calling
    people and sending out brochures and letters to strangers, take a
    deep breath.

  22. If you take these steps, your odds of being scammed will decrease substantially.
    It is a free multiplayer word game for android devices, iphones and ipods.
    Apply your culinary creativity and cook different dishes making use of only one primary ingredient.

  23. Both need to exit portfolio and the value realized should be
    used to include better stocks. Typically Indians residing other countries make real estate
    investment in India in order to get huge returns on their money but nowadays
    the numbers of domestic investors has also
    risen. raw land developments still provide a foundational investment method
    for many of the wealthiest investors in the world.

  24. I still remember one of the headlines in a newspaper where they were advertising the arrival of the cell-phone: You’ll be
    able to phone from your shopping trolley in the grocery store.

    technology of Greenward Technologies is said to be based on the principle that a pressurized cylinder
    can endure much higher bending loads than its unpressurized counterpart.
    No physical exercise would mean, no to any exertion, which further means ever
    increasing fatty deposits and high rate of
    obesity.

  25. Thanks for one’s marvelous posting! I truly enjoyed reading it,
    you may be a great author. I will make certain to bookmark your blog and will eventually come back at some
    point. I want to encourage you continue your great job, have
    a nice day!

  26. Excellent weblog right here! Also your site quite a bit up very fast!
    What host are you the use of? Can I am getting your affiliate hyperlink for your host?
    I want my website loaded up as quickly as yours lol

  27. As of yet they have been doing. This vapor is mainly comprised of a liquid best
    e cigarette solution that create the vapor. A battery starts working as soon as
    the person is not getting the same taste or not.
    Before yyou purchase one, make sure you have one charged if the other battery gets tooo low, plus they can do anything.
    So it’s always bwst to use the same amount with
    e cigarettes, then it should be good vapor production and excellent throat hit.

  28. Thanks for the marvelous posting! I quite enjoyed
    reading it, you are a great author.I will always bookmark
    your blog and may come back down the road. I want to encourage you to definitely continue your great writing, have a nice day!

  29. This design is steller! You obviously know how to
    keep a reader amused. Between your wit and your videos, I was almost moved
    to start my own blog (well, almost…HaHa!) Wonderful job.
    I really enjoyed what you had to say, and more than that, how you
    presented it. Too cool!

  30. continuously i used to read smaller articles or reviews which also clear their motive, and that is also happening with
    this article which I am reading here.

  31. You are so interesting! I don’t suppose I’ve read through
    anything like this before. So wonderful to find someone with some
    unique thoughts on this subject matter. Seriously.. thanks for
    starting this up. This site is one thing that’s needed on the web,
    someone with a bit of originality!

  32. Good day! This is my 1st comment here so I just wanted to give a quick
    shout out and tell you I genuinely enjoy reading
    through your blog posts. Can you recommend any other blogs/websites/forums that cover the same
    topics? Thanks a ton!

  33. Hey I know this is off topic but I was wondering if you knew of any widgets I could add to
    my blog that automatically tweet my newest twitter updates.
    I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would
    have some experience with something like this. Please let me know if
    you run into anything. I truly enjoy reading your blog and I look forward to
    your new updates.

  34. Great site you’ve got here.. It’s hard to find
    high-quality writing like yours nowadays. I seriously appreciate individuals
    like you! Take care!!

  35. Affiliate Marketing Domain Name Gdi Free Affiliate Program What else could we possibly need.
    As with any online marketing activity, keep the message short, sweet and
    direct and be sure to include the necessary contact information needed to send in a referral.
    Learn how to market your home based sewing business online and in your local community.

  36. Hey outstanding website! Does running a blog similar to this take
    a great deal of work? I’ve no expertise in coding however I had been hoping to start my own blog
    soon. Anyways, if you have any recommendations or tips for new blog
    owners please share. I know this is off subject nevertheless
    I just wanted to ask. Kudos!

  37. I was recommended this website via my cousin. I’m no longer positive whether this publish is written by means of him as nobody else recognise such targeted about my difficulty.
    You are incredible! Thank you!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s