Amazon’s Fake Review Problem

One reason we buy from Amazon: plenty of reviews. But what if many of Amazon’s top-reviewed items have fake, paid reviews?

I was looking for a sunrise alarm clock this morning and started searching through the many reviews, filtering by ones that mentioned “minutes,” since I wanted to learn about the product’s timer feature. This surfaced a bunch of similar-looking reviews:

Here, we see both the top and bottom review with the sentence,

The light can be pretty bright, you can adjust it where it’ll be dim and slowly brighten 30 minutes before the alarm time.

Did “Becky” and “Dione Milton” really both happen to write a review with the exact same 23-word sentence? Or, is it more likely that they are agents sourcing reviews from a script, and they sloppily pasted their reviews without rewriting them (as they were presumably instructed to do)? Note also the post dates: December 12, 2017. “Becky” and “Dione Milton” both had private profiles, where their 5-6 reviews were hidden – very similar looking.

Amazon – who has some of the world’s most advanced ML – really needs to step up its review fraud detection game. Imagine how great the Amazon shopping experience would be if we could trust its reviews.

Third party meta review sites like Fakespot will identify problems for us (in this case, the product got an “F” grade) – so why doesn’t Amazon?

Amazon: you can do better.

30 thoughts on “Amazon’s Fake Review Problem”

  1. The reviews read “verified purchase”. You think they actually order a few dozen items just so they can write a verified purchase review?

    1. Absolutely – it’s economical, given the importance and dominance of the Amazon marketplace. What’s a better alternative explanation for these reviews with duplicated sentences?

      1. Not to give it all away but…here’s a hint.

        Amazon now faces the same problem that Google faced several years ago thanks to blackhat SEO. Pagerank was vulnerable in the same way that Amazon reviews are now vulnerable.

        Google found the solution, and the solution is the same for Amazon’s issue. It could be implemented by a small team in a long weekend using only the data they already have.

        1. I can see how the blackhat analogy holds, but am skeptical that a small team could solve this over a long weekend. Maybe Amazon needs to hire you? 🙂

    2. If I want to review my own product, I will buy it to show ordered, and then resell on same Amazon. All i have to do is spend some money on shipping at most? Verify purchased doesn’t mean that guy even open the box.

  2. I agree and Amazon should take action on this. I’m pretty sure they already notice this problem but what about sales? Maybe in their mindset they prefer to allow some fake reviews in order to maintain current sale volume. Just a thought.

    1. Perhaps there are short term (e.g. quarterly) sales targets within Amazon that lead to some perverse incentives (bad for the long-term)

  3. “Amazon calculates a product’s star ratings using a machine learned model instead of a raw data average. The machine learned model takes into account factors including: the age of a review, helpfulness votes by customers and whether the reviews are from verified purchases.”

  4. Brian, you’ve uncovered and identified a gigantic problem. Unfortunately, it’s a known problem. And, it’s also a fixable one.

    If my business won the contract to fix that wide set of problems, I’d propose approximately a year to fix it as long as we had implementation authority. Working together, we might have it fixed in 9 months.

    It doesn’t make any sense that all of the various loopholes aren’t fixed. There are entirely too many ways to game the system and it’s very unfortunate. I have higher expectations for the best marketplace in the world and I’m sure you do also. They could do so much more to protect sellers, prevent fraud, detect and remove bad actors, resolve selective distribution issues, and on and on.

    Maybe it is a combination between having so many silo’d internal teams and a lack of a clear ROI for resolving these issues? If it took a year and $10M, there’s no question it would be money well spent.

    1. Scott,

      First – hopefully this was temporary – the load time for your site was about 8 seconds for me :O

      Agree that 1 year + $10 million would be justified before the problem becomes a serious reputation problem for Amazon.

      How though would you begin to propose a solution that cannot be gamed?

  5. Try using fakespot.com or reviewmeta.com. They’re both sites setup to try & check the validity of Amazon comments using different methodology – such as how many times a reviewer has left a review, certain trigger keywords known to be used by bots & how many reviews Amazon has deleted from a review by request of the merchant (yes, Amazon will do this!). This will leave you with a much better sense of whether or not a product has been ‘review-bombed’ or if the company is up to shady practices.

      1. Fakespot etc are themselves scams. They are anonymously registered (hidden identity) Adsense sites and you don’t know who they are or what’s under the hood.

        Not sure how you’re going to “test” them — you would have to launch your own products & get reviews you know are real (by interacting with your customers) — that are then all labeled fake — to know these sites are fooling you.

        So, very clever meta-scams, easy to fall for, as you’ve done here. They don’t really work, but as long as they appear to work, it doesn’t matter. People will continue to promote them, and they’ll continue to make a fortune in ad sales.

        1. Not sure if you’re just trolling here. You count the anonymous registration as a strike against them, yet you yourself appear to have provided me with a fake email address so I cannot contact you.

          Admittedly, getting ground truth data (known valid or invalid reviews) is difficult, but we can come up with signals that are clear evidence of fraud. Having a site like Fakespot examine this evidence for us is a much easier first-pass than manually weeding through thousands (or more) data points.

          I contacted someone at Fakespot, who in his thorough reply to me, appears to have good intentions. He has a background in a relevant industry that would make him very resourceful in this space.

  6. Like the post above, I also concentrate on the bad reviews. Helpful on ebay too where sellers hide their location in China and shipping takes weeks.

    1. I too typically focus on negative reviews. At some point, unfortunately, this incentivizes fake negative reviews from competition.

  7. Amazon only does something when the public spot light is on and not enough. Take a look at this Chinese seller and his products. All the reviews 5 star unverified on the same exact random day in November. I reported to Amazon and they have done absolutely nothing. These sellers in China have a boat load of disposable seller accounts if they get caught, shill buyer accounts leaving reviews, direct links to the factories selling
    Junk, and they don’t pay a dime in US state, federal, or sales tax.

    https://www.amazon.com/Vertical-Multi-function-Controller-Playstation-Dualshock-4/product-reviews/B0776XYJ85/ref=cm_cr_getr_d_paging_btm_3?ie=UTF8&reviewerType=all_reviews

  8. If Amazon starts deleting blackhat reviews there will be legitimate reviews mistakenly flagged. Probably very few, but it only takes one story going viral to be a huge PR issue, and if the review is deleted how can anyone tell whether Amazon was right to do so? People are very sensitive about claimed abuses by large corporations, even when there’s no way to verify them.

    Not to mention that any sophisticated technical approach will launch an arms race with the blackhats using Amazon’s catalog as the battleground.

  9. As for the verified purchase, it’s a technique called “brushing”. You make real purchases so you can leave fake ratings. Maybe you even return the items.

    eBay has a similar problem where a new seller needs to get a lot of positive feedback. They start by selling very cheap things (say, $0.99 phone chargers) to pump up the number of transactions and 5 start ratings. After that they have a seller account that is valuable and can transfer it or do something else.

    But this technique isn’t new. You can mess with the New York Times bestseller lists by buying your own book at NYT reporting stores. You make small purchases to fly under the corporate purchase radar.

    You’ll never solve this problem though. It’s an arms race and the other side has much more incentive to game it. Neither does Amazon or eBay have incentives to change it. Purchasers get used to it but keep buying stuff. The people who get annoyed are the makers of rival products and they aren’t going to stop selling either.

    1. I wasn’t aware of the extent of the eBay problem, and never heard it articulated before – yet as a teenager, I intuitively realized this and went so far as to sell some cheap items to build up my account reputation before selling a “large” item so it’d fetch a higher value. Couldn’t eBay simply weight the lower-dollar items less – but it sounds like perhaps they don’t?

      Seems to be a very difficult marketplace for the honest seller today.

  10. Something I have noticed many times, is large numbers of 4 and 5 star reviews for the wrong item. Sometimes the actual item that was reviewed is similar, which makes it harder to notice. Other times, it was quite obvious. Verified purchase seems pointless here, if the review of the purchased item isn’t the item under which the review is posted.

    1. This happens a lot when the seller lists multiple items (that aren’t even related) under the same umbrella listing. You know how you can sometimes select various options for an item? Sometimes, the seller makes those options completely different items. :facepalm:

  11. There are exceptions, (as in the one posted above), but why shouldn’t paid reviews be seen as a form of marketing for some sellers. Truth is, if you’re a smaller seller in an over populated product category, your listing will often not be given a respectful amount of traffic, due to the A9 not acknowledging your listing. One of the few ways around this is to have a 5 star review and a specific amount of verified reviews. Especially when going up against established sellers with hundreds of reviews. I don’t have the answers, but I’d be curious to find out how some of you propose to balance the playing field that Amazon has set up.

Leave a Reply

Your email address will not be published. Required fields are marked *