What you will learn
Today, we’re talking about review bias.
We decided to cover this after publishing our SEMRush review which ended up causing a bit of a storm on the blog. Truth is, we’re not the biggest fans of the tool and we feel that people use it more out of habit than anything else – especially when compared to a tool like Ahrefs.
As you can imagine, our opinion wasn’t popular with some people – including the CEO of SEMRush – and it resulted in some backlash that prompted us to record this episode.
The Problem With Most Recommendations
In online marketing, people tend to give high ratings for everything.
SEMRush is no exception…
Naturally, SEMRush users were surprised at our interpretation of the tool, resulting in some – frankly ridiculous – conspiracy theories being thrown around. Thing is, SEMRush actually pays twice as much as Ahrefs, so we’d love to be able to recommend them.
And that’s the problem with most “recommendations”.
When you look at other peoples affiliate earnings / income report, they often make a lot more than us when it comes to recommending different tools because they tend to be the highest paying tools.
What’s worse, we’ve noticed a correlation between the affiliate payout and quality of the tool. A good example is Active Campaign, which we’ve been advocating for a while now. While it’s a solid tool, the affiliate commissions are horribly low – but we still recommend them nonetheless.
The Truth About Review Bias
Are our reviews biased?
In short… yes. And it’s not because we have underlying mission to destroy the reputation of any given company, it’s because there isn’t a person on the planet that’s capable of being 100% unbiased.
People come to Authority Hacker to hear our opinions and experiences. They come to see what’s working from us and WHY it’s working. Although we try our best to stay subjective, it’s not realistic to maintain that at all times considering what we’re offering…
…opinions, and experiences.
Why Publish A “Negative” Review?
On the surface, it might seem pointless to publish a negative review.
While you often have a lot less to gain (no direct affiliate commissions), you’ll still have people engage with your site, join your mailing list and buy your products. That traffic can still be valuable and you’ll build credibility by actually giving an honest review, unlike your competitors.
As we already mentioned to at the start of this post, most reviews favor the product or service in order to capitalize on the benefits of doing so. And it’s been happening for years.
Companies send people/bloggers/Youtubers/journalists free stuff all the time, and that alone creates reciprocity bias. We even get free stuff at Authority Hacker, including the SEMRush account that was used in the review we published.
That’s why SEMRush was taken by surprise when we gave it a fairly low rating compared to other reviews out there. Considering that we’re already ranking with that review, it’s very likely to make a dent in their sales. That’s obviously going to upset some people.
The difference is, we’re on the consumers side and that’s an important distinction to make when you’re writing and publishing reviews of your own.
Examples Of Review Bias
A popular example of review bias is Bluehost.
Bluehost is a large hosting company with a very generous affiliate program. They also convert really, really well which naturally attracts a ton of affiliates.
Here’s an example from Pat Flynn over at SmartPassiveIncome.com:
But here’s the thing: their hosting isn’t that great compared to other – similarly priced – hosting companies. You could then assume that anyone who recommends them are doing so for the payout rather than in their audiences best interest. Right?
While that could be said for some people, keep in mind that many of these Bluehost-type recommendations aren’t exactly “bad”, they’re just not the best value for money. It could well be a genuine recommendation but they’re not aware of the better alternatives because that’s what worked for them.
Even we’re guilty of this at times, but we’re actively taking steps to rectify that. A good example is Thrive Content Builder which is getting a lot of flack lately. As a result, we’re are constantly testing other content builders and we’ll be coming out with a review in the near future.
This often leads to the mistake of thinking, “well, if it’s is useful, it must be worth recommending”. But in the same a way a Nokia 3310 is useful – it sends and receives calls and texts – you still wouldn’t swap it for your brand new iPhone.
Bottom line: being better than nothing is not a good enough reason to endorse a product or service.
How To Be Less Biased
There are a number of things you can do to be less biased with your reviews.
First of all, you should always compare like-for-like. Take headphones for example. You could say headphones A are better than headphones B because headphones A are noise-cancelling. But that would still be subjective.
Someone else could value headphones with more bass, while noise-cancellation is something they value very little – and that fact alone could change the outcome of the review. The best way to overcome this problem is to review and compare headphones that both offer the same (or similar) functionality.
Next, you need to make sure you have your facts right. If you’re criticizing anyones product or service online you better provide some evidence to back up your claim. That’s especially true in the IM world where – unlike phones where you can compare specifications – it’s a lot more difficult to make fair comparisons.
Finally, it’s important to actually acquire and use whatever it is you’re reviewing. In the past, we have reviewed stuff we didn’t personally use (on Health Ambition, not Authority Hacker), but you can always give a more objective review if you have first-hand experience.
By simply resting your opinion on what other people have said, you run the risk of echoing their bias which may or may not reflect your own views. In the end, you’re not doing your audience (or yourself) any justice.
If you can’t try the product/service for any reason, you can sometimes take advantage of other marketplaces. Amazon is a good example. You can see if someone has purchased a product because it labels them with “verified purchase”, and you can combine those for an overall opinion that ends up being pretty accurate.
The Effects Of Negative Reviews
Leaving a negative review can sometimes – as we’ve touched on – have a negative impact on the company behind that product or service.
A few years back, Mark took a trip to Thailand and decided to book a cheap hotel online. Although he didn’t pay in advance, it was still a confirmed booking. On arrival, they said his booking wasn’t received and that there were no rooms available.
Mark left a review on TripAdvisor, explaining exactly what happened. The review got a ton of engagement and there’s no doubt it would have deterred a number of people from making a booking with that hotel. Ultimately, they lost sales.
Mark also reviewed a restaurant in Budapest (yes, Mark loves to leave reviews). The food was awful and overpriced and that, of course, was reflected in the review. This time though, the manager responded by claiming Mark was working undercover for their competitor – no doubt making it even worse for themselves.
The type of restaurants and hotels that do well tend to look at negative reviews differently. As a way to improve and take actionable steps towards that goal.
Finally, last year a company in the US tried to sue us for 5 figures because we said a couple negative things about one of their supplements. What we received was a fairly long PDF written by their lawyer, saying “you have 7 days to remove this and pay us X amount of money”. Very worrying at the time.
Turns out, the statements we made were written in a way that protected us. Something along the lines of, “some users have reported defective packaging”, which was true, people HAD said that on Amazon and we were able to verify that. If we didn’t take that position, however, it could have gone differently.
Moral of the story: you have to be careful what you say online because you’re responsible for any statements (both positive and negative) you make.
At the end of the day, a review is always going to be biased at some level.
It’s not whether or not bias exists that should be up for debate, it’s whether or not that bias is in the interest of personal gain or the interest or the audience they are serving.
If someones entire business is based on affiliate income, they have every incentive to recommend the best paying products, and not the best value products. That’s the kind of bias you want to avoid.
On the other hand, if someones business is focused on offering opinions based on their own experiences and understanding of how things actually work, that’s a positive bias that you can absolutely benefit from.