How to Do Conversion Rate Optimization on Your Affiliate Site (Step by Step Guide)

Gael’s Note
Everything we will talk about in this blog post happened before the update that hit Health Ambition, Nevertheless, traffic loss doesn’t affect conversion rate so everything that is mentioned in this blog post is still valid and useful for you. Plus, Kurt’s work is clearly going to help whether some of the revenue loss we experienced for that update.

About 6 months back, we started conversion rate optimization on Health Ambition

3 months after we started the campaign, the results were solid.

• 23.3% more orders
• 26.1% more products shipped
• 26.6% more revenue

By the end of this blog post, you will know EXACTLY how to reproduce these results on your websites too (plus see some embarrassing photos of me in uniform….seriously).

We’ll break it all down, step-by-step from the beginning.

Hey guys and girls, my name is Kurt, and I’m the founder of Convertica, a CRO agency that helps authority site owners optimize their revenue per visitor. We have helped hundreds of websites at this point including the sites of many popular bloggers in this industry.

But to be honest, things have not always been easy. I used to be a butcher Apprentice when I was 18 paid $8-12h in New Zealand.

I then quit that crappy job and became a backpacker in Asia. Your typical half-assed SEO guy that would register exact match domains and spam the hell out of them to rank for keywords that would help me pay my cheap room in Chiang Mai.

But I was sloppy and my focus was more on partying than build something of value. And Google saw that.

When Penguin hit, all my PBN’s got deindexed and all my sites fell off a cliff and that is when I learned that if I wanted to last in this industry, I had to build something of value.

Unfortunately, I wasn’t as nerdy about SEO as a lot of my peers and my mentor Grant always said “Focus on what makes people take purchase decisions”.

So I did that and decided to dedicate 100% of my time to building something that lasts and focuses on making people buy. That’s when I stumbled into CRO (Conversion Rate Optimization)

CRO is something I have been nerding out on for years now as I discovered most of the industry was so focused on growing their traffic that they left little energy to transforming traffic into money in the most efficient way possible.

This is where I saw my opportunity to fit in and bring value: work on people’s site to maximise their revenue while they keep their eye on traffic.

At first, it was just me, but as demand grew we put a small team of hand selected talent together to become today’s Convertica team.

The Process we Developed.

Today I am going to share a process that works on any affiliate website. This is a process we have developed over the last few years that works on identifying the 80/20 to deliver the quickest results so you can generate more revenue, quicker. It’s the exact process we used to increase Health Ambition’s revenue by over 26%.

Once you identify insights from working on the top 3 to 5 revenue & traffic money pages for your website, we can start to move to the lower hanging fruit.

In this post we cover:

• Where to start, identifying the 80/20.
• The best tips for A/B testing success
• The highest value affiliate site elements to test
• A process that nearly guarantees positive gains
• Mistakes you must avoid (and making CRO safe for SEO)

Let’s get to it.

Ⅰ: The Basics

1.1: What is A/B Testing?

A/B or “split” testing is the process of testing two (or more) versions of a web page against one another to see which achieves its goal at a higher rate. The “goal” of a page is to get a user to take a desired action, for example; buying a product from your affiliate link.

It might seem complicated at first, but it’s actually really simple in theory:

• You select an element of your site you want to test (we’ll teach you how to do this in a bit)
• You make a change that you theorize will increase conversions
• You test the new version (the variant) against the original (the control) by sending 50% of traffic to one version and 50% to the other
• You measure which page converts the user to the desired action at a higher rate.
• You roll out the winning variant live on your website.
• Rinse and repeat the process.

What makes A/B testing so cool is it does away with theories (sort of) and tests what actually works with real users.

Rather than thinking “hey, I wonder if this works” and taking shots in the dark, you get real, definitive proof that it works without making expensive changes or risking 100% of your traffic on changes that turn out disastrous.

In essence, it gives you cold, hard data (don’t we all love that?) so you can avoid Sherlock’s capital mistake.

1.2: Why A/B Test?

If there’s one thing I’ve learned, it’s no matter how “aethstetic” a website is, there is always room for optimization and growth.

Most, and I believe about 90%+, of websites have never had a single AB test run on them. When you consider the fact that converting more of your current visitors into paying customers is by far more cost effective than bringing in more traffic, it’s strange that site owners focus so much on increasing traffic rather than increasing conversions.

Therefore, if you already have traffic and you haven’t started testing yet. You are missing a huge amount of revenue.

Also, it lets you test your hypothesis “in your own backyard”. If you notice a style of product copy performs better on one page, it gives you valuable insight into how to optimize your copy.

To illustrate how much of an impact CRO can make to your business, I have prepared the graph of 2 businesses. One that grows 10% traffic per month and one that grows 10% traffic and 10% conversion rate per month. They both start at the same level yet only 12 months later, the business that does CRO with traffic growth ends up making more than 3x the money the site only growing traffic makes.

It’s like compound interests for website revenue :).

Ⅱ: How to Run a Split Test

2.1: Quick Wins for Affiliate sites

“A/B testing sounds great, but I don’t know what to even test.” – Josh the site owner

Figuring out what to test is probably the biggest obstacle for most site owners.

Successful tests require thousands of visitors. The last thing you want to do is waste valuable traffic on changes that will have little to no impact (or worse, negative impact) on your conversions.

We’ve discovered that the best way to figure out what to test is to ask your visitors. This can be done with survey tools like hotjar or JotForm Survey Maker.

We have also discovered that most people don’t like surveying their visitors.

Luckily, there are some page-level elements that have proven time and time again to produce big wins almost every time for classic affiliate sites. For your first tests, consider these high-impact areas in order:

• If you don’t have one, test a product comparison table
• Call to action button copy and placement. (“Check Price” worked best on Health Ambition)
• Pros and cons in your mini review areas. It allows users to make an educated decision “for themselves”.
• Images. From tests we have run, nearly 100% of the time, product images will beat no product images. It allows the users to visualize them with the product.

Mobile view of the winner

The last thing you want to do is take shots in the dark. Misguided A/B testing can only lead to less desirable outcomes.

Why you can’t just test for the sake of testing: A/B testing is inherently a double-edged sword: it’s only valuable to test when you’ve got enough traffic, but the more traffic you have the higher the risk.

For example, if you have 20,000 visitors per month to a site that’s doing well (with a 5% conversion rate) and you run a poorly managed A/B test, you just sent 10,000 visitors to variation that they didn’t like and wasted a few hundred in sales (oops!).

This also means A/B testing gets more challenging as you keep going as once you have made initial easy gains, you need to come up with page variations that beat already high converting pages.

I hope that doesn’t scare you away from A/B testing for your affiliate site. It shouldn’t. A well-run A/B test is almost certain to provide huge value.

All you really need for a successful A/B test are well managed processes.

2.2: How to Structure Your Tests For Maximum Success

Now that I have given you some quick wins you can focus on when starting to A/B test, you still need to test them to see if they improve your bottom line.

My thought process is this when it comes to product review and roundup pages:

Following the 80/20 rule again, there are 3 types of users that land on a page ranking for “best X review”:

1. Someone looking to see the top 5 best products in a quickly scannable comparison table that are ready to buy but just want to see a few options side by side. ie: “Most economical”, “highest quality” or “best for enthusiasts”, for example.
2. People who aren’t quite ready to buy and want to make an educated decision. This is where a mini review area down the page for each product in the comparison table does well. Allow people to click over to Amazon or click “read review” in the comparison table so they can make a quick decision.
3. The rest are just generally browsing and aren’t ready to buy anyway.

Therefore, we need to stay focused on testing the top one third of the page in isolation.

While we are testing one area, it is very important not to test anywhere else on the page that would impact the tests.

Step 1) Identify the highest earning pages on your site.

As a rule of thumb, 80% of revenue is generated from the top 5-10 pages on your site. That percentage is skewed toward the high end, with the top 3 pages producing the majority of that revenue. Basically, what we’re trying to say is that your site is almost certainly top heavy.

So, to get the quickest results, always target the top 5 highest earners. This is exactly what we are going to do here.

Remember: Don’t waste time and money on things that have little impact. Remember the 80/20 rule!

Let’s use an Amazon affiliate account as an example on how to set this up. The same principles apply for other affiliate platforms (with only slight differences from time to time).

Pro Tip

If you don’t already have separate amazon SUB IDs on each page. Do this now, it provides great insight to where the revenue is being generated. The new Amazon content insights reports also allow you to extract the revenue generated on a page level

Step 2) Pick 1 test to run per page and roll wins sitewide

In order to get the best results, it’s crucial to test these areas in isolation starting from the highest click frequency to the lowest.

The reason we do it in isolation is because area 1 could have a 10% increase in conversion while area 2 could have a 10% decrease and completely offset the gains from the first test.

Testing in isolation evaluates each change on its own merits without the risk of polluting the results.

Multivariate testing is also a kind of testing that exists but it is usually reserved to extremely high traffic websites and generally does not apply to affiliate sites for this reason.

So all you have to do is pick one of the quick wins area I gave you above and test one of them on one of your highest earning pages.

Once you establish a win, roll the change out on all your pages to benefit from the test you ran sitewide.

Key Takeaway

Remember, A/B testing is about well-managed processes. You never want to test for testing’s sake. Use this process to determine the high-value areas so you always get maximum value from each test.

2.1: Affiliate Site CRO Checklist

When you work on your site day in and day out, it’s easy to get so used to the way things are designed that you don’t think they would affect your conversions.

This is why A/B testing is so powerful. It can highlight things you have done wrong for years and show you easy ways to get more out of your website.

To get you going with ideas of things to test, I’ve prepared a list we use internally that often gets results. It’s not as foolproof as the list I gave you earlier but it still has a very high chance of making your conversions better.

• How are Your Comparison Table Features?: Sometimes they’re needed and sometimes they aren’t. It depends on the product. The more complex a product, the more features are needed (in general).

• CTA Alignment: We’ve found that horizontal tables with the CTAs aligned vertically (along the Y-axis) outperform their counterparts.

• Do You Explain Your #1 Product?: 3-4 reasons why you chose your #1 product makes a HUGE difference.

• Is Your Comparison Table in the Right Position?: When your visitors are searching for keywords with “the best” in them, don’t beat around the bush. Put your table at the top with a clear CTA.

• Do You Have Pros AND Cons in your mini review areas?: Cons, interestingly enough, increase consumer trust and sales. Make sure to have Pros vs cons below the product.

• Images: Simple enough. Hi-res images increase sales. Emotionally connect with users through images.

Pro Tip

We’ve found that the more a visitor has to think the lower conversions will be. If you want higher conversions, make it as simple as possible for your visitor to get the products they came for. For example, a product review and payment gateway on the same page deletes an unnecessary step and could increase sales.

Key Takeaway: When testing, always focus on the highest-impact elements at first.

Ⅲ. How to Set up and Start Your First Test Like
a Boss

OK, we’ve got all of the art of an A/B test out of the way, so let’s get into the nuts and bolts. We’ll go step by step, end-to-end through your first A/B test. This should give you the blueprint for crushing your first test and reaping the benefits.

Getting set up and running isn’t all that hard, but you have to be extra careful that you get it right or else you could negatively impact your results, your site, or both.

Buckle up:

3.1: A/B Test vs. Split Test: Wait, they aren’t the same?

If you aren’t familiar with CRO, you probably use these terms interchangeably. It’s not the end of the world, but when it comes to actually running a test there’s a big difference. Here’s the skinny:

• A Split-URL Test is exactly like it sounds. The original is tested against a variant (s) hosted on a completely different URL. Think of it as two totally separate versions of your page. Split URL tests are necessary for major back-end changes (new product tables, totally new layouts, etc.)..

• An A/B Test is for smaller, page-level elements. These tests are hosted on the same URL. Here you’ll test smaller changes (buttons, images, headers, etc.) against each other.

3.2: Setting up an AB Test in Visual Website Optimizer

We’ll use Visual Website Optimizer to demonstrate how to set up a test since it’s the most common choice (due to ease of use).

To get started, you need to:

1. Click Test on the nav menu to the left and select A/B.

2. You’ll now be on the A/B dashboard where all of your tests will be stored. For your first test, click CREATE. You can’t miss it—it’s the big blue button in the top right.

3. Give your campaign a name and create a hypothesis (we cover this in detail below). We recommend creating your own custom one. This is very important, especially when you are running dozens of tests at the same time. It allows you to see your original hypotheses without getting data paralysis.

4. Enter the URL of the page (s) you wish to test. You can configure advanced options but let’s keep it simple for now.

Now, click Next to create your variation.

VWO will take you to a visual editor.

By default you will be on the Variation B view.

Hover over any element on the page and you can right click and EDIT HTML.

This is how you add in or take away the HTML you want to test on the Variation B.

You can even right click over an element and just change the text too.

Now, you will have to know basic HTML and CSS, or have a developer on hand to do it for you, but it’s as easy as that.

3.3: Our Testing Process

Step 1: Set up your AB Test the RIGHT WAY

We used to do things the old fashioned way—set up the control and one test, then send 50% of traffic to each one. Turns out that the Amazon sales data wasn’t really aligning with our results (we explain in detail in a moment).

Convertica designed this MacGyver hack to get around the issue: instead of just one control, we duplicate the original variation twice. Then we send 45% of the traffic to each duplicate variation and 10% to the original (making sense?).

• Original Variation (10% of traffic)
• Control Variation (45% of traffic)
• B Variation (45% of traffic)

Each duplicate of the original is set up with a unique Amazon tracking ID and the changes are made ONLY TO VARIANT B.

Still with us?

The “Control” page is an exact duplicate of the original and the “Variant B” page is our hypothesis.

You can duplicate your variations when we are in the visual editor of VWO.

See below.

Now make sure to label your Variants correctly.

How to set up traffic alocation correctly.

On the last page of the setup after the goals, you can allocate the correct traffic here.

Why 3 Variants – The Reason Why 90% of Affiliate site Split Tests FAIL

Most split tests stick to the old fashioned way, and this is the the main reason why they fail.

The reason being because in this situation the original will get around 1.5x-2x the page views. This is because:

• VWO doesn’t’ track visitors who have disabled javascript
• VWO doesn’t track visitors who’ve disabled cookies
• VWO doesn’t track visitors with a slow internet connection

So these 3 segments won’t be picked up by VWO and will be sent to your original page, skewing the Amazon sales data in a big way.

By segmenting the traffic 45/45/10, we achieve perfect segmentation and calibrate the data. Pretty cool huh?

3.4: Defining Conversions – How to set up goals

A conversion is when your visitor takes the desired action. You’re probably most familiar with your site’s main goal: to get your visitor to buy your affiliate product.

But with VWO, it’s a bit different than what you might be used to. The cool thing about VWO is it lets you tie each action a visitor takes while on your site to a goal, and a conversion is when the visitor takes that desired action. These are called “conversion goals”.

Why tracking goals the right way is important

Goals determine if a test was a success or failure, so it’s safe to say they are extremely important. So pay special attention to this part. You must track everything on the page to determine if you’ve really reached your goal.

Let us repeat: TRACK EVERYTHING.

Why? Here’s an example:

Say you want to test the top CTA on an affiliate page that has 4 CTAs. You form your hypothesis, set up the test, and run it, and you are pleasantly surprised to see an increase of 47% from the first CTA.

However, if you had tracked the other CTAs, the picture would be way different.

• 2nd CTA: 1% increase.
• 3rd CTA: – 10% decrease.
• 4th CTA: – 39% decrease.

The test you thought was a 47% success was actually a -1% failure!

The funky thing with Amazon affiliate sites is that clicks don’t always tell the full story. Your goal is to increase sales, right? That’s why you have to track the end goal (revenue).

That requires tracking everything on the page.

Sure, you got more clicks on your top CTA, but this can have a counterintuitive negative effect on the page as a whole. Sometimes you visitor will click that nice shiny CTA at the top, see it’s not what they need and bounce without taking the time to look at all the other awesome products you have in stock (theortically).

That’s why tracking revenue in detail is so important.

Now, onto the good stuff. How to actually set up goals and what they mean:

How to Set up Goals

In VWO, the goals cover every possible action your visitor can take (talk about comprehensive).

From the test dashboard, select Goals and you’ll be presented with a drop-down menu full of some real valuable goals to track.

In VWO, you can track:

• Page Visits: If you want to track how many visitors land on a certain page (a Products page, for example), you can enter the URL into the box and VWO will track all visits to that page.

• Form Submits: If your site has forms, whether they be for contact information, event registration, or an email newsletter, tracking form submits can give you valuable insight into how a form is performing. You will need to enter two URLs (the page where the form is available and the page where the data is submitted). Be warned, if you have multiple forms on the same page, VWO will track them all.

• Click on Links: Pretty straightforward. If your goal is to get a visitor to click on a link (s), enter the URL of the page with the link and the URL of the page that the link points to. In case multiple links point to the same URL, you can use the Click on Elements goal instead.

• Click on Elements: This goal allows you track clicks on certain HTML elements of your page. For example, nav menus, buttons, or images. Specify the URL of the page with the element and then specify the CSS selector path. If tracking multiple elements (like the hyperlinks above), you will need a unique CSS identifier for each one.

• Track Revenue: This one is a bit tricky. To track revenue, you must implement a Javascript tag (along with the VWO smart code) onto the page. The page is typically the one where users mke the transaction. Implementing on this page ensures VWO only track successful transactions that generate revenue. For example, say your site requires users to submit a billing form. To set up a revenue goal, define your revenue tracking goal to be every successful form submission.

For amazon affiliate sites, if you are not cloaking links, you will just need to input the ASIN of the product you are linking to on amazon with the “clicks on link” attribute and set to “contains”.

We don’t want to use the full URL as a lot of the time there will be slight variations in the full URL string that will cause the link not to be tracked—for instance, if you are using two different SUB IDs.

Add a Goal for EVERY affiliate link on the page that uses a different ASIN identifier.

This is very important.

3.5: Setting up Sub Affiliate IDs in Amazon to Track Revenue

If you want your A/B tests to bear the most fruit, you’ve got to track everything end to end. That means both on your side and Amazon’s. So, now that we’ve got tracking ready to go on your end, it’s time we set it up on the revenue side.

Since you now have two versions of the same page, you need to set up separate Amazon affiliate IDs so you know where the money is coming from. You could run a test for 10 days and at the end Amazon would show you’ve made X amount of dollars.

What it won’t tell you is where that money came from. That’s why we create sub-IDs—so we can easily pull the data at the end of the test to see how each variation performed on the revenue side.

Here’s how a basic 10-day test would work with 2 variations:

• Any amazon links on AB Variant Original will have the ID convertica-20
• Any amazon links on AB Variant B will have the ID convertica2-20
• You run the test from 6/1 – 6/10
• When it’s finished, you generate the per SUB ID from Amazon (or relevant affiliate dashboard) and it shows you how each variation performed in terms of revenue.

How to Generate Tracking IDs on Amazon

Generating new tracking IDs is just a matter of a few clicks. From your affiliate account homepage, click on your email address and select Manage Your Tracking IDs.

Next, click the Add Tracking ID button. It’s that big yellow thing jumping out at you.

Now, all you have to do is name your new tracking ID and click Create.

Please remember: Create a unique SUB ID for both variations of the split test. We need to make sure there is no other traffic coming in from other URLs.

How to Generate Amazon Affiliate Reports

Once the test is finished, it’s time to generate affiliate reports to see how you each variation performed.

From the header menu, select Reports and then select Download Reports from the drop-down

Now it gets a tad bit tricky. Make sure to follow this step exactly so you get the proper data.

First, tick every box under Fees and select all of the tracking IDs from the Tracking ID dropdown menu. Now, select the right date range (the length of the test) and click Generate Reports.

This report provides valuable insight into what’s actually happening behind the scenes. An increase in clicks does not necessarily mean an increase in conversions. The report will show if you’re actually increasing the thing that matters most: Sales.

Tip

When you’re uncertain whether the test your planning will positively or negatively affect your conversions, try limiting the traffic you want to include in the campaign. You can set this on VWO (screenshot).

This way, you can reduce the impact in case your test doesn’t perform well. If you see positive results early in the test, you can slide it back up to 100% of the traffic to gain more insights and reach a conclusion faster.

3.6: Wait up, how is this safe for SEO?

“Wait a minute. Creating entirely separate versions of my web pages? Isn’t that going to confuse Google bots? And we’re only changing buttons, right? So won’t it all be duplicate content”? – most affiliate site owners

As an affiliate site owner, you are overly protective of your web traffic. After all, you’ve worked your ass of to get thousands of people coming to your site every month. This web traffic anxiety is a major obstacle that prevents affiliate site owners from taking up A/B testing.

The thought of losing search engine rankings due to a penalty from the wizards of Mountain View puts a lot of people off.

Trust us, it’s 2018 and Google even does A/B testing of its own. You have nothing to worry about. A/B testing WILL NOT negatively affect your search rankings.

Did You Know?

Google actually offers their own A/B testing service, Google Optimize? We think it’s safe to say they encourage A/B testing.

In fact, here is a direct quote from Google’s official blog.. …”we’re glad you’re testing! A/B and multivariate testing are great ways of making sure that what you’re offering really appeals to your users.

In our experience, the two major concerns when it comes to AB testing and SEO penalties are:

• Content Cloaking: You know that showing one set of content to humans and another to Googlebot is against Webmaster Guidelines, so when you see that Googlebot might see one variation and other visitors see a different one, it raises some cloaking concerns. Totally understandable.
  
  But don’t worry, A/B testing software (such as Visual Website Optimizer) is on top of things. It swaps content on the page using JavaScript, so some users see different content (of variations) and bots see the original page only. Once Googlebot arrives, it is assigned a variation and continues to see only that variation until the test is over.
  
  Google even quells your fears by letting you know that they’ll recrawl your site after a version has “won”. Basically, what we’re trying to say is that A/B testing is not content cloaking so don’t worry (we explain in detail below).

• Duplicate Content: Duplicate content is content that appears in more than one place on the web. We’re sure that you’re well aware that this is a big no no, and that Google’s furry-themed updates have all but squashed it out of the web (thank god).
  
  Trust me, I was no SEO saint back in my heyday, but duplicate content drives me mad. It’s easy to see why site owners might have these concerns though—”a nearly exact copy of my webpage will be created, won’t that be the same content in multiple places”?.
  
  Google recommends you use can0ncials when using duplicate pages for split testing.

Why A/B testing is not content cloaking

Cloaking content was all rage back in the Wild Wild West days of SEO (think 1990s).

Remember the days of unbridled keyword stuffing? (buy laptops online laptops laptops store online!). The way Google determines what your site is all about is by sending crawlers to fetch information to assimilate into its never-ending Borg-like database (Google is Borg. Resistance is futile).

These crawlers look at your keywords and assume that’s what your site is all about. This contributes heavily to your ranking for those terms.

So, smart SEO geeks (like me!) used to display a page full of keywords when a search engine bot visited to crawl/index the page. If a user visited the same page, they’d display the default (normal) version. This way, they could get all of those SEO benefits without putting visitors off.

This strategy was so effective the web became a wasteland of keyword-stuffed pages and low-quality search results. Google needed to clean things up a bit. Thanks to some cool algorithm updates, cloaking is obsolete.

You Should Know: A/B testing IS NOT cloaking.

Google’s PageRank algorithm has totally eliminated keyword stuffing, instantly banishing any stuffed websites to page 10 and beyond (the Google graveyard).

So, there is little incentive for search engines to penalize content cloaking. Moreover, unlike yesteryears’ static HTML sites, today’s search engines have come to expect highly dynamic AJAX driven sites.

So, they no longer consider swapping content dynamically as cloaking. If you use a multivariate testing software to swap different parts of your page, it is not cloaking.

Key lesson: Doing A/B testing by dynamically swapping elements on page (using JavaScript) is NOT content cloaking. It is absolutely safe and should not have any negative impact on search engine optimization efforts.

Not only are all of the major search engines well aware of testing, they encourage it! And all of the major testing tools have built-in mechanisms to ensure you tests are 100% SEO safe.

Verdict: A/B testing is safe for SEO.

3.7: Before You Start

Before you click the start test button, there are a few odds and ends that need to be put in place.

• Setting up tracking: Remember to get your tracking set up before you get the test started. You need to have definable goals and tracking methods in place to ensure you meet them. This could be tracking any of your goals—revenue, conversions, submissions, etc.

• Creating your hypothesis: The old adage goes “if you can’t explain why you’re doing something, you probably shouldn’t be doing it”. A/B testing is no different. Like all good science, your process starts with a hypothesis. It’s structure should be “ If X, then Y, because Z (your rationale). It could be “if I upgrade my images, visitors will buy more products, because Convertica told me so”. Kidding. The rationale is actually “because hundreds of A/B tests have proven high-quality images impact sales in a positive way”.

Pro Tip

Build data into your rationale. Each new visitor is a learning experience. The more data you gather, the more accurate your hypothesis can be.

Ⅳ: How to Manage Your Test

A/B testing doesn’t have to be hard. It’s really all about well managed processes. Data-driven, repeatable, proven processes are the key to success.

This is where so many fail. You can devise, hypothesize, and create the greatest test in the history of creation, but if you don’t manage it right, it won’t produce results. Here are the keys to running a successful test:

4.1: How Long to Run the Test

Patience. Avoid this common rookie mistake or your tests will be pointless.

When it comes to A/B testing, size matters. And you must have the patience to let the test run its course. Resist the temptation to pull the plug as soon as it looks like your hypothesis has been confirmed. Each test is different, so there’s no catch-all length of time. What really matters are statistical significance and sample size.

Statistical Significance

Statistical significance is the likelihood that your results are NOT due to random change.
Basically, it’s how confident you can be that your results are reliable. There are a million reasons why one variant can outperform another (for example, pure luck or seasonal triggers). Run your tests until VWO shows a significance of at least 95%.

Statistical significance: A result is statistically significant if it is not likely due to random chance.

Sample Size

Most of you are probably familiar with sample size. But when it comes to A/B testing, it’s extremely important. The smaller your sample size the larger the chance that your results are random chance. Don’t even think about drawing conclusions until at least 1,000 visitors.

If you cut yours tests off too early, you risk skewing your results in a major way. This would render all of your efforts pointless.

The reason why I can’t give you a number of visits to go for is because based on the conversion gains or loss, the number of visits will vary. So if a page gains 50% conversion rate with the variation, only a few hundred visits will be enough. But if the page only gains 0.5% conversion rate with the test, it will take tens of thousands of visitors.

But you don’t have to do the maths, let the software do it for you.

4.2: Ok so I have run my test for a month now and have enough data, what now?

There are a couple of ways we can analyze our split testing results.

You will simply be able to go into detailed report > variations and see which one has performed better. You can see above, that when we have selected the Variation 1 tab, 3 out of 4 of the links have a higher click through rate.

We need to add the total clicks and divide it by the total variant views for each goal and it will give you “clicks per visitor”. Once you have that number for each of the goals, you can compare which test converted higher.

Pro Tip

For data geeks. Really double down on by seeing what users generated what conversions down to device, country and even browser. You can do this below.

Ⅴ: Common Split Testing Mistakes All Newbies Make

Every A/B test has the potential to increase your affiliate sales or at least provide valuable insight to help you further optimize your business.

The thing with A/B testing is you not only have to set it up the right way and target the right elements, you also need to run it the right way. If you’re making some of these very common mistakes, your tests could have major negative impact.

5.1: 5 Mistakes You Must Avoid

We already mentioned them above, but we will reiterate them here:

• Calling a winner too early: Remember, it’s all about statistical significance. Resist the urge to be proven right and give it enough time to play out. You want 95% + certainty when you call a winner.
• Not refining your data: A test might show an overall negative result to conversions but if you refine, you may see a major increase in conversions for one device and a major decrease for another. This shows you’ve made the right choice for one and need more adjustment on the other.
• Focusing on the wrong elements: Make sure to test elements that have a major effect on sales (we covered those above). Use tracking and analytics (or survey customers) to further focus your testing. If a page isn’t part of your sales funnel, don’t waste your time.
• Testing too many items at once: Testing multiple elements at the same time saves time, right? WRONG. Cross pollinating your test results leads to disaster, trust us. Keep it simple at first.
• Not having a hypothesis: You can’t just go into a scientific process without a hypothesis. You need to know what you’re testing, why, and what you hope to achieve.

We know it sounds a bit crazy to say after bombarding you with thousands of words, but CRO really isn’t all that hard. It all comes down to well managed processes.

This process might be the result of the over 1,000 tests we’ve run in the last year, but it’s actually 20 years in the making. And if you had told me when I was a butcher’s apprentice making $8 an hour that one day I’d be CRO’ing websites doing tens of thousands in profit every month, I would have laughed it off.

We are incredibly proud of what we’ve achieved here at Convertica, but I’d be lying if I said I didn’t have A LOT of help along the way. Like I said, it’s important to have people telling you that you’re headed in the right direction. That’s what I hope this CRO blueprint is for you. I wish I’d had it way back in 2010!

If you want to read more of our CRO focused content, don’t hesitate to check our blog out over at Convertica! 🙂