Psssst.
Wanna know how to make your content more SEO friendly…for free?
How to make it rank higher for its target keyword in just a few days.
And also rank for a whole new set of keywords at the same time?
That’s exactly what something called TF-IDF can help you achieve.
What is it?
Patience Padawan, patience.
The really cool bits of what I’m about to share with you are:
- Anyone can do it
- It takes almost no effort
- There’s a free tool to help
The net result here is you can potentially get more organic traffic for spending 10 – 15 minutes “upgrading” your pages.
This is what we have done on several pages of this site, and here are the traffic bumps we got in most cases:
And this:
Or maybe even this:
And while you are at it, you can apply our date update freshness trick (which we did apply to several pages shown above) to get an even bigger bump.
Note however that some of the date ranges above include the Christmas period which overinflates some numbers.
We also had some pages that did NOT see an increase in organic traffic despite being run through the same process. So don’t expect a magic pill tactic. For us, we saw traffic increases about 75% of the times we applied TF*IDF (not scientifically tracked).
So if you are in a competitive space, it’s probably worth a look.
So yes, it’s not a one hit wonder, but overall, the experience has been positive enough for us that we will keep using it in the foreseeable future.
Here’s a quick promise: By the time you finish reading this blog post you’ll know enough to go use this SEO tactic to give your rankings and traffic a boost.
Excited?
You should be!
What Is TF-IDF?
It stands for Term Frequency-Inverse Document Frequency.
Wait, come back, this isn’t a math lesson!
It’s not as complicated as it sounds.
But it’s also why mathematicians shouldn’t be allowed to name things.
TF- IDF has been around since 1957 when computer scientist (yes, there were computers at that time) Hans Peter Luhn first came up with it.
It’s a really simple idea, but a brilliant one.
How TF-IDF Works
Let’s look at this from the perspective of a search engine since they are the ones using this kind of algorithm to establish relevancy.
Let’s say you have an index with just 10,000 pages, but you want to quickly find pages about a certain topic.
You decide the best way to narrow your search is to look for pages that only contain certain words.
So you create a piece of software that scans every single word in every page and collates the most frequently used strings of 1, 2 and 3 words.
That is Term Frequency (TF).
Unfortunately, the results include lots of very common words like “the”, “and”, “but”, “where”, etc which pollutes the results.
This is where Inverse Document Frequency (IDF) comes in.
IDF tries to estimate the value of a word or word string by looking at how unique it is across the entire index of pages.
Thus weeding out common language from establishing relevancy.
By combining these 2 algorithms, search engines manage to extract keyword relevance from web pages.
It also gives them a good idea of which ones should show up at the top of the SERPs.
Once they have done that, search engines can then use the few pages deemed as relevant for a term and look for correlations with new pages entering the index.
That means search engines will not only look at the one keyword you target but at the entire TF*IDF spectrum of the relevant pages, and then compare it to yours.
So, for example, for this article about TF*IDF, adding complex notions like the fact that IDF is “logarithmically scaled” and therefore harder to abuse by sneaky SEO’s will increase our relevancy for the term “TF*IDF” because the Wikipedia page for TF*IDF also talks about the same thing.
And this kids is why cheap generic writers are not a good fit for SEO.
They lack the specific niche vocabulary that will make your content be seen as relevant by these algorithms.
And as a result, your cheap content will plain not rank.
Let’s talk about how we can fix this for you.
Wait…Is This Keyword Density 2.0?
Nope…not really.
It’s not keyword density as you might remember it.
Like when all the “gurus” were advising site owners to include their primary keyword once in the first paragraph, twice in the body, and once at the end of the blog post?
This isn’t the same thing.
That approach actually worked once upon a time.
Until Google released the Florida update in 2003, which was the first warning sign that keyword stuffing is a bad idea.
The 2011 Google Panda update wiped tens of thousands of low quality (thin content) affiliate sites off the face of the Internet for much the same reason.
People stopped talking about keyword density, except in hushed voices at SEO conferences.
Just in case Matt Cutts had the place bugged.
TF-IDF is more about overall page relevancy and far less about “keyword density”.
And this relevancy comes from focusing on all the other words related to a specific search query, and not just your core keyword.
You’re not looking to shoehorn keywords into certain parts of a page.
Instead, you’re just feeding Google what it wants- thematically relevant content.
Why TF-IDF Is Important for SEO
It’s important because you have two choices when it comes to ranking in Google:
- Use your best guess at what works
- Reverse engineer Google’s results
As much as Google would like you to believe that there’s no “formula” applied in how they rank content (ummm…algorithm), there absolutely is.
And we’re not speculating here – we’ve tested it and we can and do back that up with results.
When To Use TF-IDF
The best time to use it is when you’ve invested a lot of resources in creating a great piece of content that’s not doing so well.
Maybe it’s lingering on page two of the SERPs.
Or maybe it’s been around for a while and the traffic and rankings are starting to drop off.
Established content is a prime candidate for a TF-IDF makeover.
But it has just as much value when you’re creating new blog posts.
You can use it to optimize your content in a way the majority of your competitors simply are not.
Then you get results like this:
Now, let’s look at how you actually use this SEO tactic, but using a tool to make your life easier.
What TF*IDF Tools Are Available?
TF-IDF has been around for a long time, but it’s only recently that mainstream SEOs have started to pay attention to it.
So we’ve yet to be swamped by dozens of software developers all shilling for your monthly subscription monies.
But there are already some very interesting on-page tools that can help with your TF-IDF analysis.
Some of them are more generalized on-page SEO tools, while others are dedicated TF-IDF solutions.
Before we dive into them, I think it’s important to note that real TF*IDF is not really what any tool does.
If you remember the previous sections, IDF is supposed to check your page against every single page on the internet.
And well, that’s not exactly practical in terms of resources.
So each tool kind of comes up with an alternative solution that’s close enough.
In most cases, they either have a reference index they maintain internally to compare any given page to, or they focus more on correlation across the set of pages that rank for a given keyword.
We will be reaching out to them to highlight the process they use in each mini review below.
Dedicated TF-IDF tools
Text Tools
General On-page Analysis Tools
The software and services listed here are not 100% dedicated TF-IDF tools, so it’s not fair to refer to them as such.
It also has come to our attention that several of these tools do not use TF-IDF per se but variations of it such as LSI keywords or “most prominent keywords” for various computing and SEO orientation reasons.
But each of them does provide you with the ability to at least figure out what words are missing from your page and produce roughly similar results to what we outlined previously in this post so we deemed them relevant.
MarketMuse
Cora
PageOptimizer Pro
Seobility
Surfer SEO
And yes, we have tested each of these tools.
In terms of overall value for money, it’s hard to beat Website Auditor, because it doesn’t cost anything, or at least the free version doesn’t.
For a grand total of zero dollars/euros/pounds per month, you get pretty comprehensive TF-IDF research functionality.
Is there a downside?
Yup, you can’t save projects with the no-cost version, so you need to start every project from scratch each time you’re doing any analysis.
That might not seem like a big deal, but it can grate on you after a while.
Now, the obvious question you’ll ask is what on-page tool we use for Authority Hacker content, and our other sites?
Although we have a lot of love for Website Auditor, we recently swapped over to Surfer SEO.
Website Auditor is an excellent tool, but in terms of user-friendliness, range of features, and overall results, we prefer to use Surfer.
The fact it’s cloud-based is another perk – we’re not tied to a local software installation.
One thing to note is that Surfer SEO actually uses a “prominent phrases” algorithm that cleans up the most common words of the english language.
Basically it’s not pure TF*IDF but the idea is the same and results have been similar to what we experienced with Website Auditor.
But for now, let’s show you the free way of adding TF*IDF keywords to your pages with Website Auditor.
How To Run A TF-IDF Analysis – Free
First things first, you’ll need to download, install and launch Website Auditor – it’s free, so you have no excuses.
Now choose a URL you want to analyze for a specific keyword.
In our case, we’re going to analyze one of our own pages for the keyword “make money blogging”.
Step 1 – Enter your URL
Click ‘Finish’
P.S. We’re not creating projects because we’re using the free version of the tool – it doesn’t allow you to save any data.
You might notice a little spinning logo in the top left-hand corner of the page saying ‘Rebuild Project’:
This is normal, so give it a few minutes to finish doing its thing.
Step 2 – The Audit
The next step is to click on the ‘Content Analysis’ menu and ‘Page Audit’ should load automatically.
We then choose what page we want to audit and click ‘Next’.
Now it’s time to choose the keyword we want to analyze.
We want to optimize the page for “make money blogging”, so that’s the keyword we enter.
Click on ‘Finish’ to get…started.
Watch out for the progress bar in the bottom left-hand corner of the page.
It takes a few moments for the software to fully analyze your page, so don’t interrupt it.
Take a few seconds to check out the ‘Page Audit’ report once it loads:
We can see here that our keyword isn’t in our Meta Description, so that’s something we need to take care of later.
Spending a minute or two in the ‘Page Audit’ section is always a good idea – it’s easy to miss those little things that can make a big difference.
Let’s keep moving.
Step 3 – TF-IDF Analysis
Now let’s get to the TF-IDF analysis by clicking on ‘TF-IDF’ in the left-hand menu.
Website Auditor automatically scans the top 10 results in Google and compares your page against them.
It’s always worth checking that the sites you’re compared against are blogs on the same topic.
Why?
Because there’s no point comparing your post to a Wikipedia page, Amazon product page, or something like that.
You do that by clicking on the little gear icon in the top right-hand corner.
Now for the really cool bit – the keyword results.
The ‘Keyword’ column shows you all the keywords that are relevant to the pages analyzed for your core keyword.
Basically, these are words that Google expects to find in a blog post on “make money blogging”.
We only use the “Multi-word Keywords” for our TF-IDF process, but not everyone uses this approach.
This is usually a pretty long list, but don’t worry about that for right now.
There are 4 possible recommendations for each keyword:
- Ok – no need to do anything
- Use more – add additional instances of this keyword
- Use less – remove some instances of this keyword
- Add – add this keyword at least once
Do you need to include every single one of the keywords on the list?
In an ideal world, you would, but you’ll notice that you’ve already included some of them without even trying.
This is one of the other benefits of using writers who specialize in a niche – they naturally include many of these key terms without being asked to.
For the rest, focus on the keywords used by the top 40% – 50% of your competitors.
These are likely to hold more “weight” in Google than keywords featured towards the end of the list.
Upgrading Your Content
Website Auditor can only check live pages, which means you need to add the missing keywords to your content, update it and then run the analysis again.
You’ll have to do this anyway, but it’s a shame that Website Auditor doesn’t have a preview function/text editor like MarketMuse.
But hey, it’s free, so I’ll stop nitpicking.
How can you make sure your keywords are included on your page?
The first thing is to use notepad, Sheets or Numbers to create a list of the keywords you want to include.
Then copy and paste your blog post into Google Docs.
Work through the post, adding the missing TF-IDF keywords into your headings, subheadings, and page content.
Take your time – this isn’t a race.
When you’re done just press CTRL + F (CMD +F on a Mac) and type in the keyword you’re trying to optimize for:
You’ll see how many times that keyword now occurs in your document.
In our earlier analysis we saw that “make money blogging” was only included 8 times in our post:
So we’ve bumped that up to 11 within the post, and we can squeeze another instance into the Meta Description tag, for a grand total of 12.
Why are we not including exactly 16 instances of our phrase/keyword?
Because we like to test making incremental changes and see what results we get.
All part of the “minimum viable product” approach.
Remember to run this same check for all the other keywords you add to your page.
The purpose of TF-IDF is to improve the relevancy of your blog post using several multi-word phrases – not just focus on a single phrase or keyword.
The Final Step
Once you’re happy you’ve added enough keywords into your document, upload the new version of your page, and run the TF-IDF analysis again.
How quickly can you expect to see a bump in traffic?
That all depends on how much domain authority you have and how often your site gets spidered.
Experiences vary, but you should see a change within a few weeks.
What Do The Results Look Like?
So, yeah this all sounds great.
But the real question is – does it actually work?
Here’s some proof.
This is the net result of only using TF-IDF to optimize a single page of content.
And these results took less than a month to show up.
‘Page Sessions’ increased by 98.01%.
Or in real numbers, this one page went from 902 sessions in December 2018 to almost 1,800 sessions in January 2019.
No vanity metrics here – just lots of extra traffic.
And from doing nothing more than adding some relevant keywords to our content mix.
There’s no such thing as a “ranking formula”…right?
Summing It Up
Is TF-IDF some kind of secret sauce for on-page SEO?
No.
But it is a scientific way of upgrading your content to match Google’s “requirements”.
It’s also a perfect addition to regular keyword research.
But which of the above tools should you choose?
If you’re working on a budget then Website Auditor makes the most sense.
Surfer SEO, on the other hand, comes highly recommended as a “premium solution” – you do get a whole lot of value for $29 per month.
Your biggest competitors are using TF-IDF, so shouldn’t you be doing the same?
