I have a tool crush on Screaming Frog. I consider it one of the keys to scalability and I don’t know what I would do without it. Dan Sharp deserves a medal because his tool has saved SEOs countless hours of manual work. Screaming Frog can be used for a variety of things, but my new favorite use is for identifying internal link building opportunities.
Note: If the keyword is in the template of your site (i.e. on every page) this process will not work, this is optimal for long tail or secondary keywords.
Step 1) Identify the Keywords you want to improve rankings for
Conductor Searchlight is awesome for this, you can find the KWs with the most search volume that are just shy of ranking well.
Step 2) Put Keywords into Screaming Frog Custom Filters
Step 3) Run Screaming Frog & Export your Custom Reports
Step 4) Do a Majestic Bulk Upload of all your URLs to get Metrics
You need to run bulk look ups for each export, then once you export it from Majestic, delete all the data you don’t need and add a column with the internal link phrase in each export and combine all the data:
Step 5) Clean Up Your Data
Clean up your data, if a page has zero links pointing towards it, maybe it isn’t worth your time to add that internal link. You need to use your best judgement, but if the page is lacking on link juice, I generally remove it from my data set.
Step 6) Get Your Pivot On
Throw all this data into a pivot table and select just your most powerful pages for internal links. Going off of Trust Flow, Citation Flow and AC Rank metrics allow you to prioritize your internal link building. For those of you wondering how to set up and use pivot tables, you wont find that information here… go to Annielytics for that stuff, she makes data sexy. But if you want to see how I set it up, here is a Pivot Table of SEER Interactive’s internal link opportunities optimizing for the keywords: link building, audit, international and pay per click.
John you have no fan page in facebook? i want follow you, but cant find!!
@Soy un Gnomo Nope, no facebook fan page – however, I am pretty active on Twitter. Please feel free to follow me there @JHTScherck – thanks for the interest!
Xenu can also come in handy in such case. I usually look for lonely pages with no or less internal links. It is always good to give them new life through internal linking.
@geniusgeeks As far as I know, Xenu doesn’t crawl the on page text, so although you can find pages with only a few internal links – it’s not great for finding specific opportunities that already exist on your site. However, if Xenu can find individual words on a page – please let me know!
Thanks for this post I have been so focused on off page optimization I will need to re-visit my internal pages for new opportunities for SERP results.
Dude, really like this post. Smart thinking and easy to understand. Nice one!
@s_rvll Thanks buddy!
for the part about pivot+majestic, you could save time here using Excel for SEO and get the authority data you are looking for.
digitalmarketer True. Good point, I may start doing that myself.
Great post here JHTScherck — any recs on how to manipulate screaming frog for sites that have > 50,000 pages. I’m looking for ways to speed this process up. It might just be the nature of the beast, in which case I’ll make due!
Either run the site in segments throw screaming frog, or use 80 legs.
John-Henry,Excellent step by step guide. Indeed, Screaming Frog is really a very helpful tool. In fact aside from determining link building opportunities, the tool is now – during SEO’s Penguin era – considered as a vital tool to check the status of link removals.
spookseo Oh yeah – the UK team at SEOgadget uses it regularly for just that 🙂
Another way to do this is using Google and MozBar. Set your search settings to 100 results, turn on the SERP overlay and do a search on your site along the lines of [site:mydomain.com ~keyword] or [site:mydomain “keyword”]