Essential technical SEO audit tools for in-depth website analysis

Conducting a high-quality technical SEO audit for a new client is an important foundation stone for the long-term success of your web development efforts. Overlooking technical, content or off-page related issues affecting a client’s website could result in poor search visibility improvement or even a decline in performance. Fortunately, there are many SEO audit tools, both free and paid, that can help you to avoid missing any problems. This post provides web developers, designers, SEOs and other digital marketing professionals with an extensive list of resources to use throughout the audit process.

Update. Following some feedback on Twitter I should make clear that tools must never replace the important job of human analysis and interpretation. Tools are only as good as their users, as the wise old saying goes. For an insightful list of human checks, take a look at Andy Drinkwater’s excellent checklist or see my own checklist here.

Contents of this post

With this post my aim is to provide a definitive list of auditing resources, some well-known and others less so, organised by their primary function. You can navigate directly to the relevant category using the following jump links. N.B. This post is a work in progress. If there is a tool missing that you’d like to see added, please leave a comment below or send me a message. If your recommendation is strong enough, I will add a link to it within this post and credit you for it too.

1. Site crawl tools
2. Essential Google checks
3. Market research checks
4. Ranking checks
5. Algorithm update checks
6. Coding & development analysis
7. Log file analysis & server checks
8. Website speed and performance checks
9. Mobile friendliness tests
10. Website security analysis tools
11. Local SEO audit tools
12. International SEO tools
13. Optimisation analysis tools
14. Content auditing and analysis tools
15. Backlink analysis tools
16. Social media activity analysis

Site crawl tools

Usually, the first step I take when beginning an audit for a new client is to run a site crawl. A good quality crawler tool will highlight the most important technical and on-page issues affecting a particular web property. Seasoned SEOs should be familiar with at least one or two of these providers.

Screaming Frog is probably the most well known of the crawler tools available, used by thousands of SEOs globally. Suitable for crawling both small and large websites, its dashboards provide real-time analysis of typical SEO issues such as incorrectly formatted meta data and URLs through to buried content and redirect chains.

————————————————————————————————

Sitebulb is a relatively new contender in the website crawler market, built to “help you spot the underlying patterns that reveal the issues or opportunities you actually want to find”. It’s essentially a much more visual version of Screaming Frog but with added cool features such as interactive crawl maps that help you to visualise the content hubs and architecture of a particular site.

Edit. SF has now added visualisation features too, available in version 10.0 released 19.09.18.

————————————————————————————————

DeepCrawl is the go-to crawler solution for enterprise-level organisations with very large websites. Sites with many millions of URLs will probably want to invest in this software, which has successfully helped companies such as Autotrader and Made.com.

————————————————————————————————

Netpeak Spider is another relatively new contender in the web crawler market, which differentiates itself with flexible pricing and action-focused audit results. The team at Netpeak offered me a full-access free trial and I was impressed by the output of the software. I particularly like the dashboards, which provide a clear visual picture of issues that need to be addressed.

You can also quickly identify specific issues and the percentage of pages the issue affects.

You can gain full access to the software from just $19 per month, so this is a great option for SEO freelancers or smaller businesses that want to bring technical SEO functions in-house.

————————————————————————————————

Essential Google checks

Whilst the crawler tool is still running and fetching URLs from my client’s website, my next step is to take a look at the key data provided by Google. For me, this starts with Google Analytics and Search Console.

If you are dealing with a client who is struggling to improve their visibility on Google, then Google Search Console will provide some important clues as to why that might be. In fact, the new version of GSC (fully launched in July 2018) provides a much more granular view of the issues, such as specific reasons for URLs being excluded from their results.

————————————————————————————————

After reviewing Search Console, I’ll then take a look at Google Analytics to run a few key checks and note down my thoughts on the data. Typically, this involves asking the following questions.

• Is the analytics profile set-up and firing correctly?
• Any steep increases or drops in organic search traffic?
• Any drop in traffic to particular landing pages, especially the homepage?
• Are engagement rates on the site good or poor?
• How do mobile device engagement metrics compare to desktop?
• Has goal and e-commerce tracking been set-up correctly?

————————————————————————————————

Take the initial analysis of your client’s data a step further with Big Metrics. Helps identify keyword cannibalisation at scale, gain insight into newly ranking keywords and keep an archive of Google Search Console data, among many other things. Thanks to Dean Cruddace at Cultured Digital for this suggestion.

————————————————————————————————

Market research checks

After analysing the client’s Google data my next step is to look at the wider market they inhabit. You can’t carry out a successful audit for a client without properly understanding their marketplace, the competition they face and the volume of their search niche. There are many tools available that can help you to build this picture but the following are some of the most efficient available.

SEMrush could easily be added to several sections of this post including the review of backlink tools, later on. For me personally though, I find their software most useful for market research and planning. You can quickly see a competitor’s best keywords, for example, or the likely difficulty level of targeting a particular search phrase.

————————————————————————————————

I often use SimilarWeb when conducting opportunity assessments for clients as the visual data they provide gives a good overview of the likely levels of traffic to a site from each of the main online channels. This can help to identify the level of importance that SEO plays for a client’s competitor.

————————————————————————————————

I really like this free keyword tool by Neil Patel. Quickly generating keyword ideas using other tools available can often be quite hard work. This one gives you a fast overview of volume for a particular niche whilst enabling you to export and sort extensive keyword suggestions. This isn’t strictly a technical SEO audit tool but it does help to you to grasp the market and the size of the opportunity before you embark on writing up your audit.

————————————————————————————————

Ranking checks

After conducting the initial market research my next step is to set-up the client in a rank tracker and add in a handful of the key-phrases found using the tools mentioned above.

There are tons of rank trackers out there and all-in-one service providers such as Raven Tools and Moz include them in their monthly SEO packages. However, if you don’t want to pay the premium fees to use services like these, SERPRobot is a cheap but very effective alternative. For a low monthly fee you can set-up tracking for multiple websites, desktop & mobile, across several major search engines locally and internationally. When I start an audit for a client, I usually add a dozen or so keywords in here to help me get an idea of how well they are ranking in the specific search location they are targeting.

————————————————————————————————

If you’re working with a client who is targeting a very specific location, I Search From is a great tool for discovering where they rank when searching from that location, as well as being able to see the competitors that rank alongside them.

————————————————————————————————

Algorithm update checks

If, during the market research and ranking checks phase of your audit, you notice that a client has recently lost organic traffic, rankings or both, you will want to investigate whether or not this could be related to an algorithm update. There are a few tools that can help on this front, including the following.

Barracuda’s Panguin tool is a work of genius in its simplicity and execution. To use it you will need to be signed into Google and allow the app to access your Google Analytics data. From there, simply select the account you want to analyse then the tool will colour overlay the dates of major algo updates onto the traffic view. A simple way to see if a drop in organic traffic corresponds to a certain update.

————————————————————————————————

If a client has very recently experienced a drop in traffic or rankings, Algoroo might offer a clue if it’s algorithm related.

————————————————————————————————

MozCast’s weather report can also be used as a comparison to Algoroo (the former offering a longer term display of ranking flux).

————————————————————————————————

Coding & development analysis

By now, my crawl of a new client’s website should be complete and my insight into the health of their domain and the marketplace they are operating in should be much stronger. So, after analysing the results of the crawl test it is now time to look in more detail at how the site is coded and any issues that this might present. There are numerous tools to help on this front.

Whilst crawler tools like Screaming Frog help you to diagnose canonicalisation issues, I like to run this brilliant tool by the London based SEO agency, Found, as a backup. I like the visual display of passes or fails on canonical domain and canonical IP testing. You can also quickly see if common duplicate versions of the homepage (such as index.html) need to be redirected.

————————————————————————————————

This is a great tool for gaining a very quick insight into how a particular web property has been built. If it’s a WordPress site, for example, it will even tell you which plugins are installed. You can also see if a client has scripts such as Google Tag Manager in place.

————————————————————————————————

This is also a useful tool for checking if Google scripts such as Analytics, Tag Manager and Optimize are appearing on all of a client’s web pages.

————————————————————————————————

If a client is having specific problems with scripts and tags on their website such as Google Analytics and Google Tag Manager, the Web Analytics Solution Profiler, or WASP for short, is the perfect tool to assist.

————————————————————————————————

Although W3C compliance does not have a direct impact on SEO, the closer a page is to being 100% compliant will help search engines to crawl it more efficiently and distinguish the different HTML elements. This tool helps you to troubleshoot and fix those issues.

————————————————————————————————

This simple tool from the Russian search engine giant, Yandex, helps you to check for any syntax errors in a site’s robots.txt file. An alternative to Google Search Console’s robots.txt tester. Also, take a look at this excellent tool on Website Planet that clearly outlines errors and warnings pertaining to robots.txt.

————————————————————————————————

SEO Chat’s Free XML Sitemap Validator is a useful tool for checking the composition of a client’s sitemap. For larger websites, it’s imperative for search engines to be able to crawl the XML as efficiently as possible. Any issues should be flagged up in the audit.

————————————————————————————————

SEO Review Tool’s simple but handy Internal Link Analyzer is useful for quickly identifying the number of links on any given page, any duplicate links or missing anchor/alt text.

————————————————————————————————

This excellent plugin from Austrian link analysis experts, Link Research Tools, is really useful for checking if redirects are SEO-friendly or potentially harming a client’s website. The visual interface works very well, highlighting clearly how search engines and users are directed to resources within a site. Download it for Chrome here.

————————————————————————————————

LinkMiner is brilliant for quickly identifying broken links on any given web page. Simply download it for Chrome on desktop, then click the button in the top right of the screen.

————————————————————————————————

Identifying if a client has implemented their schema markup correctly is much easier with this testing tool from Google. Some clients may have failed to markup information such as their business details, products, videos and reviews, so it’s always good to check this in Google Search Console before you unleash this tool on individual pages.

————————————————————————————————

Whilst some may find it easier to simply look at the source code of a website, then copy and paste a screengrab of inline CSS (if there is any), this tool is good if you’re feeling particularly rushed and need a quick summary report of the issues.

————————————————————————————————

Many thanks to my friend Martin Wilson at Albion for reminding me to include this tool on my list. Whilst not specifically a coding tool, it is very useful for checking the history of a client’s domain. Sometimes the client’s domain may have been under ownership by another entity at some point in time. This tool can help to spot any issues with historical content.

————————————————————————————————

Log file analysis and server checks

Using the tools above offers insight into how a website is coded but it’s also important to understand how it is served to users and if search engine crawlers appear to have any issues accessing web pages and other resources. Sites hosted on a high-quality server, with minimum downtime, offer a distinct advantage over ones with regular server-level accessibility issues.

MX Toolbox provides a good, comprehensive health check of any domain you run a test on. The results include web server, DNS and blacklist errors and warnings.

————————————————————————————————

Whilst sharing the same server as low-quality sites might not necessarily present an SEO issue, it’s good to be aware of the neighbours in your client’s hood. This is a handy tool for quickly discovering which domains your client might be sharing a server with.

————————————————————————————————

This tool quickly returns the geolocation data for any domain that you enter. Useful to know if a client’s customer base is focused in a certain country.

————————————————————————————————

If a client is having indexing issues with their website, an important part of the investigation will involve log file analysis. In order to do this, you will need to ask the client to download the latest files from their server or web hosting platform. Analysing these files is made easy with Screaming Frog’s SEO Log File Analyser. Simply import the zip or gzip file and run the program, which analyses your server logs for crawler traffic such as Googlebot.

————————————————————————————————

Website speed and performance checks

In the smartphone era, website speed is everything. Users on the move, or with limited data, do not want to have to wait a long time to download a resource that might turn out to be low quality. It’s therefore essential for sites to load quickly and efficiently on both desktop and mobile devices.

PageSpeed Insights is the essential Google tool for desktop and mobile optimisation analysis. Simply enter any URL and it will output page stats and optimisation suggestions for each device type.

————————————————————————————————

Google Lighthouse is very handy Chrome extension for analysing performance issues on the fly. Install it in the Chrome browser then simply click the lighthouse symbol to generate a report for the URL you are on. The tool will return a list of scores and recommendations around performance, progressive web app, accessibility, best practices and SEO, as shown below.

————————————————————————————————

Test My Site is another useful Google tool that quickly shows you how well the search engine giant thinks your site is optimised for mobile users, complete with an industry comparison chart so you can see how well a site fares against its competitors.

————————————————————————————————

I usually run a GTMetrix report on a client’s website fairly early on in my audit process as the results are so insightful and informative. The presentation of the information makes it easy to digest, with tabs for each issue which open up to list every instance of the problem on a given URL.

————————————————————————————————

Different tools return different results, dependent on the test server region and the issues checked. It’s always good to run multiple tools by different providers and on different servers to get a broader picture of performance issues. Pingdom offers a clear visual grading of issues identified.

————————————————————————————————

WebPagetest.org is an open source project, developed and supported by Google. It distinguishes initial ‘document complete’ load time from the fully loaded page, for added insight.

————————————————————————————————

Mobile friendliness tests

Now that Google has moved over to Mobile-First indexing, it’s imperative for sites to not just load quickly but to look good and provide a high quality experience for mobile users. Whilst simply getting your smartphone out and using your client’s website on it should be an obvious starting point; there are also an abundance of tools to help dig into the issues at a deeper level.

To see if a page passes basic mobile friendliness criteria, simply process the URL using Google’s Mobile-Friendly Test. This test will only show you mobile usability issues for a given page. For insight into site-wide usability issues, you’ll need to review Google Search Console data. However, this tool is handy for checking if Google is unable to render the mobile version due to certain resources being blocked, for example.

————————————————————————————————

Want to know how mobile-friendly Bing thinks your page is? Try Bing’s Mobile Friendliness Test as a compliment to your Google tests.

————————————————————————————————

Assessing mobile usability issues isn’t just about running a site through a tool and seeing if it passes. You need to actually look at the site on various different devices to manually check how it looks. How do you do that if you’ve only got one smartphone? Tools such as Screenfly help you to quickly get an idea about how your client’s site renders on various different screen sizes and device types.

————————————————————————————————

A cool alternative to Screenfly is mobiReady which also generates useful data for improving mobile performance.

————————————————————————————————

Website security analysis tools

Google’s mission to make the web more secure, by setting a deadline for publishers to move their sites over to HTTPS by July 2018, saw millions of sites scramble to upgrade in time earlier this year. However, many publishers failed to implement the move meticulously by not applying the correct redirects (meaning both the HTTP and HTTPS versions of pages were still accessible), failing to address mixed content issues or simply not having received the memo about needing to secure their sites altogether. With major hacks leading to significant reputational, financial and ranking loss; ensuring that a client’s website is as secure as possible should be an important element of any technical SEO audit. Here are some handy tools to help on this front.

After using Screaming Frog to check that all versions of pages are correctly redirecting to HTTPS, my next step is to check the configuration of SSL. This excellent deep analysis tool checks the validity of any SSL certificate and determines if your site could be in danger of attack by oddly named vulnerabilities such as POODLE. The output is clear and actionable.

————————————————————————————————

Whilst conducting a full penetration test campaign against a client’s website might be a little outside the scope of a standard technical SEO audit; using Pentest-Tool’s light scan to check for high-risk vulnerabilities certainly adds a layer of gravitas to your report whilst potentially rescuing your client from a possible future hack.

————————————————————————————————

If a client has moved to HTTPS but is still getting the ‘not secure’ message in Google Chrome, Why No Padlock? will help you to determine the specific issues for any given URL. Can be used in conjunction with Screaming Frog’s ‘insecure content’ feature to identify and suggest a remedy to all mixed content appearing within a site.

————————————————————————————————

Local SEO audit tools

After completing all of the technical tests for a client, my next step is to look at the various content and off-page related issues that could be preventing them from achieving better search visibility. For clients providing services at a local level, it’s important to health check the set-up of their GMB profiles; ensuring that they are not using obvious virtual addresses or contravening other local search best practice guidelines. Whilst much of this work is inevitably a manual trawl, there are plenty of useful tools to help automate some of the legwork.

For a broad overview, Moz Local provides a nice snapshot of the client’s current state of affairs. The results are visual and actionable.

————————————————————————————————

This isn’t strictly a technical SEO tool (you’re more likely to use it to help implement a client’s local search campaign), however, anyone looking to provide a high detail local audit might want to show how their client’s competitors are faring in terms of citation growth. This tool by Whitespark will help on this front.

————————————————————————————————

I’ve been using BrightLocal for many years now and it is, by far, my preferred tool for local analysis. Again, many of the features are not specifically designed for the auditing phase of a client’s campaign but components such as their GMB Audit certainly are.

————————————————————————————————

International SEO tools

At the other end of the spectrum, clients targeting national key-phrases in multiple international markets may sometimes fail to implement the essential components correctly. Typically, this revolves around hreflang errors which can vary from the use of incorrect country codes to specifying language-specific content for a certain region but not then providing it in that language.

Whilst, again, crawler tools such as Screaming Frog are useful for identifying issues with hreflang, I like the visual output of Distilled’s tool.

————————————————————————————————

Very similar in nature to Distilled’s tool, the hreflang tag testing tool also enables you to crawl xml sitemaps.

————————————————————————————————

Optimisation analysis tools

Following the release of Google Hummingbird way back in 2013, many SEOs heralded the new era of natural content for websites. No more forcing exact match key-phrases into copy as Google should now be able to better understand the context and meaning of a given web page. Whilst, in theory, this should be the case, the reality is that Google still needs the webmaster’s help. Use of the target key-phrase in a site’s title tag, for example, still remains one of the most powerful on-page ranking factors. Here are some of the better tools to help grasp the standard of a client’s on-page optimisation efforts.

I really like the output of SEOptimer. As you can see from the results below, social skills are not my strong point!

————————————————————————————————

Woorank is very similar to SEOptimer although many of the features are only available to paid subscribers.

————————————————————————————————

I really like the layout of Varvy’s optimisation suggestions. It pulls out key information and visualises it simply.

————————————————————————————————

For a sophisticated suite of paid SEO analysis tools, at a very reasonable cost, Mangools should be on the demand list of every SEO Manager in the land. This beautifully designed software covers not only website analysis but keyword research, SERP analysis, rank tracking and backlink auditing. Highly recommended for on-going SEO activity.

————————————————————————————————

Content auditing and analysis tools

Whilst the “content is king” mantra has been drilled into SEOs and webmasters since before the days of Google Panda, its directive still remains at the heart of good optimisation practice. Without a high-quality, user-friendly website featuring original & engaging content; the client is unlikely to be successful in a competitive search niche without a ton of backlinks (which, in turn, is reliant on that high quality content). Whilst auditing a client’s content is chiefly a manual process (reading and assessing copy, video and other assets) there is a technical dimension to the assessment process too, that can only be carried out using tools. Here are some of the essentials.

Excessive duplicate content on any web property can potentially severely limit its search visibility. Siteliner “systematically checks your site for internal duplicate content, highlighting it on each page, intelligently excluding common content such as menus and navigation”.

————————————————————————————————

Built by the same folk who created Siteliner; Copyscape searches for copies of any given page across the web so you can check if a client has been plagiarised or vice versa.

————————————————————————————————

A great alternative to Copyscape is Plagiarism Checker Free, a very handy tool for checking if any copy supplied to a website might be taken from elsewhere on the web. They also offer insightful smart tips for avoiding plagiarism.

————————————————————————————————

Google Panda alerted webmasters to the potential deadliness of “thin content” pages, with many thousands of sites taking a huge hit to their traffic and rankings for featuring an excessive volume of pages with little or no text. Datayze’s Thin Content Checker is a useful tool for crawling the word count across a website’s pages and comparing this to the percentage of non-anchor text and unique phrases. The output can be seen below.

————————————————————————————————

Whilst generating ideas for a content marketing campaign may be outside of the scope of a typical technical SEO audit; it may help a client to point out some examples of successful content produced by competitors. Buzzsumo is great on this front as it quickly shows content that has fared well on social media. A search on their content analyzer for “seo audit tools”, for example, throws up some interesting competitor articles to this piece!

————————————————————————————————

Readable.io prominently features the lofty claims that they will “help you improve the readability of your content, increase ROI and boost sales”. I liked the free version of this tool; it’s enough to provide a brief assessment of a client’s web copy. However, the ReadablePro features look like essential tools for committed web copywriters.

————————————————————————————————

If it looks as though a client hasn’t built out content hubs via information pages, blogs and other useful resources, Answer the Public is a great tool for generating topics that can be written about to help build these hubs. Again, it could be argued that this isn’t strictly speaking a technical SEO audit tool, it’s more for content planning, but if a client has particularly weak content it’s good to flag this at the audit level; and throwing in some content suggestions adds another level of detail to the report.

————————————————————————————————

Backlink analysis tools

Backlink analysis is one of the final areas I will look at when conducting a technical SEO audit but also one of the most important. A client may have very high quality content on their site but without an authoritative backlink profile they will be unlikely to rank well for more competitive key-phrases. At the other end of the spectrum, sites that have engaged in particularly egregious, low-quality link building may struggle to rank well despite having some high-quality links (and despite the fact that Google Penguin supposedly now discounts spam links rather than penalises a site for having them). There are many backlink auditing tools on the market, here are some of the stronger ones.

Majestic’s MJ12Bot™ “scans billions of URLs a day, feeding our fresh and historic indexes with massive volumes of link data” (source). The company is well-known in the SEO industry for their web-scale data points Flow Metrics™ – Trust Flow™ and Citation Flow™.

————————————————————————————————

ahrefs.com is my preferred backlink analysis tool that can also be used for a range of other purposes including key-phrase research, content gap analysis, rank tracking and even technical auditing. The team behind the software always seem to be adding to it and innovating, plus their processing power is a cut above; their crawler processes up to 6 billion pages a day. As you can see my backlinks have been on the wane, as my website was offline for a few years. Need to turn this around!

————————————————————————————————

This software is prohibitively expensive for me but I was lucky enough to be among the first to trial LinkResearchTools when they launched it about a decade ago. Their link data is supposedly the most in-depth available with processing capabilities that are even stronger than ahrefs.com. They cite big brands such as Groupon, Booking.com and LinkedIn as customers they’ve helped to clean and improve their backlink profiles.

————————————————————————————————

If you want to take the backlink analysis section of your audit to a higher level, you could include detailed advice for your client on anchor text distribution. Ensuring the anchor text profile for a domain looks, and actually is, natural in its composition is an important element to effective SEO. However, certain brand building strategies can naturally result in too many homogeneous anchor links. OutreachMama’s extensive anchor text guide includes a cool tool for suggesting natural ways to link based on user intent.

————————————————————————————————

Social media activity analysis

A strong social media presence can be vital in helping amplify a client’s content and thereby helping to generate more backlinks to it. Social Signals occur when a piece of content gets shared many times on social channels such as Twitter and Facebook. Many studies have found a strong correlation between social activity and rankings. At the audit stage, it’s useful to highlight the strengths and weaknesses of a client’s current social media set-up. Other tools already mentioned in this report, such as SimilarWeb and ahrefs.com, are also great for this part of the audit. Here are some others you might find useful.

One of the tools I use most often at this stage is Followerwonk. Their ‘Analyze’ tool is particularly useful for assessing the authority of the users who follow a client on Twitter. A client may have many thousands of followers but if the majority of those are fake profiles, their account may be next to useless in helping to promote a new item of content.

————————————————————————————————

This is quite a cool free tool that you can use to gauge the reach and social media strength of any company, person or brand. They appear to be mixing me up with another famous Hubbard though, unfortunately.

————————————————————————————————

Summary

And that concludes my attempt to provide a definitive list of technical SEO audit tools. I’m sure there are many high-quality tools that aren’t on here which should be. If you’re a developer, SEO, web marketer or just a web enthusiast, you might have some suggestions for other tools I should add above. If so, drop me an email or leave a comment below and I’ll try to review it as soon as I can. If it makes the grade, I’ll add it to the list and credit you for it too (although please note I will only link out to other SEO or web development themed sites).