How to build an empire with SEO analytics

One of the key problems for anyone undertaking search engine optimisation is knowing where to focus your efforts. SEO is such a broad and all-encompassing discipline that there is more that could be done to any site in terms of incremental improvements than there would time to do it all. How does one know where to focus? How can you be sure you are not wasting your time on the wrong things?

Moreover, once you have implemented a change to your website, how do you know whether it is making things better or not? It is just as easy to make a change that does harm as it is to make one that improves things.

This is why search analytics is such an important part of any SEO activity. Measuring key SEO performance indicators helps to give context and insight into what is working and what is not working. That statement in itself is somewhat stating the obvious, but in what specific ways can search analytics make a difference to your business? In this post I am going to explore just that – what are the different ways in which SEO analytics can help, and what tools and process should you use to get the best insights.

What is SEO analytics?

First, it would be useful to set a definition of what we mean by search analytics or SEO analytics. I’ve used these terms interchangeably so far because people call them different names meaning the same thing. In fact, there are many other names used where people are referring to the same activity – SEO reporting, SEO attribution, organic search reporting and so on. What we are really talking about here is the use of data and insights to inform SEO strategy. From this point on, I am going to only refer to it as SEO analytics, to make things easier to follow.

What are the benefits?

Let’s look first at some of the ways we can use SEO analytics. How can data and insights be used to make a difference?

Tracking and performance

The obvious use is for understanding SEO performance and by adjusting SEO strategy based on data. This means tracking the performance of selected search queries to understand where you rank, tracking the traffic driven by search engines (and if possible, via which search queries), and most importantly, the business actions that result from traffic driven by search engines.

For example, if you sell red widgets and you are tracking the query “buy red widgets”, you want to know if your e-commerce website ranks well, and if it is improving or not. You also want to know how much traffic you might be getting for that query, and if it results in sales more so than, say, “buy scarlet widgets” or “buy crimson widgets”.

We will discuss tools later in this post, but getting some of the performance data can be tricky. Google, for example, won’t pass keyword data to analytics tools in most cases and so it can be difficult to know which keywords have driven traffic and then gone on to result in business goals such as sales. However, there are other ways to slice the data that can tell you this story which we will also look at later in this post.

Early warning system for critical errors

It is inevitable that at some point something will go wrong with your website. A typical enterprise website will utilise perhaps dozens of different web services and platforms while trying to deliver a unified experience to its users. It is only a matter of time before two services conflict, perhaps, or one fails.

Often, when this happens there is some obvious outcome that cannot be missed – the website is not reachable for example. Not good, clearly, but at least you know it is happening and can work to resolve it. Sometimes though, the errors can be more subtle and you need some way of being alerted to the problem.

Recently, I saw a project where a CMS grabbed the wrong module template within one type of page after a deployment of new code. This module looked and behaved the same for users as the correct one, but crucially included a noindex directive, essentially telling search engines to remove the pages containing it from their search results. It wasn’t across all pages, but only some, and the website contained tens of thousands of URLs.

However, by monitoring the number of indexed URLs in Google, it was possible to catch the error very quickly and remediate it immediately. Without SEO analytics running, this site might have gone for a week before anyone noticed the issue, after which time recovery would be an incredibly long and arduous process.

This is a great example of how SEO analytics has saved projects, but there are a whole range of different errors and effects that we typically monitor for, without which there could be potentially very serious problems introduced.

The kind of things that are easy to monitor and could save your SEO bacon include: monitoring 4XX errors, monitoring server errors, monitoring soft 404s, monitoring site speed and response time, and undertaking test crawls.

Tracking what work has been done

Keeping a record of what changes have been made to the website and what tactical activities have been actioned is an important part of the analytics process. By keeping this record, it is possible to review the impact of changes and of tactics to build a picture of what is helping and what is hindering.

This is particularly important on those occasions that performance takes a backward turn, which it will inevitably do from time to time. The record of changes and tactics enables you to decide whether to reverse some changes, or to change tactics.

Sometimes, there is no obvious link between changes in performance and a change to the site or tactic you have employed. On these occasions, you might need to see if Google has changed something.

Tracking algorithm and SERPs changes

Where you can’t link a change in performance to something you have done, sometimes it might relate to a change made by one of the search engines, and in the UK, if the change in performance is noticeable, it is likely such a change would be in Google.

There are two kinds of change that might impact performance. The first is a change in the algorithms used by search engines to rank and categorise websites. These are sometimes updated to promote or penalise websites that match certain criteria. For example, Google tries to present its users with sites which provide a good user experience, and sometimes it updates its algorithms in ways that promote this aim.

The other kind of change that can impact your performance is a change in the way search engine results are displayed, which can mean that without climbing or falling in the rankings, the traffic you receive from search results might be impacted. An example of this might be if Google were to add a knowledge panel or quick answers to a search result you rank for.

Understanding how authoritative you are

Search engines try to provide the best experience they can to their users, and this means providing search results from websites that they feel are trustworthy. One of the core indicators for search engines about how trustworthy your site is understood to be is authority metrics, which look at the quantity and quality of links to it.

Tracking these authority metrics is important in order to understand the success or otherwise of work you are doing in order to been seen as more authoritative.

Product and service expansion

The dimensions and metrics tracked by SEO analytics can also be useful in identifying opportunities to expand your service or product range. If you find users are landing on your website while looking for a product or service that you don’t currently offer, you can decide if you have the capability to also offer that product or service.

Understanding industry trends

SEO analytics is also useful in providing information about industry trends that can help you position brand or product content differently to capitalise on such opportunities.

For example, you might notice that particular terminology for your product or service is becoming more mainstream, or that some kinds of product variant might be growing in popularity. Such insight can help you shift focus of effort to actively promoting and optimising these products and terms in order to align with search trends.

Monitoring competitor SWOTs

It is important not to stay blinkered by watching only the performance of your own website. Understanding the performance of you competitors can also identify opportunities for you to move into areas in which they are weak, or to protect yourself in areas in which they are strong.

Bear in mind that it is probable that your competitors are also actively optimising SEO, and so you need to be aware of where they are making progress or where they are blocking your progress.

Tools and methods

As you can see, there are extensive benefits to SEO analytics, but such benefits rely on a broad range of different data points. How is it possible to access these data points, and how can one bring them all together usefully?

There are many options that can be incorporated into your toolkit. Here I will explore some of those, although bear in mind there are hundreds of different tools and suppliers in the SEO ecosystem. You need to pick the right toolkit for your own needs.

Manual keyword tracking

Understanding how well your website ranks for important search queries is a crucial part of measuring your SEO performance. One way you can do this is by doing the search yourself in Google.

The problems with this are many, though. Critically, it is very time consuming, unless you genuinely are only interested in a small handful of search queries.

Additionally, undertaking a manual search can impact the search results themselves, especially if you do the search frequently – the context of your search, such as the location you are in when you search, the time of day, how often you click the search result itself and many other factors might mean you see different results to everyone else. You may even impact the potential of your website to rank well for such searches, although there is little evidence that Google uses things like bounce rate as direct ranking factors.

Furthermore, if you undertake many searches, you may trigger some of Google’s “overuse” filtering, which can prevent you (or anyone else on your IP address) from doing a Google search easily for a few hours.

In short, manual searches are more useful for ad hoc analyses, such as investigating SERPs features such as the knowledge panel, rather than for rank tracking. It is usually better to use some kind of rank tracking software or service.

Cloud-based keyword tracking

There are a number of cloud based rank tracking services available, including those such as Moz Pro, SEMRush, Advanced Web Ranking and many others.

These services collect SERPs data periodically into their own data warehouses and collect reports for the keywords you store in your account with them. Usually, they have a good record of historical keyword data.

As these services do not collect data via your IP address, they will not collect data that is personalised to your context. Nor are they likely to impact ranking through their use.

Additionally, they won’t cause Google to block your IP address for overuse, which I can tell you is a great benefit when you work in an office with dozens of web developers.

Client-side keyword tracking

An alternative approach to cloud-based keyword tracking and manual tracking is to use rank tracking software, of which there are many varieties, such as SEO PowerSuite or Rank Tracker.

The advantage of these kinds of software is that they usually have no real limitation to how many keywords you can track in a project. However, the fact that your IP address will almost certainly be blocked by Google puts some practical limitations in place, especially when the web developers in your office keep asking you why Google is asking them if they are a robot before answering their questions. That gets old quickly. Client-side rank tracking software is explicitly against Google’s terms of service, and this is why Google will block IP addresses when it sees too many searches occurring too quickly from one IP address.

I have found client-side keyword tracking to be helpful in the past, but it has very few benefits over cloud based systems, and very many drawbacks. My recommendation is always to steer clear of this kind of tracking.

Google Search Console and Bing Webmaster Tools

Another source of ranking data can be found within Google Search Console and Bing Webmaster Tools. Both of these resources can tell you where you website appeared in search results and for which keywords, including data about how often the search results are clicked.

The downside to the data in these two tools is that they will only show you data for queries for which you have appeared in search at least once. Therefore, they don’t give the best view of keywords that you might want to consider in order to broaden your keyword profile. That is to say, you don’t see what you want to rank for unless you already do.

However, the queries information in both these tools is incredibly valuable. It is useful for identifying opportunities to nudge rankings up across terms where you get good click-through rates, or to look to optimise click-through rates where rank is already high. This is incredibly useful tactical data.

Moreover, the queries reports are not the only data you get from these tools. Both provide valuable information about how search engine crawlers understand your site, such as what errors they find, how often they crawl, whether they can parse your sitemap or robots.txt file and so on. All of this is important to know and act on, and therefore these two tools are very important.

Google Analytics

Google Analytics provides excellent traffic data about your website, including categorising traffic based on whether it has come from an organic search or through some other channel (traffic from paid search ads is a separate channel in this sense).

This allows you to track the volume of traffic driven by search, and thereby draw conclusions about the effectiveness of your SEO activity.

However, don’t expect much useful keyword data here. Google blocks the query used for most searches meaning that typically over 90% of the search traffic is recorded as using the query “not provided”.

What is more useful, though is the data around which landing pages are successful through SEO. Comparing this data to that recorded through tools such as Google Search Console can provide valuable insight into where to focus optimisation efforts.

Website crawl

While the services listed above give some valuable insight into how your website has been found, and by how it is crawled by Google and Bing, there is no substitute for the rich detail that can be gathered by running your own crawlers on your own website.

There are a number of tools that can be used to help you do this, including Screaming Frog, Xenu and others, each giving their own benefits.

Crawling the website yourself helps to identify where key information might be missing (e.g. are there pages with no TITLE tag), where there a barriers to crawling – can your own crawl find all of your site, and a numerous list of other issues.

Crawls like these are important, and are easily overlooked as part of search analytics. Being able to count the issues you find, and prioritise them is a critical measure for SEO process. New issues are inevitably introduced to websites all the time as they grow, evolve and launch new content. A critical objective of all websites should be to reduce or eliminate crawl errors.

Data Studio

Google Data Studio is not in itself a measurement tool. It is a data visualisation tool. Where you have a requirement to report data to someone else, such as senior management, Data Studio allows you to do this in a consistent way.

Data can be imported or there are ways to link to live data such that reports and dashboards can be updated daily.

A key part of data analytics and insight is to have consistency in reporting – looking at the progress of the same data in the same context over time. Data Studio makes this much easier.

DIY search analytics

DIY search analytics is not difficult in principle. You need three things to do it successfully:

1. Tools to gather and report the data
2. Knowledge and experience to understand the data
3. Time and resources to analyse and derive insights from the data

Without all three of these, it is difficult to be able to do anything useful with search analytics.

If you do have all three of these, though, search analytics is not especially tricky. A how-to blog post for all the tools above would take too long to include here (and I guess might be the subject of another blog post later).

The opportunity costs of DIY search analytics are primarily related to your time (although some of the tools you will use will have some level of cost involved). You need to take the take to install the tools and set them up, the time to learn the tool interfaces, the time to learn how to interpret the data, the time to apply your knowledge and experience and derive insights, and the time to turn these insights into actions and recommendations.

The cost of DIY search analytics is, therefore, not the simply the financial commitment to the tools, but the commitment to the time it takes to use them. Even as seasoned professionals at Storm, we might spend a day or two a month per client just on search analytics, and this time is longer on new projects or on more complex ecommerce or transaction driven websites that’s almost a whole working month of every year. Search analytics is that important.

How can your agency help?

If you don’t have the time to commit to doing search analytics yourself, it doesn’t make it any less important. Your SEO agency exists to carry out this service for you.

That working month of your own time is almost certainly better invested in managing and growing your business. Let your agency take the strain on this kind of activity. Not only will you get your time back, but you will also get actions and recommendations based on years of experience in SEO and e-commerce.

If you need help

Getting the best from SEO analytics is not tricky, but on your own it can become overwhelming. To find out how Storm can help you capitalise on this vital activity, why not get in touch?