Testing is critical for any digital marketing program. For most digital marketing channels we hold a great deal of control over the variables and can test any one or combination of variables in an attempt to drive a specific outcome. For example, in email we can test the audience, subject line, creative, copy, CTA, or landing page. But what about testing in channels that we have less control over?

For SEO, we can’t call up Sundar Pichai and suggest tweaks to the algorithm that might benefit us. Although has anyone really tried? So then how can we test SEO? well, figure out the variables that are in our control and devise a plan to manipulate them. There are specific things we encode in our web pages that tell Google exactly what to display for our website in the search results. These things are called meta tags and are fully within our control – meaning we can change the copy at-will and Google will reindex the change to render it in the search results.

Here’s what we tell Google via your site’s meta tags – these are Adobe’s…

Here’s how Google then displays that information in the search results…

There are two types of A/B tests that we can consider running on SEO – neither is perfect science for the testing snobs out there – and the choice about which to use depends on the kind of business you have and pages you want to rank for in search.

Option 1: For most ecommerce sites, or sites that have inventory, test your search results pages

Consider sites like Zappos, Chewy, or Indeed. These properties have many pages for which they want to rank. In the case of Zappos, those pages may include mens sneakers, womens sneakers, Nike tennis shoes, or sandals. Since Zappos would likely have the same general structure for their meta tags – title and description – across all these pages, they could setup a test by keeping some tags in format A and change another set of tags to format B and see how the organic search traffic differs from both sets.

Considerations

• Identify and group pages together to try to develop a representative sample. For instance, you shouldn’t group all variations of mens’ (mens shoes, mens sandals, mens socks, mens boots, etc.) together in your control and the analogous product set for womens’ as your experiment. This wouldn’t be representative of the larger population and introduces a sample bias.  

Advantages

• Can run tests in parallel to eliminate any time-related variables
• Can reach statistical significance more quickly by grouping pages together to generate a larger sample size

Disadvantages

• Securing tech resources can be challenging and for these types of updates, you’ll likely need the support of your tech team

Option 2: If your site doesn’t have inventory or search results, test for your high value pages – the ones your care about ranking in search

If you live in the B2B world, this is probably your better option. Or on the B2C side, if you don’t sell your product or service through your website, you won’t have a set of search results you care to rank for – rather, you probably care more about relevant terms.

Take Adobe, for example. Presumably Photoshop is a valuable product for them, seeing as the product name has turned into a verb for most people. So ranking for that page is really important to Adobe. Assuming for a moment that they get enough traffic to the Photoshop page to make the timeline for a test reasonable, Adobe would have to sequentially test its meta tags – meaning they would update the copy in their tags and do a pre/post analysis on performance since all of their traffic would see the new copy in the search results in Google after Adobe made the change for the Photoshop page.

Considerations

• Find pages with enough traffic that your test won’t take three years to complete

Advantages

• Can test pages that don’t have analogous sets of pages to test against

Disadvantages

• Sequential testing will be impacted by changes over time and those changes cannot be controlled by the structure of the test. For example, maybe a major news story broke about Photoshop. This could result in an unnatural level of search traffic for Photoshop, and as a result, the sample becomes categorically different and invalidates the test.

Full disclosure, I hadn’t even considered this until I read this brilliant blog post from the team at Esty about how they tested their title tags for SEO. It got my gears turning that we can and should treat SEO like other marketing channels and test and learn to improve performance. It’s a great read and goes into a lot more technical detail.

Have you experimented with testing for SEO? What have you tested? How has it worked?