Test Less

This is going to be controversial. This point of view may not sit well. But hear it out. When it comes to testing – quality over quantity. Test less.

Testing – whether that be onsite for things like color, copy, or user flows or offsite through marketing programs like email, paid search, or display – is a critical part of marketing success. Technology has significantly lowered the barrier to test various treatments across our digital marketing efforts to understand which has the greatest impact to drive the intended action. This gives us the ability to “test into” decisions. No more HIPPOs or hunches. It’s all about the data, baby.

But it’s a slippery slope. Because testing has become so easy, we introduce new risks. There are two key reasons I’d suggest reducing the number of tests you run.

Inability to identify causation

If you don’t know the difference between correlation and causation – read up on it here. The challenge with overloading your testing schedule is that it becomes harder and harder as you introduce more variables to (aka more tests) to know exactly which variable is impacting performance. There are only so many treatments you can introduce together into a test segment and reasonably discern which one of them is responsible for causing a change in performance.

Testing stuff that doesn’t matter

Testing for the sake of testing. If it doesn’t have a purpose or goal, should you really be testing it? Simply put: no. There’s a limited pool of resources you have to work with – both the internal resources to design tests as well as the pool of users to test against. Be smart and deliberate about what you choose to test. Pick the things that will have a measurable impact on your KPIs. Develop a plan to test treatments that have the chance to yield the most value to the business and prioritize them. Forgo tests that don’t have a clear business purpose.

How does your business approach testing? Do you have a good handle on testing or are things getting out of hand?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.