Don’t Test Whispers

Key to marketing success is a disciplined approach to testing ideas and action. After all, marketing writing and consulting is filled with tremendously attractive and detailed theories about action “X” causing result “Y”. Yet all these theories were developed based on specific experiences under specific circumstances. So there’s no guarantee that taking them and applying them in your world will create the same result.

So we should test, test test. And yet…testing experience shows that far more things are tested than are found to conclusively help or hurt. Why? One quite common testing error is to “test whispers” – small changes that simply can’t have a large enough impact to drive measurable change.

I once watched Rubbermaid test whispers in focus groups where a series of 5 statements of brand differentiation were evaluated. But rather than vary the statements with ideas that were truly significant to consumers, the statements traded off tiny wording changes. (I found it ironically enjoyable to watch the focus group participants quite frankly explain that all the statements said the same thing.)

Focusing on testing important things is even more critical in retail markets. In part, a mature market is extraordinarily noisy. And changes in the retail world are never isolated enough to be perfectly separated from other actions – often actions we might not even be aware of like promotional choices made by a local retail store manager.

In part, the statistical variations in normal day-to-day activity are quite often far larger than the size of change we’re hoping to test. We hired a statistician to evaluate some results a decade ago or so for a tool client who sold through Sears. The size of change that retail merchandising efforts produced (e.g. appearing in the Sunday circulars) far outpaced any of the changes we’d see from advertising spending (which was a long-term & constant effort). The statistician identified impact from the advertising, but in order to damp out the impact of the merchandising efforts he could say no more than “there was impact” because he had lost all valid ability to quantify the size of the impact. (Reminded me of my qualitative vs. quantitative chemistry work in school…)

One might think testing becomes far different when we shift to direct response marketing. It doesn’t. Even in DR one has to be careful to test actions that are significant enough to generate a change you can measure with confidence.

So read on in my direct response television based article from this month’s Response Magazine. (Link here.)

Testing offers tremendous organizational advantage – especially responding to political attack with knowledge gained from testing. So embrace testing. And focus your testing efforts where you can successfully evaluate the test results.

Copyright 2013 – Doug Garnett – All Rights Reserved

About Doug Garnett
Doug Garnett is an expert introducing innovative consumer products and services to market while driving higher return on innovation investment. His career has been spent in innovation and he is the president of Protonik, LLC - an innovation consultancy focused on marketing and innovation. Prior to founding Protonik, he was founder and CEO of ad agency Atomic Direct.

2 Responses to Don’t Test Whispers

  1. richard hren says:

    agree wholeheartedly. while I have often been surprised by the impact of “trivial” changes, these effects are usually few and far between. the testing of tweaks should be done only after you have exhausted the evaluation of the big hitters that will have a more profound effect on your business or organization.

    • Doug Garnett says:

      You make a good point. There are those surprises where a small change makes a big difference. In my experience, though, those turn out to have been big differences in that they solve a massive consumer issue – just one that wasn’t known at the time.

      For example, I was asked to review a failed 1/2 hour infomercial for fitness watch called the Mio watch. What was missing? They never told us whether it was a watch… Focused on all those cool extras but forgot the fundamental basics that are key to purchase.

      For my agency’s part, over the past 20 years we’ve built a methodical research based strategy process up front that identifies those kinds of issues so that our execution deals with them. It’s been a very long time since I’ve dealt with one of our own ads where a little thing could throw it off track.

      Guess what I’m suggesting is that if we do our homework, it makes whispers even less likely to cause change…because the fundamental work is more solid and more successful to begin with.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: