How and What to Test Effectively

Reading Results (and Economics)

Do the results appear to be logical? Are they believable? You’ll need at least a 5-percent variance, or difference, in the revenue per catalog mailed for the results to be real. A variance of less than 5 percent may not be statistically valid.

So what do you do? Re-test. If the difference is slight the second time around, perhaps you should stick with your control. When reading the results, factor in any cost differences. Exclude the costs associated with the test itself. Include only the costs associated with the rollout itself, should you decide to do so.