Three Mini-Rules of Thumb

2. “It’s all luck.” This explanation says that all lists are actually about average, so any unusually good (or bad) performance by a specific list on a specific mailing is mostly just a fluke, which on subsequent mailings will go away, letting the list’s true averageness shine through.

Of these two explanations, the second sounds more plausible to me, and has the added advantage of being scientifically testable. It also implies the converse: that a list that performs badly on a test ought to perform better on rollout, and that’s a testable hypothesis. However, since no real cataloger is going to actually roll out to a bad list just to find out if it improves compared with its test, we’ll probably never know for sure whether this is why lists perform more weakly on rollout than on test. But they do.