Mysterious & Bizarre Results

Cataloging usually is a predictable world. We track everything, study it, place data in spreadsheets and end up knowing pretty well in advance how things will work out.

In fact, if businesses had human personalities, cataloging would be your Aunt Matilda and Uncle Gus: safe, predictable, no surprises.

But then there comes a day when Uncle Gus calls to say he has flown to Rio with his secretary, and Aunt Matilda has joined the circus. What do you do when the predictable becomes, well, unpredictable?

The Mystery of the Rotten Rollout

After several years of slow growth, the mid-sized niche cataloger decided he needed (and could afford) dramatically faster growth. After devoting a year to extensive list testing, with good coding and tracking of results, he confidently rolled out in a big way to the winning lists.

But the results stunned him. There seemed to be no correlation between the results he’d seen in his tests and those coming from his rollouts. Lists that were strong in his tests were weak and vice versa. There was no pattern; results seemed random. Had the world turned upside down? Did testing mean nothing?

In this type of situation, the first challenge is dealing with the anger and suspicion it produces. When a cataloger has done his or her homework and set up and followed good systems, being rewarded with grim results can produce hostile feelings about every vendor in the chain, from the list broker to the post office.

And this cataloger had done many things right. His catalog software was working properly. His operators were asking for and properly entering customer keycodes. His list orders had been correctly written, fulfilled and mailed. And he was fulfilling orders from in-stock merchandise, so the culprit wasn’t a failure to fulfill or a high return rate.

Unfortunately, this cataloger didn’t worry about statistical significance during testing. In his haste to achieve fast growth, he maximized the number of lists he could test by renting too few names from each list.

How few names are too few? Rule of thumb: If your results spreadsheet doesn’t show at least 50 orders from an individual group, then you don’t have statistical significance for that group, and you can’t count on the results from the group being repeatable.

In this cataloger’s case, order counts for various test groups had been between 10 and 30 orders — enough to persuade the cataloger the trends were real, but not enough to satisfy a statistician.

In the end, the statisticians were right: The results weren’t repeatable. When the cataloger rolled out to what he thought were his winners, he actually was rolling out to a purely random group of lists, which ended up producing poor results — like any random group of lists would.

Takeaway tip: Believe the statisticians; build your list tests to produce at least 50 orders per cell, and don’t roll out with less.

The Mystery of the Shrinking Housefile

A cataloger’s housefile suddenly began shrinking. This cataloger had been in business for many years, and during those years all his housefile segments had performed as expected.

The cataloger segmented his file by recency, frequency, lifetime monetary (RFM) value. And each year his older segments would shrink a bit, which is normal.

But one year his list began shrinking across the board. He didn’t notice at first, because the initial effect was small. But within a year, it was obvious that something bizarre was shrinking all segments in his housefile. Customers simply were disappearing from the file.

Had the laws of mathematics changed? Was some mysterious disease eating through his disk drives?

At first, this seemed like a fairly easy mystery to solve, because we knew so many things it couldn’t be. For instance, it couldn’t be telemarketing, fulfillment or mailing, because the effect was confined to his database. It had to be a computer problem, so how hard could it be to track down?

Turns out it was extremely difficult. Everyone’s first thought was a computer virus, but that wasn’t it. Nor was it a bug in his software or a corruption in his software installation. Everything seemed fine.

The ultimate solution lay in his customer file itself. All computers these days are “PC-compatible,” but there’s one little-known aspect about them. By holding down the “alt” key, typing any number between one and 255 into the numeric keypad, and then releasing the “alt” key, you can generate any keystroke your keyboard is capable of producing. (These are called “shortcut keys.”)

For example, if you hold down “alt,” type in 65, then release “alt,” a capital “A” appears on your screen. It seems harmless, but it’s not if you type certain other numbers. One number in particular if typed, will indicate “end of file.” If your operator inadvertently presses “alt” and types that number on the keypad (easy to do these days, where “alt” triggers Windows menus, and the keypad is used by many operators to enter digits), this invisible “end of file” character will be entered directly into your database. And almost every database in the world will recognize that as “end of file,” and will chop your database right there. It’s as if you had an invisible stop sign in the middle of your customer file.

Takeaway tip: Train operators not to use the “alt” or “control” keys when they’re typing normal characters, especially if they use the numeric keypad for entering digits. And if your data file suddenly seems short, have a specialist filter it to find invalid hidden characters.

The Mystery of the Missing Phone Lines

This longtime cataloger’s product line was solid, its customer base enthusiastic, its mailing plan massive and its catalog presentation strong.

And yet as the days and weeks of the season passed, sales kept trending short of expectations. The in-house call center was functioning well, and all the call monitoring statistics were good. There were few abandons, short on-hold times and no saturation of the incoming lines. Still, sales stayed low.

Had the cataloger’s products suddenly lost their appeal? Were customers suddenly migrating to competitors?

Depressed sales are an especially tough mystery to solve, because they can be caused by so many things. In this case, the problem was compounded by the fact that the cataloger was doing everything well — wonderful mailing plan, seed catalogs reporting in like clockwork, enthusiastic customers on the phone, well-trained call center operators, and close monitoring at all levels by expert and motivated executive staffers.

But week after week, sales stayed low. If it continued for long, it would have a profound impact on this cataloger’s finances.

What happened? While preparing for the busy holiday season, the cataloger installed many incoming lines to handle the 800-number volume. Unfortunately, the phone vendor didn’t correctly hook up about half of the lines.

Throughout the entire first three quarters of the year when sales volume normally was modest, nobody noticed anything amiss, because sales were meeting expectations. Only when the holiday season arrived and sales failed to jump far enough did the effect get noticed. Yet no one thought of the phone lines because they’d been installed so long ago and had been working in what appeared to be a normal fashion up until that time.

Takeaway tip: As telephone solutions become more complex, be suspicious of possible problems. Assign staffers to intentionally call in simultaneously to saturate your incoming lines. Confirm that you have as many incoming lines as you think you have, and that they’re all functioning at maximum volume. And of course, do this before the holiday season begins.

Susan J. McIntyre is president of McIntyre Direct, a full-service catalog agency and consulting firm based in Portland, Ore. She can be reached at (503) 286-1400.