Merge it and Purge it!

In a direct marketing business, a customer list (i.e., housefile) is your most valuable asset. Therefore, it's important to manage your housefile and to keep it segmented and always up-to-date.

To keep your housefile current, you need to merge/purge it regularly. What does this mean? Merge/purge is a process that identifies and removes unwanted or duplicate names and combines those records into one record for every customer on your list. However, it's more than a deduping process, which I will discuss later.

Eliminating duplicate records on your housefile when mailing to outside prospect names is important for several reasons, mostly financial. Mailing duplicate names is expensive. It can be avoided by running a merge/purge program. This is one of the services offered by a computer service bureau. How you instruct your service bureau to run the merge can affect your mailing results. How to assign list priorities, treat multibuyers, deal with family groups and more are all important to the end result.

There are two types of duplicates that a merge identifies, which are as follows:

1. Duplicates between files, which are known as Inter File Duplicates.
2. Duplicates within a file are known as Intra File Duplicates.

When there are duplicates on the file, the merge can assign the surviving record randomly across all lists or based on a defined priority by list. In other words, the merge assigns which list will retain the record that appears on more than one list.

The three main options for the different levels a merge can be run are (1) one per household, (2) one per address and (3) one per individual. I also want to mention that business-to-business merges have an additional level of complexity.

While the merge assigns the surviving record, it also creates a file of the duplicated records for future mailings. These are called multibuyers. They are names that have “hit” on one or more prospect lists. Normally, they're categorized by how many times the record appears in the merge (i.e., 2x, 3x and more). A two-time multi, for example, is a record that appeared in the merge twice. It was mailed for the first time under the list where it was assigned, and it can be mailed a second time in a future mailing for no additional list rental expense. Multis are only captured for outside lists since there's NO benefit to pulling housefile names out of their RFM (recency, frequency, and monetary value) cells.

The surviving multibuyer information is also retained in the merge. When the merge assigns duplicates based on random allocation or on a prioritized allocation, the duplicate names that are retained are tagged as multibuyers. Typically, the merge report identifies “singles” that appear in the merge only once, multis that appear in the merge more than once (and were retained on that list), and “output,” which is the combination of the two.

Family groups are identified so that “same” lists do not falsely create a multibuyer. For example, if you take two different selects from the same list owner, they should be unduplicated before they're sent to your service bureau. If they're not, the duplicates between the files could be considered a multibuyer in the merge. In this example, they're not multis but are, instead, the same record being pulled twice. By identifying these in the same family group, the merge would consider these intra file duplicates, which can be deducted from your list rental invoice.

Always put your housefile (i.e., your customers) in the top priority with the best performing RFM groups at the top of the priority and the lower performing RFM cells at the bottom of the priority. If your file has gone through an update, there will be very few duplicates. However, it's still important that any duplicates found are associated with the “best” possible RFM cells. There are times when you will only mail or re-mail a portion of your housefile. If the duplicates within the housefile were random, you might drop a record that's from the higher RFM priority. By prioritizing your housefile, you'll be sure to mail the best performers. Assign your housefile buyers to the same family groups. Therefore, duplicates within the housefile will be considered intra file duplicates. By doing so, you will know what multis credited to the housefile will have also appeared on another list (other than the housefile).

If you're not mailing all the housefile records you put in the merge, make sure you don't drop what has been identified as multis. These are primarily records that came from an outside list that hit to your housefile and they will perform well.

Unfortunately for “direct” companies, your housefile is an off-balance sheet asset that has no financing or borrowing leverage with lenders. Banks and other lending institutions don’t have a clue about the value of the housefile. As a direct marketer it's important for you to understand and to appreciate the value of your housefile.

Your merge should be used to maximize your mailing, not just to remove duplicate records. By considering how you would like duplicates and priorities assigned prior to the merge, you'll get the most out of your merge.

Stephen R. Lett is the Founder and Chairman of Lett Direct, Inc., a catalog consulting firm specializing in circulation planning, forecasting, digital marketing, and analysis since 1995.