Sell More Via the Web

Some online retailers seeking to improve their sites can afford substantial investments in sophisticated analysis tools and costly usability consulting. Others can’t. This article aims to help anyone who wants to tune a Web site “on the cheap.”

Step 1: Decide What Needs Improvement on Your Site

After listening to comments from customers and employees, you may already know some of the trouble spots on your Web site. You also may note weaknesses on your site after surfing and shopping from your competitors. And you may even use Neilsen’s law of Web usability, which states — quite obviously — your visitors spend most of their Web time not on your site. Therefore, you must pay special attention to instances in which your site uses navigation or layouts different from what most other online merchants are using.

As a rule of thumb, the four most important components of an e-commerce site are the product page, site search, cart and homepage. Running “discount usability sessions” is an inexpensive way to finding possible improvements to those important components of your site. Discount usability engineering (a phrase coined by Web guru Jakob Neilsen whom I mentioned earlier) is a great way to determine what on your site trips up users. Read more on this method at Neilsen’s site, www.useit.com. Here’s a quick overview.

Recruit five local Web users who aren’t familiar with your site. Pick folks who resemble your target demographic. Schedule five 45-minute sessions on the hour, one for each user. Offer each user a modest thank-you gift (e.g., gift certificate, cash) for participating.

Pick a friendly person in your organization who’s comfortable with the Web to serve as your moderator. Takeaway Tip: Don’t pick someone who works on your site.

Equip a quiet, private room with a computer and a video camera. Takeaway Tip: When running usability sessions, don’t use your T3 connection and Web designer’s gigantic display. To mimic reality, users should experience your site on a dialup or slow DSL using a regular-sized screen.

Have the moderator welcome each user and introduce the exercise. Let the participants know that you’re watching people use your site so you can make it better. The moderator should tell them he or she doesn’t work on the site, so they won’t feel uncomfortable sharing anything about the site they find confusing. Tell participants they’re being videotaped.

The moderator then asks users to complete a task on the site, sharing their thoughts as they go. Have users search for, select and purchase products, and ask them to share their thoughts and expectations as they click buttons and navigate the site.

The moderator should take notes, detailing where users stumbled and when the site didn’t do what was expected. The moderator must encourage users to keep sharing their thoughts, but the moderator shouldn’t lead the users in their Web browsing.

The goal of discount usability sessions isn’t quantitative statistics. Rather, you’re seeking trends and “Aha!” insights. Such insights needn’t be earthshaking, just places where many of your users hesitate or stumble. Here are examples of insights seen in different usability sessions:

1. On the “view cart page,” below the table showing cart contents, there were three buttons, which read, from left to right, “Checkout,” “Continue Shopping” and “Empty Cart.” In testing, multiple users instinctively hit the right-most button to continue shopping, but they unintentionally purged their carts.

2. The product hierarchy included “Sale Specials” as a product category. As a result, once a certain dress or top was placed in the sale area, it no longer showed up under the women’s category. Users expected to find sale items alongside the other dresses, and thus couldn’t find them.

3. The product page displayed a grid of color swatches below the large item image. When there were more than three color options, the grid pushed the copy block, the price and availability below the fold on a standard browser.

Step 2: Find a Better Alternative

Once you’ve identified possible site problems, potential solutions often become clear. For example, for the site problems we identified above:

1. Remove the “Empty Cart” button all together. Move “Checkout” to the right-most spot, and make “Checkout” more prominent.

2. Be sure sale items also display in their original product category.

3. Move the copy block up to the right of the image, and move the grid below the image. This way, price and copy always are visible without scrolling.

Because testing is expensive, be sure to test only those changes significant enough to matter. For instance, testing if your navigation should be blue or green probably isn’t worth it. But testing a single hero product homepage treatment vs. a top-selling product grid treatment probably is worth it.

Step 3: Run an A/B Test Via the Day-swapping Method

Larger retailers can build or buy expensive software to run multiple versions of their sites simultaneously. Such packages randomly assign visitors to the control or the alternative treatment; ensure a visitor experiences the same treatment consistently throughout the visit (and on subsequent visits); accumulate data for analysis; and report on results.

With a little work, a smaller merchant can gain much of this benefit without needing the costly software via a “day-swapping” test. Here’s how.

Build two versions of your page in different files. For example, for testing your homepage, you’d create index-control.html (a straight copy of your current homepage, index.html) and index-alternative.html (a new page implementing a new idea.)

Create simple scripts to copy one of the versions onto the live page. For example, if you were doing a homepage test on a Unix or Linux platform, the two trivial bash scripts noted in the chart “Bash Scripts” would perform the copy (after adjusting filenames to match your server configuration).

Monday at 9 a.m., run the “make-control-active” script. Tuesday at 9 a.m., run the “make-alternative-active” script. Repeat this process for two weeks, alternating each day. After 14 days, collect daily metrics for the period, such as site sales by day and average pages viewed by visitor by day. Create a spreadsheet for each metric by day of the week, noting how many times the alternative beats the control. See the chart “Site Sales” for an example.

As a rule of thumb, if the alternative beats the control six or seven days out of the seven-day pairs, it’s significantly better. If the results are reversed, the alternative is significantly worse. This simple design sidesteps:

1. day-of-week effects due to the matched design;

2. season sales trends due to the alternation; and

3. outliers, because this non-parametric test doesn’t place undue emphasis on the occasional huge order.

This design works best for merchants who close a large portion of their sales on a customer’s first visit, rather than for merchants who typically experience long delays between visit and order.

For more statistical power, run the test longer. If you ran the test for four weeks, you’d still alternate day by day. After four weeks, you’d claim the alternative the winner if it beat the control on 12, 13 or 14 of the 14 day-pairs, and claim the alternative a loser if it beat the control only zero, one or two days of the 14.

You might feel these cutoffs are too rigorous, and that it’ll be hard to find a new alternative to perform so strongly. Yes, it will be hard. You’ll find that many of your tests yield indifferent results, with the alternative neither worse nor better than the control. That’s the reality of trying to improve an already reasonably good Web site.

Conclusion

You work hard to get visitors to your site. Using the one-two punch of discount usability sessions followed by A/B testing, with some luck and a good deal of patience you can increase site conversion. Now’s the ideal time to start, so you can have your Web site in tip-top shape for the holidays.

Alan Rimm-Kaufman, Ph.D., leads the Rimm-Kaufman Group, a services and consulting firm helping catalogers and e-tailers profitably grow their Web businesses. He can be reached via his Web site at www.rimmkaufman.com.