A Rookie’s Guide to A/B Testing for M-Commerce

Mobile commerce (m-commerce) has grown 42 percent annually for the past four years. Consumers are becoming comfortable transacting on their phones, and more retailers are investing in mobile to woo these small-screen shoppers. As the market grows, however, so too does the competition for thumb space and user retention.

To compete, brands need to win over users quickly, and A/B testing can help. Focus on these three areas of the user experience when getting started with m-commerce experiments:

• onboarding;
• activation; and
• retention.

Onboarding
A great first impression is critical for users to adopt your app. It’s so important that, at any given time, more than 50 percent of our customers’ experiments are focused on improving onboarding. This could involve testing the content and imagery featured on the first few screens of an app, or the process of getting a user to submit information. Whatever you choose to test, the goal is the same — help users experience the magical moment of making a purchase as quickly as possible.

Let’s take Frank & Oak for example. The men’s fashion company sets itself apart by creating a custom shopping experience tailored to each individual’s interests and behaviors. In order to get a personalized experience, however, users first need to sign up on Frank & Oak’s app.

As a result, Frank & Oak started its A/B testing initiatives by tweaking the sign-up process. It first tried changing the field forms and adding the ability to sign up via Facebook. After this first test, Frank & Oak tested to see whether an additional option to log in with Google would increase signups. It turns out that adding a “connect with Google” button increased mobile signups by 150 percent.

While Frank & Oak saw impressive results from a few simple tests, you may need to go through more iterations before finding a winning strategy. Other onboarding tests that you can run include log-in funnels, information collection, text and imagery, and sign-up options. You can see the final variation here.

Activation
Once users get through the onboarding process, it’s time to activate them. In commerce, this means getting them to purchase something as quickly as possible. To have a shot, our research shows you’ll need to get them to buy within the first two sessions. Otherwise, it’s unlikely they'll ever come back.

For example, when Karmaloop analyzed its customer activation data, it noticed that if a first-time user placed an item in their Wish List, they were much less likely to complete the purchase than if they had added an item to their cart. Once Karmaloop identified this trend, it set up a test to de-emphasize the Wish List button in the user interface.

Discouraging users to interact with its Wish List enabled Karmaloop to better capture purchase intent. This simple test led to more activations and increased sales by 35 percent.

What can you learn from Karmaloop's experiment? When looking to improve your customer activation, challenge the status quo; don’t assume that a current feature is achieving your goals. Instead, test your assumptions and use funnel analytics to help brainstorm ideas.

Retention
You pushed customers through onboarding and activation, and they bought something. Woot! But you’re not done. The last piece of the puzzle is retention. How do you turn customers into repeat buyers?

Hosting targeted sales on a regular basis is an effective retention strategy. For example, Rue La La features a “What’s Hot” section in its app specifically to develop scarcity and drive repeat purchases. Other apps garner customer loyalty through targeted notification campaigns, whether via email or push.

In the Rue La La example, it could easily test the effectiveness of the “What’s Hot” section on user retention by setting up a test where “What’s Hot” is replaced with another section of the app or taken out altogether. Rue La La could then analyze if users who saw the section came back and purchased more often. Other ideas for testing retention include last-chance selection, trending lists, adding discounts or promos for in-app purchases, and changing photography to make items more appealing.

Make Changes for Good
The tests discussed here offer ideas to get you started, but to be successful with A/B testing, you’ll have to continually challenge the norm and make sure your team is invested in listening to and acting on the results. Some of the biggest improvements can come from the smallest tweaks, or even testing a feature you thought was performing well. Once you find a way to optimize your app, you’ll need help from your team to rally around making the change permanent.

Kelsey Ricard is the growth lead for Taplytics, a mobile optimization provider that helps companies improve the performance of their mobile apps, leading to increases in customer engagement, retention and revenues.