How to Increase Digital Advertising's ROI by Eliminating Bad Bots

Any experienced digital marketer will tell you, the secret to a successful advertising campaign is simple: targeting. We all want to show the right message, with the right product, at the right time, to just the right type of person and in the right channel.

In fact, “targeted advertising” is so fundamental to the digital space that we all simply refer to it as “advertising.” That said, what happens when your “target audience” becomes infested with bots, competitors and automated click farms that siphon away your precious ad budget?

Below, I break down how to measure the impact of ad fraud (also known as invalid traffic) on your website, how to detect some forms in real time, and how advertising teams can use new methods to defend and stop ad fraud in its tracks.

How Much ‘Bought’ Traffic is Actually Bot Traffic?

The first question we often ask ourselves is how much of my paid traffic is being wasted. Unfortunately, the answer isn’t quite as straightforward as vendors suggest, but we can get a good approximation via some simple segmentation techniques in Google Analytics.

Look at engagement.

To begin, we want to see the percentage of visitors engaged with the website. Engagement here has a broad meaning, with only the most finicky of web visitors being excluded. For our use here, an engaged user is one who either has a session length of at least 10 seconds or visits more than one page. Below are three examples of the segment applied to three different businesses with decreasing levels of ad fraud.

This first example shows a small percentage of engaged users, suggesting that a large portion (~58 percent) of website visitors don't in fact show any engagement signals. Although it’s unfair to say that 58 percent of users are fraudulent, an engagement rate at this low a level suggests an underlying issue — either website behavior such as speed and/or acquisition issues like ad fraud.

This next website is smaller (substantially less paid media), yet has a larger percentage of visitors who show engagement. This level of engagement is about standard for an e-commerce website, with engagement rates of 60 percent being the benchmark without enhanced protection methods.

Finally, this example shows a customer with a high engagement rate (90 percent) and illustrates the impact that active protection and advanced targeting has.

This segment is more useful when viewed by channel, allowing advertising teams to understand the rate by differentiating sources of traffic. For example, a customer recently found that its affiliate campaigns had a substantially lower engagement rate than its Google Ads campaign. Once an advertising fraud protection system was implemented, the two channels came within 10 percent of each other. This suggests the gap was driven largely by fraudulent clicks from the affiliate channel.

The key takeaway here is to estimate the level of wasted traffic by looking at engagement rates. Engagement rates <60 percent often point to either substantial site speed issues leading to visitors simply abandoning the website, tracking errors (less common, but possible), or a high rate of suspicious and automated traffic (bot traffic). This segment can also be run on per channel, and given some knowledge of the cost-per-click on those channels we can get approximated “ad waste” dollars.

For example, if the Facebook channel has a 35 percent engagement rate with an average daily spend of $5,000, then we can estimate that our loss due to fraud is somewhere between $1,250 (assuming a 60 percent engagement rate) to $3,250 (assuming a 90 percent engagement rate). For customers in the mid-market of e-commerce, we’ve found the average daily loss for Google Ads and Facebook to be roughly $11,000.

Defending Against Ad Fraud

The only way to defend against fraudulent clicks on ads (and the drain on your media budget) is to stop showing ads to fraudulent users. In essence, the secret to effectively fighting ad fraud is targeting, or more precisely “de-targeting.”

What is De-Targeting?

De-targeting is the process of adding exclusion rules to your targeted audiences — i.e., those definitions that you use to define the to whom/what and when to show your product. It used to be that the most common de-targeting method was IP exclusion lists. Media agencies would create a set of known IP addresses where fraudulent clicks come from, including data centers, VPN providers, etc., and add these to their ad platforms to ensure that at least some ads were available to be clicked. The problem is simply scale.

The largest IP exclusion available is on Google AdWords, but the limit there is a meager 500 IP addresses. Microsoft Bing is smaller still, with only 100 available IPs to add to de-targeting. Finally, Facebook offers no option to de-target by IP address. Setting aside the managerial overhead of maintaining the “top” IPs to exclude, the issue for online retailers is that the de-targeting option is simply too small.

To add some additional protection, media buyers can add other exclusions to their audiences. Common examples include excluding by country, excluding by browser (or restricting ads to only display to certain known browsers),or other custom exclusions based on more complex data attributes.

Edge Computing + De-Targeting

With the rise of edge computing — where brands can change content actions on the fly in response to every browser request — a new model for dynamic de-targeting has emerged. When combined with edge-enabled bot detection, modern online stores are implementing features like “bot tagging” or “bot redirection.” The idea is fairly straightforward. When a request arrives at the website with all the tell-tale signs of a bad actor, simply redirect the request to a new subpath.

For example, if your site is store.com and a request arrives from a known bot or bad IP, we simply redirect the browser to store.com/badbot. This allows the tracking engines like Google, Facebook and others to run on the website, ensuring that every ad platform you use today collects information about this user having visited the quarantined version of the website. Media buyers then add a single exclusion rule such as, “exclude any visitor who visited the page store.com/badbot.” This effectively uses the retargeting power of ad platforms to automatically de-target bad actors.

And there you have it — how to manage and eliminate bad bots by de-targeting interlopers. Redirection is protection, especially when it comes to fraudulent traffic to your online store.

Jake Loveless is the CEO of Edgemesh, the global web acceleration company he co-founded with two partners in 2016.