Are You Auto-Blocking Good Customers? How to Tell
Automation may be the Holy Grail of efficient fraud management, but setting up a system that works for your business takes a healthy dose of care and thoughtfulness. It’s one thing to set your fraud tool up to automatically block and accept orders, based on a user’s risk. But how do you know that you’re setting the right risk thresholds?
The challenge of measuring false positives
It’s easy to tell when you’ve accepted an order that actually should have been rejected. You get hit with a chargeback or some other unpleasant surprise like an angry call from an upset cardholder. But what about the inverse situation? How do you know that the orders you rejected were actually fraudsters and not good customers caught up in the fray?
This is called a false positive, or customer insult, and is a more difficult problem to solve. If a customer can’t complete their order on your website they might call you to figure it out. However, chances are better they’ll end up going to one of your competitors – who won’t block them from purchasing the goods. Not only have you lost that particular sale, but will that customer ever return to your site? This is your worst case scenario – and one that keeps fraud and risk teams up at night. According to the Sift Science Fraud Trends 2016 Report, 76% of businesses facing fraud are concerned about turning away good customers.
How to tune your auto-block thresholds
Thankfully, there are ways to remedy the situation. It’s starts with accurately measuring how many false positives you’re inadvertently auto-blocking, and then using those findings to tweak any automation thresholds you have in place.
The first step is to isolate a small control group, from among all of the orders that would be blocked as part of your usual fraud process. Then, let all of these orders through, accepting them regardless of your fraud tool’s decision. Finally, as you start to receive chargebacks, go through the orders from the control group to determine how many of them are actually fraudulent. Once you know this proportion, you can determine whether you need to adjust your auto-block thresholds.
How this method works in practice
Here’s an example of how a surfboard manufacturer could measure their auto-block false positive rate, using Sift Science:
Surfboards Inc. runs an online shop in an attempt to expand their customer base, and uses Sift Science to prevent chargebacks from fraudulent orders. Based on their experience using the tool, they’ve found that (generally speaking) any order with a Sift Score greater than 90 is highly likely to be fraudulent and should be blocked automatically.
But they want to test this hypothesis. So they decide to let through 1% of orders that would usually be blocked by Sift Science. This way, they can evaluate what percentage of their automatically blocked orders (orders with scores above 90) result in chargebacks.
After examining the data, Surfboards Inc. realizes that of the orders in their control group, 5% did not result in a chargeback.
Total orders w/ Sift Score > 90 = 1,000
Chargebacks = 950
No chargebacks = 50
False positive rate = 5%
They would love to move this number even lower, so they change their fraud logic to automatically reject orders with scores over 95 instead. They repeat this process on a rolling basis, continually evaluating their control group.
“But I can’t afford to let fraud through…”
Of course, not every business will be comfortable accepting risky orders – even for a short time. Maybe you’re a smaller company, or you’re concerned that fraud would seriously damage your bottom line. In that case, there are other tactics that could give you insight into how many of your rejected transactions are actually good customers. For example, you could request that anyone who places an order that’s blocked contact you via email or phone.
This way, you can determine more carefully if the person trying to purchase from you is a fraudster or not. The tradeoff between these two methods is that with the latter, you spend significant energy on verifying users; with the former, you’re allowing a small percentage of fraud through. The path you choose will heavily depend on your business’ specific situation, goals, and needs.
Is it even possible to have a 0% false positive rate?
Of course – with Sift Science! Just kidding. We would love to make that claim, but in all honesty, if you’re like most businesses who are facing fraud the only way to have a 0% false positive rate at all times would be to let through 100% of your customers.
But in that case, of course, you’d be opening yourself to the risk of a sky-high fraud rate. Or else, you’d have to hire a giant team of people to manually review a large proportion of your orders and weed out the fraud. That’s why we highly recommend automating as much of your fraud review as possible, and using a method like the one described earlier to continually tune your fraud management strategy.
We hope this helps add some color to how you can make sure your fraud automation approach is working for you. Questions? Feel free to reach out anytime at support@siftscience.com.
