Lost transparency, blackbox ML, and other hidden risks of outsourced fraud solutions

Machine learning-based fraud decision engines are sometimes viewed as mysterious black boxes that only provide minimal insight into why a decision was made on a login or a transaction. It’s a valid concern; not all fraud solution providers provide intuitive decision explainability. Some solutions fail to provide any transparency at all on the transactions they approve, essentially offering a blackbox ML experience.

Prioritizing decision explainability

At Sift, we invest in clear decision explainability—a vital component for any fraud team to perform their day-to-day responsibilities effectively. That includes:

Clarity for fraud analysts. Understanding why the system flagged a transaction for review is typically the starting point of a manual review. Decision explainability solves this by helping to point the analyst in the right direction of what they should be looking for and taking into consideration during the review. For example, if there is significant distance between the IP address location and the customer billing address, this is not a cut-and-dry sign of fraud—but it is worth taking a deeper look at.

Accuracy for risk strategists: Measuring decision accuracy is fundamental to the role of a risk strategist responsible for maintaining acceptable approval and block rates. Without the ability to understand the nuances of their decision strategy, they’re forced to rely on the vendor prioritizing continual optimization as needed. But when a fraud attack occurs, risk strategists need to be able to uncover the details of the attack so they can explain the increase in declines needed to mitigate the attack. Using a decision-as-a-service vendor, they’re not autonomous or empowered, and must rely on the vendor’s explanation and timeline. 

Relying on a vendor to explain upticks in declines or emerging fraud attacks puts businesses in a difficult position because they cannot self-serve and must rely on a third party to do an analysis on the transactions. One particular vendor actually developed a metric called “normalized approval rate,” which groups multiple fraud events into one single transaction to make it appear like the approval rate is higher than it really is.

The problem is that with this approach, it becomes totally unclear if false declines are grouped under this type of metric. This can be misleading to executives or other stakeholders that are not close to the product and how it works, and may give a false impression that the solution is more effective than it really is.

Transparency into data, control over outcomes

Not having transparency into the decisions made by your fraud solution means not having transparency into the daily operations of your business. The ability to understand why customers are approved and denied is fundamental to providing an exceptional customer experience. When faced with the decision of using a hands on platform that empowers the fraud team or paying for a service that handles fraud decisioning on your behalf, it’s critical to fully understand the implications on your ability to manage your business autonomously and service your customers.

Sifts offers transparency into decision explainability in multiple ways.

Sift’s Risk Summary feature surfaces and categorizes risk signals that were relevant to the transaction being investigated. This information is presented at the top of the transaction view within our console, which ensures fraud analysts are presented with important details before starting their investigation. 

In addition to displaying summary information, we display some of the top contributing signals that influenced that particular Sift score.

Beyond the transaction level, Sift displays transparency into our automated decision capabilities via our reporting tools that include top level metrics, rule performance and fraud analyst productivity. Unlike some solutions, we show the data as-is—never manipulating it to falsely inflate performance metrics. This gives users the confidence and clarity they need to understand how different types of fraud are impacting revenue and contraction, and the data they require to make smart decisions for their business.

Take our Digital Trust & Safety Assessment to discover what’s working for fraud operations at your company, and how you can fine-tune them for faster, more secure growth.

This is part 2 in our series on the importance of transparency, control, and community in fraud prevention. Read part 1 here, and stay tuned for the final installment.