For digital lenders, the logic governing credit decisioning wields the power to shape their return on investment (ROI). A simple rate change could considerably change lending outcomes — except that it could take months for some lenders to implement it — especially those who have their business rules embedded in code. For other digital lenders that do use a business rules engine (BRE), the question is whether to shift to a   feature-rich BRE that offers more value. 

In this blog, we will discuss two features of our BRE Sentinel viz. Canary Testing and Champion/Challenger (CC) Analysis and the nature of the value that risk teams at lenders organisations can derive from them.

Canary testing is a method by which risk teams can test new waters in digital lending, without getting their feet wet. Put simply, it’s a way for digital lenders to test strategies under real-world conditions and gain a better understanding of how they will perform in a live environment. To put this further in perspective, here are a few scenarios.

Let’s say you are a lender who recently partnered with a platform whose users are predominantly thin file borrowers — a user base you are relatively new to. Your vintage analysis may not be applicable here, given the loan origination channel is new. You may try running simulations — except your historical data may or may not be representative of the newly acquired user base. Trial and error may work, however it’s time consuming and inefficient. 

Here’s where Canary Testing comes to the rescue. You can run a variation of an existing policy/workflow, or an entirely new policy/workflow in the live production environment without disrupting your day-to-day operations. While your primary policy is responsible for making credit decisions, the canary policy operates discreetly in the background, logging its decisions for later analysis. This approach ensures that you won't be venturing blindly into a new risk/borrower segment.

The Champion/Challenger approach is best described as a litmus test for financial strategies or what is popularly called A/B testing. In credit decisioning, the Champion refers to the tried-and-tested lending policy and the Challenger strategy serves as the experimental element, testing new variables or methodologies to determine if you can enhance the accuracy or approval rates of credit assessments. Just as the litmus test reveals changes in acidity or alkalinity, the Champion/Challenger approach exposes how alterations in credit evaluation tactics can impact overall decisioning outcomes.

Once you are armed with rich analytics from Canary testing, you can build your Challenger policy accordingly and introduce it to a subset of your borrower base, perhaps starting with 20%, employing the CC method. Subsequently, you can leverage the insights derived from this process to iteratively enhance your policy and progressively expand its implementation to larger segments of your borrower base. This way you can mitigate the risk of an adverse impact on business performance, live operation or customer experience. Let’s consider a few use cases, to better understand the strength of this combination. 

The cost per dollar of online lending fraud is growing at a faster rate than e-commerce and retail fraud. Lenders are responding to this with tough screening processes that end up rejecting good customers as well. Here’s how BRE features such as Canary and Champion/Challenger can help strike a balance between the warring sides of borrower convenience and fraud prevention. 

You can use Canary testing to evaluate impact of test parameters, gauge relevance of specific data sources, or assess scope for related drivers. Once you have a sense of what could work, you can randomly present Champion and up to five Challenger versions of the policy/workflow in specific proportions in live environment. Determining which version performs better will help you identify the impact on fraud rates and customer satisfaction. 

This is one of the most effective ways to quantify the trade-offs between customer experience and fraud losses. For example, let’s say you want to increase application completion rates without raising fraud levels. You can achieve this by trying out different thresholds/workflows for identity verification, and thereby determine impact on fraud rates and drop offs. 

However this kind of a rapid testing cycle needs continuous monitoring and real-time data. For instance, our BRE Sentinel allows you to set monitors to the Challenger policy so that you are immediately notified of any undesirable outcomes or unexpected deviations. 

To a lender, having a BRE with monitoring and analytics features like Champion/Challenger and Canary Testing is like having a lending autopilot system. It gives you peace of mind  when dealing with critical data because you can set monitors and get real-time analytics as and when required,

Credit risk strategies are unlikely to interest senior management unless there is a shift in focus from risk to profit. The best result a credit risk team can achieve is going beyond loss minimisation to profits maximisation, while keeping risk within acceptable limits. 

However, even after making this shift in mindset, the potential incremental gains projected by credit risk teams often get overshadowed by big ideas. The challenge lies in demonstrating how small, incremental improvements can cumulatively create a delta of change. Historical data provide insights into average performance, and does not account for marginal performance accurately.

Here’s where access to forward-looking analytical features such as Canary testing and Champion/Challenger become critical — for they can accurately assess the future impact of a strategy change.

With this, risk teams will be able to gather real-time marginal performance data in a series of small and controlled experiments.  The results of these experiments can be used to populate the profit model which can then be used to extrapolate the likely impact of the strategy when rolled out at scale.

Defining the scope of the combined power of Canary and Champion/Challenger takes imagination. While I am not the most imaginative, I can easily think of a couple of other instances where these features could come in handy. 

Let’s say a newly introduced third-party data source (bureau) in your Challenger policy resulted in errors. On our BRE Sentinel, you can set alerts that flag such errors immediately, making room for instant correction. How? You can use the Champion/Challenger feature to retrigger evaluation for borrowers that faced error, without the need for a second bureau pull, thereby reducing cost and preventing a potential revenue loss. 

Another area of improvement pertains to Personal Loans (PL) which are typically sanctioned based on bureau and banking data. You can easily assess the impact of a new data source and diversify your PL loan portfolio accordingly. You could also experiment with identity verification policies and workflow for Business Loans to arrive at optimum journeys that exclude unnecessary checks and reduce cost.

We have barely scratched the surface. There are numerous other scenarios involving change where a Challenger strategy can do wonders. Here are a few;

• New data sources, scorecards, or channels 

• Changes in regulatory or competitive landscape

• Shift in business targets, e.g. risk to revenue 

• Macroeconomic pressures

• New products or for fixing errors

• Specific targets, e.g. reduce exposure, mitigate risk

To know more about how these features work and how they help you manoeuvre such unique circumstances, get in touch with us .