CASE STUDY · FINTECH AI UNDERWRITING & CREDIT DECISIONING
Building an AI credit scoring model for a digital lender.
- 8On a holdout set, approval rate up 8 points at the same default rate
01 · PROJECT OVERVIEW
The client and the brief.
An anonymised digital lender. Name withheld under confidentiality.
Labelled data, a baseline model, holdout validation, reason codes and fairness checks.
- 1Scoring was manual, and a large share of applicants were thin-file: little or no credit history.
- 2Underwriters applied judgement unevenly.
- 3Good borrowers with short histories were declined by default because the old scorecard had nothing to say about them.
- 4The lender wanted to approve more of these people without taking on more risk, and needed to show why each decision was made.
02 · WHAT WENT WRONG
What went wrong, and how we fixed it.
Every engagement hits problems. These were the ones that mattered here, and what we changed so they did not come back.
- PROBLEM 01Our first labels were wrong
Some loans counted as repaid were really restructured after missed payments, so the model learnt that a risky pattern was safe. We spotted it when a segment performed too well to be believable. We worked with the collections team to recode restructured loans as stressed, rebuilt the labels and retrained.
- PROBLEM 02Results moved, but they were now trustworthy
- PROBLEM 03A second hurdle came late
Reason codes were accurate but read like model internals, such as a feature name and a score band. Risk and customer support both said an applicant could not act on them. We rewrote the mapping with the risk team so each code became a short sentence about something the applicant could understand, such as limited repayment history or high existing obligations, and tested the wording with the support team.
- PROBLEM 04We also capped the number of reasons shown, so the most important ones were not buried
03 · THE PROCESS
How we ran it.
The work ran in four phases with a weekly review with the head of risk.
- 01Step 1
Data and labels: we gathered past applications and loan outcomes, agreed what counts as a default, and set the observation window.
- 02Step 2
Baseline: a simple, well-understood model first, to set the bar.
- 03Step 3
Candidate model: added features from bank statements and repayment behaviour, and compared against the baseline on the same holdout.
- 04Step 4
Review and hand-off: fairness results, reason code wording and model documentation went to risk and compliance for sign-off.
Every week we showed the same four numbers: approval rate, default rate on the holdout, share of thin-file applicants approved and stability across groups.
04 · RESULTS
What changed.
Measured against the way the work ran before the engagement began.
- NOW 01
Risk reviews model performance on a regular schedule, using the monitoring view.
- NOW 02
Underwriters see reason codes next to every score and record why when they override.
- NOW 03
New features go through the same holdout and fairness check before they are used.
- NOW 04
Label definitions are written down and owned by risk, not left to whoever builds the next model.
- 8
On a holdout set, approval rate up 8 points at the same default rate
Every decision carries reason codes
05 · WHAT WE BUILT
How it works.
- Feature pipeline built once and used for both training and live scoring, to avoid training-serving mismatch.
- Reason codes generated from each decision's largest contributing factors and mapped to plain-language wording.
- Fairness checks compared approval and error rates across groups, and we reviewed inputs for proxy effects.
- Model, data snapshot and code versioned together, so any decision could be reproduced.
- A scoring service with a log of inputs, outputs and reason codes for every call.
06 · QUICK ANSWERS
The short version.
- What problem did the client have?
- Scoring was manual, and a large share of applicants were thin-file: little or no credit history. Underwriters applied judgement unevenly. Good borrowers with short histories were declined by default because the old scorecard had nothing to say about them. The lender wanted to approve more of these people without taking on more risk, and needed to show why each decision was made.
- What did Brandsmashers build?
- Labelled data, a baseline model, holdout validation, reason codes and fairness checks.
- How was the work run?
- The work ran in four phases with a weekly review with the head of risk.
- What were the results?
- On a holdout set, approval rate up 8 points at the same default rate; every decision carries reason codes.
YOUR TURN
Have a similar requirement?
Tell us what is stuck. We will come back with a plan and matched, pre-vetted engineers.
