Skip to content
// EXAM HALL24 QUESTIONS · 30 MINUTES

Bank Data Scientist Practice Test

What a bank's data can and cannot tell you: rare events, one credit model's pack, and the calls on fairness and evidence.

BY MICHAEL SHANG · SIXTEEN YEARS IN COMMERCIAL BANKING · UPDATED

// THE SHORT ANSWER

A bank data scientist practice test asks whether you can tell what a bank's data does and does not show. This paper has three parts: rare events, precision and recall, joins and tests; one credit model's pack, set on an exhibit, with drift and the economics of a cut-off; and calls on fairness, evidence and customer data. There is no coding in it, so it does not replace your own technical round. Every question is original, and none comes from any bank's test. Sit it against a thirty-minute clock, or take it as practice.

BASIS  Practitioner judgment from sixteen years in commercial banking, set out in How Bankers Think (Highbank Press, 2026). Every borrower on this site is a composite constructed for teaching, not a client. The market is mid-market commercial lending as practised in New Zealand and Australia; where a convention differs elsewhere, the page says so.

// HOW TO SIT IT

Answer below as practice, and each read opens as you go. Or sit it as a test: 30 minutes, with every read held back until you finish.

PART A

What the data can tell you

Rare events, messy tables, and the difference between a pattern and a cause.

PART A · QUESTION 1

1% of transactions are fraud. A model catches 90% of fraud, and wrongly flags 5% of genuine transactions. Take 10,000 transactions.

What share of flagged transactions are really fraud?

PART A · QUESTION 2

You are asked how many active customers the bank has. Marketing's dashboard says 1.2 million, finance says 0.9 million, and the regulator's return says 1.0 million. All are correct on their own terms.

What do you do?

PART A · QUESTION 3

A fraud model is reported as 99.5% accurate. Fraud is 0.5% of transactions.

What does that accuracy figure tell you?

PART A · QUESTION 4

A model flags 1,000 transactions, and 200 of them are fraud. There were 400 frauds in all.

What are its precision and its recall?

PART A · QUESTION 5

You join 10,000 loans to a table of guarantors, to add a guarantor flag. The result has 12,400 rows, and total lending has risen by 24%.

What has most likely happened?

PART A · QUESTION 6

A new loan page was shown to 5,000 customers, and 110 applied. The old page was shown to 5,000, and 100 applied. The product owner says: 'That is a 10% uplift. Ship it.'

What does the test show?

PART A · QUESTION 7

Customers who use the mobile app default 40% less often. The head of digital says: 'Push the app, and defaults will fall.'

Does that follow?

PART A · QUESTION 8

An income field is blank for 15% of applicants, and those applicants default more often. A colleague fills the blanks with zero, so that the model can run.

What do you do?

PART B

The model pack

One model, nine questions. Read the last row of each table first.

EXHIBIT Q · THE MODEL PACK

A credit card default model, as it goes to the quarterly model committee.

How well it ranks (Gini)
Gini
Build sample, 2023–240.68
Holdout, same period0.66
Applications, latest six months0.49
Whole book, all vintages, defaults over twelve months, by score band
AccountsDefaultsRate
Band A4,000200.5%
Band B3,000602.0%
Band C2,0001206.0%
Band D1,00020020.0%
Book10,0004004.0%

A good account earns US$200 a year, and a default costs US$2,000. The table covers approved accounts only. Last June the bank began taking applications through a comparison website.

PART B · QUESTION 9

From Exhibit Q.

What is the default rate across the book, and how does band D compare?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 10

Suppose the bank stops lending to band D in Exhibit Q.

What does that do to the year's result?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 11

The bank then asks about stopping band C too, in Exhibit Q.

What does that add to the year's result?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 12

In Exhibit Q, ranking power falls from 0.66 on the holdout to 0.49 in the latest six months.

What do you look at first?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 13

The product head reads Exhibit Q and says: 'The holdout is 0.66. It was approved on that. We are fine.'

What do you tell them?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 14

Exhibit Q's band table covers approved accounts only. A colleague says: 'Band C still pays its way, so we can loosen the policy on declines.'

What is the gap in that?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 15

Exhibit Q's comparison website offers a new feature: 'applied after 11pm' strongly predicts default.

What do you do?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 16

If band D in Exhibit Q is declined, some good customers go with it.

How many good customers are turned away for each default avoided?

BACK TO EXHIBIT Q ↑
PART B · QUESTION 17

You rebuild Exhibit Q's model. It scores 0.60 on the latest six months, but it uses 240 features, and nobody can say why any one person was declined.

What do you do?

BACK TO EXHIBIT Q ↑

PART C

The calls

Bias, evidence and customer data, in a place where someone will ask you to prove it.

PART C · QUESTION 18

Your early-warning model for business loans scores remarkably well on test data. One strong feature turns out to be 'number of calls from the collections team'.

What do you conclude?

PART C · QUESTION 19

A credit scoring model does not use ethnicity. An analysis finds people from one community are declined far more often at the same repayment record. Postcode is a strong feature.

What do you do?

PART C · QUESTION 20

A fraud model went live a year ago and has not been looked at since. Its alert volumes look normal. A colleague says: 'It is not broken, so leave it alone.'

What do you say?

PART C · QUESTION 21

To save time, a colleague pasted 200 customer complaints, names included, into a public AI chatbot to summarise them.

What do you do?

PART C · QUESTION 22

Your analysis of a new product shows no effect. The sponsor asks you to 'look at a few more segments' until something positive appears.

What do you do?

PART C · QUESTION 23

You find a three-month-old bug: a feature in the live credit model has been reading last year's income. Approvals are affected, and you do not know how many.

What do you do?

PART C · QUESTION 24

A declined customer asks why. The decision came from a score built on 40 features, and a branch colleague asks you for something to tell them.

What do you give them?

0 OF 24 ANSWERED · 0 WITH THE SENIOR READ · 0 STANDS UP · 0 OFF THE LINE

// THE REST OF THE ROUND

Every question here is original; none is any bank's real test. Written for: Data scientist.