Backtesting Probability-of-Default Grades When Defaults Are Correlated: Exact One-Factor Acceptance Intervals, a Variance Bound on the Information in a Single Year, and an Application to 30,000 Credit-Card Accounts

Banks test their probability-of-default (PD) grades by comparing the default rate in each grade with the assigned PD, usually by a binomial or Jeffreys test that treats borrowers as independent. Defaults are not independent: a common economic factor moves all of them. I prove that in the one-factor Gaussian model the variance of a grade's default rate equals the binomial variance plus a term equal to the variance of the conditional PD, which does not shrink with the number of borrowers. The information in one year of data is therefore capped: with an asset correlation of 0.04, the Basel value for credit cards, a grade of any size is worth at most 40 to 85 independent borrowers at the PDs studied here. As a consequence, the binomial test rejects a correct PD in 40% to 89% of years for grades of 1,000 to 10,000 borrowers, and its size tends to 1 as the grade grows. Detecting a PD understated by 25% needs 20 years of data at a 5% PD and 36 years at 1%. I apply the tests to a seven-grade logistic PD model fitted to the UCI credit-card default data (30,000 accounts, hold-out AUC 0.787). Within the single observation period the hold-out default counts agree with the grade PDs (joint chi-square p = 0.116). I explain why that within-period test is valid but says nothing about calibration in future years.