The Effects of Choice-based Sampling and Small-sample Bias on Past Fair Lending Exams (WP 2001-2)

Abstract

The Office of the Comptroller of the Currency uses choice-based sampling and limited sample sizes for statistically modeled fair lending exams. Both choice-based sampling and small sample sizes introduce bias into the maximum likelihood logit estimator, the standard estimator used by the OCC. This study applies results from Amemiya (1980) and Scott and Wild (1991) to estimate these biases for 16 recent exams.

The results show that of 29 tests of the null hypothesis of no racial effect conducted during the 16 exams, the outcome of two would change if small sample bias were taken into account, and the outcome of six would change if choice-based sampling bias were taken into account. Overall, the bias from choice-based sampling is generally larger. Although this study does not attempt to establish whether better sampling strategies would have changed examination conclusions based on any of the 29 hypothesis tests, the findings show that such strategies would have prescribed more thorough manual followup reviews for at least five of the 29 tests.

Author

Jason Dietrich