Are Early Warning Models Still Useful Tools for Bank Supervisors? (WP 2010-3)

Abstract

This paper answers two timely, important research questions. Has the accuracy of statistical early warning models declined in recent years as the economic environment faced by banks has become more volatile? Has it become necessary to frequently re-specify or re-estimate these models to produce reasonably accurate forecasts of bank risk?

To answer these questions, a set of Cox proportional hazard composite CAMELS1downgrade models are estimated for a sample of low-risk community banks at five different year-end dates ranging from 1997 through 2002. The estimated models are used to produce out-of-sample risk estimates for up to six future forecast periods. The models generally have Type I and Type II error rates in the low- to mid-30 percent range in each of the forecast years, including the most recent one. Forecast accuracy does not consistently or sharply decline with model age, indicating frequent re-specification or re-estimation is unnecessary. In addition, a supplemental analysis of forecast accuracy indicates that a considerable number of banks categorized as Type II errors by the models in each forecast period appear to be high risk ex post. The implication is that the "true" Type II error rates of the models are lower than the conventional figures reported in the tables.

Author

Gary W. Whalen