Office of the Comptroller of the Currency - Ensuring a Safe and Sound Federal Banking System for all Americans. Site Map | Text Size: S M L

Alphabetical Listing


Are Early Warning Models Still Useful Tools for Bank Supervisors?

by Gary W. Whalen

December 2010

Abstract

This paper answers two timely, important research questions. Has the accuracy of statistical early warning models declined in recent years as the economic environment faced by banks has become more volatile? Has it become necessary to frequently re-specify or re-estimate these models to produce reasonably accurate forecasts of bank risk?

To answer these questions, a set of Cox proportional hazard composite CAMELS1 downgrade models are estimated for a sample of low-risk community banks at five different year-end dates ranging from 1997 through 2002. The estimated models are used to produce out-of-sample risk estimates for up to six future forecast periods. The models generally have Type I and Type II error rates in the low- to mid-30 percent range in each of the forecast years, including the most recent one. Forecast accuracy does not consistently or sharply decline with model age, indicating frequent re-specification or re-estimation is unnecessary. In addition, a supplemental analysis of forecast accuracy indicates that a considerable number of banks categorized as Type II errors by the models in each forecast period appear to be high risk ex post. The implication is that the “true” Type II error rates of the models are lower than the conventional figures reported in the tables.

Disclaimer

Any whole or partial reproduction of material in this paper should include the following citation: Gary W. Whalen, "Are Early Warning Models Still Useful Tools for Bank Supervisors?," Office of the Comptroller of the Currency, Economics Working Paper 2010-3, December 2010.

Availability

This Economics Working Paper (PDF) is available for viewing in Adobe's PDF format. The PDF viewer is available for download free from Adobe in versions for different platforms.


 1 CAMELS is an acronym for the risk assessment score assigned to banks by supervisors. The acronym reflects the component scores that are measures of a bank’s capital adequacy, asset quality, management, earnings, liquidity, and sensitivity to market risk.