© Błażej Kochański. Article available under the CC BY-SA 4.0 licence
ARTICLE
ABSTRACT
The AUC, i.e. the area under the receiver operating characteristic (ROC) curve, or its scaled version, the Gini coefficient, are the standard measures of the discriminatory power of credit scoring. Using binormal ROC curve models, we show how the shape of the curves affects the economic benefits of using scoring models with the same AUC. Based on the results, we propose that the shape parameter of the fitted ROC curve is reported alongside its AUC/Gini whenever the quality of a scorecard is discussed.
KEYWORDS
credit scoring, receiver operational characteristic, AUC
REFERENCES
Adams, N. M., Hand, D. J., (1999). Comparing classifiers when the misallocation costs are uncertain. Pattern Recognition, Vol. 32(7), pp. 1139–1147.
Bandos, A. I., Guo, B., Gur, D., (2017). Estimating the area under ROC curve when the fitted Binormal Curves Demonstrate Improper Shape. Academic Radiology, Vol. 24(2), pp. 209–219.
Blöchlinger, A., Leippold, M., (2006). Economic benefit of powerful credit scoring. Journal of Banking & Finance, 30(3), pp. 851–873.
Chen, W., Hu, N., (2016). Proper bibeta ROC model: algorithm, software, and performance evaluation, in: Medical Imaging 2016: Image Perception, Observer Performance, and Technology Assessment. Medical Imaging 2016: Image Perception, Observer Performance, and Technology Assessment, SPIE, pp. 97–104.
Davidov, O., Nov, Y., (2012). Improving an estimator of Hsieh and Turnbull for the binormal ROC curve. Journal of Statistical Planning and Inference, Vol. 142(4), pp. 872–877.
Dorfman, D. D., Berbaum, K. S., Metz, C. E., Lenth, R. V., Hanley, J. A., Dagga, H. A., (1997). Proper receiver operating characteristic analysis: The bigamma model. Academic Radiology, 4(2), pp. 138–149.
England, W. L., (1988). An exponential model used for optimal threshold selection on ROC curves. Medical Decision Making, Vol. 8(2), pp. 120–131.
Garrido, F., Verbeke, W., Bravo, C., (2018). A Robust profit measure for binary classification model evaluation. Expert Systems with Applications, Vol. 92, pp. 154– 160.
Hahm, J.-H., Lee, S., (2011). Economic effects of positive credit information sharing: the case of Korea. Applied Economics, Vol. 43(30), pp. 4879–4890.
Hand, D. J., (2009). Measuring classifier performance: A coherent alternative to the area under the ROC curve. Machine Learning, Vol. 77(1), pp. 103–123.
Hand, D. J., Anagnostopoulos, C., (2013). When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance? Pattern Recognition Letters, Vol. 34(5), pp. 492–495.
Hanley, J. A., (1996). The Use of the “Binormal” Model for Parametric ROC Analysis of Quantitative Diagnostic Tests. Statistics in Medicine, 15(14), pp. 1575–1585.
Hanley, J. A., McNeil, B. J., (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, Vol. 143(1), pp. 29–36.
Hsieh, F., Turnbull, B. W., (1996). Nonparametric and semiparametric estimation of the receiver operating characteristic curve. The Annals of Statistics, Vol. 24(1), pp. 25–40.
Hughes, G., Bhattacharya, B., (2013). Symmetry Properties of Bi-Normal and Bi- Gamma Receiver Operating Characteristic Curves are Described by Kullback- Leibler Divergences. Entropy, Vol. 15(4), pp. 1342–1356.
Idczak, A. P., (2019). Remarks on statistical measures for assessing quality of scoring models. Acta Universitatis Lodziensis. Folia Oeconomica, Vol. 4(343), pp. 21–38.
Jokiel-Rokita, A., Topolnicki, R., (2019). Minimum distance estimation of the binormal ROC curve. Statistical Papers, Vol. 60(6), pp. 2161–2183.
Killeen, P. R., Taylor, T. J., (2004). Symmetric receiver operating characteristics. Journal of Mathematical Psychology, Vol. 48(6), pp. 432–434.
Kochański, B., (2022). Which curve fits best: fitting ROC curve models to empirical credit-scoring data. Risks, Vol. 10(10), p. 184.
Kürüm, E., Yildirak, K., Weber, G.-W., (2012). A classification problem of credit risk rating investigated and solved by optimisation of the ROC curve. Central European Journal of Operations Research, 20(3), pp. 529–557.
Lobo, J. M., Jiménez-Valverde, A., Real, R., (2008). AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography, Vol. 17(2), pp. 145–151.
Řezáč, M., Koláček, J., (2012). Lift-based quality indexes for credit scoring models as an alternative to Gini and KS. Journal of Statistics: Advances in Theory and Applications, Vol. 7(1), pp. 1–23.
Řezáč, M., Řezáč, F., (2011). How to Measure the Quality of Credit Scoring Models. Czech Journal of Economics and Finance (Finance a úvěr), Vol. 61(5), pp. 486–507.
Satchell, S. Xia, W., (2008). Analytic models of the ROC Curve: Applications to credit rating model validation, in G. Christodoulakis and S. Satchell (eds). The Analytics of Risk Model Validation. Burlington: Academic Press (Quantitative Finance), pp. 113–133.
Tang, T.-C., Chi, L.-C., (2005). Predicting multilateral trade credit risks: comparisons of Logit and Fuzzy Logic models using ROC curve analysis. Expert Systems with Applications, Vol. 28(3), pp. 547–556.
Verbeke, W., Dejaeger, K., Martens, D., Hur, J., Baesens, B., (2012). New insights into churn prediction in the telecommunication sector: A profit driven data mining approach, Vol. 218(1), pp. 211–229.
Verbraken, T., Verbeke, W., Baesens, B., (2013). A Novel Profit Maximizing Metric for Measuring Classification Performance of Customer Churn Prediction Models. IEEE Transactions on Knowledge and Data Engineering, Vol. 25(5), pp. 961–973.
Walsh, S. J., (1997). Limitations to the Robustness of Binormal Roc Curves: Effects of Model Misspecification and Location of Decision Thresholds on Bias, Precision, Size and Power. Statistics in Medicine, 16(6), pp. 669–679.