Applied Economics

ISSN: 0003-6846 (Print) 1466-4283 (Online) Journal homepage: [Link]

A Race for Long Horizon Bankruptcy Prediction

Edward I. Altman, Małgorzata Iwanicz-Drozdowska, Erkki K. Laitinen & Arto

Suvas

To cite this article: Edward I. Altman, Małgorzata Iwanicz-Drozdowska, Erkki K. Laitinen &
Arto Suvas (2020): A Race for Long Horizon Bankruptcy Prediction, Applied Economics, DOI:
10.1080/00036846.2020.1730762

To link to this article: [Link]

Published online: 27 Feb 2020.

Submit your article to this journal

Article views: 106

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

[Link]
APPLIED ECONOMICS
[Link]

A Race for Long Horizon Bankruptcy Prediction

Edward I. Altmana, Małgorzata Iwanicz-Drozdowska b
, Erkki K. Laitinenc and Arto Suvasc
a
Stern School of Business, New York University, New York, NY, USA; bInstitute of Finance, Financial System Department, SGH - Warsaw School
of Economics, Warsaw, Poland; cSchool of Accounting and Finance, University of Vaasa, Vaasa, Finland

ABSTRACT KEYWORDS
This study compares the accuracy and efficiency of five different estimation methods for predicting SMEs; estimation technique;
financial distress of small and medium-sized enterprises. We apply different methods for a large set of variable selection; cut-off;
financial and non-financial variables, using filter and wrapper selection, to predict bankruptcy up to cost-return ratio
10 years before the event in an open, European economy. Our findings show that logistic regression and JEL CLASSIFICATION
neural networks are superior to other approaches. We document how the cost-return ratio considerably G32; G33
affects the location of optimal cut-off points and attainable profit in credit decisions. Once a loan
provider selects a particular prediction model, an effort should be made to find the optimal cut-off score
to maximize the efficiency of the technique. Indeed, this often involves determining several cut-off
levels where the portfolio of products and services exhibits different cost-return characteristics.

I. Introduction 1989; Moulton, Thomas, and Pruett 1996; Altman,

Sabato, and Wilson 2010; Altman et al. 2016).
Failure prediction models are used by many stake-
In traditional failure models, the predictive abil-
holders in predicting or avoiding a failure. High
ity of financial variables is high for the one-year
accuracy, a long prediction horizon, and interpret-
horizon, but decreases quickly after that (du Jardin
ability, as well as low cost are desired properties of
and Severin 2011; du Jardin 2015). However, devel-
these models. The early roots of scientific bank-
opment of an effective prediction model for
ruptcy prediction modelling are usually traced back
a longer horizon is a very challenging task due to
to the univariate approach (Beaver 1966), the multi-
instability of financial ratios and fluctuations
variate approach (Altman 1968), the use of non-
alongside the economic cycle. In particular, finan-
financial variables (Keasey and Watson 1987) and,
cial ratios of small and medium-sized enterprises
at later stages, the application of various estimation
(SMEs) are often unstable over time and do not
techniques to improve prediction accuracy (e.g.,
contain reliable annual information, which makes
Dimitras, Zanakis, and Zopounidis 1996; Bellovary,
the use of non-financial variables important
Giacomino, and Akers 2007; du Jardin 2015).
(Balcaen and Ooghe 2006; Altman and Sabato
Beaver (1966) observed that certain financial
2007; Altman, Sabato, and Wilson 2010; Altman
ratios follow a systematic process over time and
et al. 2016). However, the use of non-financial
have some predictive ability up to five years before
variables is hardly possible on a cross-country
the failure. A number of studies (Argenti 1976;
basis due to their incomparability and limited
Hambrick and D’Aveni 1988; D’Aveni 1989;
availability. Therefore, studies on non-financial
Laitinen 1991; Ooghe and de Prijcker 2008) also
predictors are usually focused on single countries
show that the failure process can take a number of
(e.g., the U.S., Finland, Italy, Spain).
years (even to 5–8 years). Many researchers have
Moreover, stakeholders target profit maximiza-
focused on longer horizons, analysing different fail-
tion, moving the cut-off scores up or down in order
ure processes (Argenti 1976; Laitinen 1991; du
to both avoid extending credit to potential bank-
Jardin and Severin 2011, 2012; du Jardin 2015) and
ruptcies (Type I error) and to avoid losing attrac-
also focusing on non-financial variables to explain
tive customers (Type II error). Therefore, the
different processes in the long term (Ooghe and de
performance of prediction methods should be
Prijcker 2008; Hambrick and D’Aveni 1988; D’Aveni

CONTACT Małgorzata Iwanicz-Drozdowska miwani@[Link] Warsaw School of Economics, Al. Niepodległości 162 Warsaw 02-554, Poland
© 2020 Informa UK Limited, trading as Taylor & Francis Group
2 E. I. ALTMAN ET AL.

assessed through the lens of cost-return analysis use of different variable selection and estimation
(e.g., Altman, Haldeman, and Narayanan 1977; techniques. In this section, we also draw research
Weiss and Capkun 2005). Because the relative propositions. In the third section, the sample of
cost of Type I error is high, it is important that bankrupt and non-bankrupt firms, variables, vari-
the model be efficient, especially in classifying able selection methods, and estimation methods are
bankrupt firms correctly. There are a vast number presented. The fourth section presents and discusses
of different statistical methods developed for failure the empirical results on the accuracy and value of
prediction designed to increase accuracy or to different methods, while the last section summarizes
lengthen the horizon or both (Dimitras, Zanakis, our findings and presents limitations of the study as
and Zopounidis 1996; Bellovary, Giacomino, and well as challenges for future research.
Akers 2007; Du Jardin 2015). However, the value of
these methods is rarely assessed in the context of
II. Literature review and propositions
non-financial variables and small firms (Altman
and Sabato 2007). Therefore, there is a call to How to select variables properly and how many
compare the performance of different methods variables should be used are important questions
based on both financial and non-financial vari- in failure prediction modelling. Balcaen and Ooghe
ables, especially in the small business context. (2006) reviewed business failure studies from the
The purpose of this study is twofold. We aim to last 35 years and concluded that there is a lack of
assess the predictive ability of five different estima- any theoretical framework for variable selection
tion methods (logistic regression (LR); decision tree and little consensus on which financial variables
(DT), gradient boosting (GB), neural network (NN), are the best for discriminating between failed and
and support vector machine (SVM)) with filter and non-failed firms.
wrapper variable selection methods in a large cross- Because of the multitude of variables available
sectional sample of 59,099 Finnish firms over a ten- (Bellovary, Giacomino, and Akers 2007 identified
year period 2003–2013. Moreover, we assess the 752 variables used in bankruptcy prediction stu-
profit at each method’s optimal cut-off in compar- dies), the selection of an efficient combination of
ison with a hypothetical ‘perfect credit decision’ with predictors is an important issue in bankruptcy pre-
zero Type I and II errors. The majority of the sample diction modelling (du Jardin 2009). For Back,
firms are SMEs domiciled in Finland, for which we Laitinen, and Sere (1996a, 1996b) compared step-
have access to both financial and non-financial data. wise discriminant analysis, stepwise logistic regres-
We contribute to the research on failure predic- sion, and genetic algorithm in the selection of
tion in three ways. First, compared to recent studies predictors. They showed that these methods lead
(e.g., Barboza, Kimura, and Altman 2017; Jones, not only to different predictors but also to
Johnstone, and Wilson 2017), we use a very wide a different number of predictors. These findings
set of input variables (including non-financial vari- are supported by du Jardin (2010), who compared
ables). Second, we analyse the effect of horizon three different selection methods that are especially
length (up to 10 years) on the comparative perfor- suitable for neural network (NN) analysis.
mance of different methods. Third, we assess the In this study, we compare variants of two com-
value of the methods using the cost-return ratio for monly used variable (feature) selection methods:
classification. Thus, we show the effect of relative the filter method and the wrapper method. One
costs on the value of different methods. We important characteristic of filter methods is that
demonstrate that LR and NN are the most efficient each feature is considered separately, thereby
methods in bankruptcy prediction for all the hor- ignoring dependencies among variables. Wrapper
izons in our data. Similarly, we show that GB and approaches instead include the interaction between
DT lead to the least accurate prediction results. feature subset search and model selection, and the
The paper is organized as follows. In this intro- ability to take into account feature dependencies.
ductory section, the motivation and objectives of the (See e.g. Beniwal and Arora (2012). We use the
study are briefly discussed. The second section linear R-squared method as the filter selection ver-
reviews prior studies on failure prediction with the sion and the logistic (forward) stepwise LR as the
 APPLIED ECONOMICS 3

wrapper method procedure. SVM, GB, and DT a result of this lack of consensus, the selection of the
models are examined with their default settings, modelling method is left entirely to the researcher.
whereby they employ their own built-in mechan- Du Jardin (2009) analysed 190 papers on bankruptcy
isms for variable selection. prediction and found that more than fifty methods
We assume that the weaker the symptoms of were used in those studies. The results usually
bankruptcy are, the more important it is that the revealed only minor differences in prediction accu-
variable selection method be consistent with the racy among the methods.
modelling method and that it can detect possible Chen (2011a), using data for Taiwanese compa-
dependencies among predictors. These character- nies, employed LDA, LR, DT, NN, SVM, and evolu-
istics are of relevance to our research on the pre- tionary computation techniques, supported by the
diction accuracy for several horizons, since the selection of ratios with the use of principal compo-
longer the horizon, the weaker the symptoms of nent analysis (PCA). The conclusions were that tra-
bankruptcy. Thus, we expect that, especially for ditional statistical methods were better in handling
longer horizons, the filter method will be outper- large datasets, while AI techniques performed better
formed by the wrapper method, as the latter is with small datasets. Moreover, SVM with evolution-
more closely related to the modelling in this study ary computation provided a good balance of high
and takes into account variable dependencies. accuracy in the short and long term as well as for
Therefore, we present the following two-part distressed and non-distressed firms.
research propositions (P1): Du Jardin (2015) used five popular methods (DA,
LR, NN, survival analysis, and self-organizing map-
Proposition P1a: The filter method is outperformed by
the wrapper method in selecting variables for failure
ping) to predict bankruptcy for horizons of one, two,
prediction models. and three years for French firms. He did not test the
differences in accuracy among the methods statisti-
Proposition P1b: The difference in performance between cally, but his research proved intuitively that the dif-
the wrapper method and the filter method increases with
ferences are minor and dependent on the industry
the prediction horizon.
and the prediction horizon. Especially with a short
There are a great many studies on failure model horizon, the differences are negligible. With a very
estimation methods. One of the first studies pro- short horizon, the signs of bankruptcy may be so
viding a wide review of various modelling techni- obvious that any statistical method can make an accu-
ques was carried out by Dimitras, Zanakis, and rate prediction. When the horizon increases and the
Zopounidis (1996). At that time, multiple DA and symptoms of bankruptcy become weaker, the differ-
LR prevailed as tools for bankruptcy prediction. ences in accuracy among the methods can increase.
However, at the later stage traditional statistical Jones, Johnstone, and Wilson (2017) examined the
tools have been replaced by artificial intelligence predictive performance of several ‘simple classifiers’
techniques (e.g., Kumar and Ravi 2007; Moro, (LR, DA, and probit), ‘more advanced techniques’
Cortez, and Rita 2015; Do Prado et al. 2016). (NN, SVM), and ‘new age’ methods (random forests,
There are a number of studies comparing the AdaBoost, and generalized boosting) in a large
performance of different estimation methods in U.S. public company data set. While LR and DA
bankruptcy prediction. Laitinen and Kankaanpää performed reasonably well, the ‘new age’ methods
(1999), for example, compared six alternative meth- had superior predictive performance. Similar evi-
ods (DA, LR, recursive partitioning, survival analy- dence was provided by Barboza, Kimura, and
sis, NN, and the Human Information Processing Altman (2017), who tested, on a large set of North
approach), reporting that none of the statistical American firms, the performance of machine learning
methods was superior to any other. Balcaen and methods (SVM, bagging, boosting, and random for-
Ooghe (2006) concentrated on classic methods and est), showing they outperformed traditional LDA, LR,
concluded that despite the extensive research, there and NN.
seemed to be no superior modelling method in So far, the results of studies designed for com-
bankruptcy prediction. They further claimed that paring the performance of different methods have
most studies reach heterogeneous conclusions. As been mixed. However, the overall conclusion has
4 E. I. ALTMAN ET AL.

been that artificial intelligence outperform tradi- The total number of observations fulfiling our
tional statistical techniques. Not much attention criteria is 59,099. Over the 2004–2013 period, some
has been paid to the long-term horizon, but keep- of those firms (10,183) ceased operations due to
ing in mind the PD lifetime required by IFRS 9, merger, demerger, or voluntary liquidation. The
failure prediction modelling should move in this number of firms that were active at the end of
direction. In this study, we try to fill this gap. We 2013 is 46,949. This comprises 79.4% of the original
expect that with a short prediction horizon, the data set, whereas 1,967 firms (3.3% of the data set)
symptoms of failure are so obvious that the differ- went bankrupt, and the remaining firms dissolved
ences in accuracy among alternative methods will for various reasons.
not be significant. However, when the horizon Due to the fact that financial variables are
increases, the symptoms become weaker and, con- usually highly correlated with one another, which
sequently, the differences in performance among because of multicollinearity can be disruptive for
the methods of varying sophistication increase. The the accuracy of the model (e.g., Balcaen and Ooghe
longer the horizon, the larger the differences in 2006), researchers have paid attention to non-
performance between less and more sophisticated financial variables, which typically are not closely
modelling methods. Therefore, we set the following correlated with one another or with financial vari-
three-part proposition (P2): ables (see Altman et al. 2016). In addition, these
variables are not exposed to window-dressing or
Proposition P2a: With a short horizon, there are no
differences in the prediction performance of different
smoothing in the same way as financial variables.
estimation methods. Therefore, non-financial variables are useful in
small-business failure prediction studies. These
Proposition P2b: The difference in the prediction perfor- kinds of variables can be derived from different
mance of different estimation techniques increases with
sources, such as credit registers, other official reg-
the prediction horizon.
isters related to business activity, market intelli-
Proposition P2c: More sophisticated methods perform gence, press announcements and even annual
better than less sophisticated methods with a longer reports. In practice, their availability depends on
horizon.
the size of the firm, its legal form, and the country-
specific legal framework.
This study is based on a large set of both finan-
III. Data and methodology cial and non-financial variables in order to produce
Data and variables results that are as general as possible. The list of
variables and their definitions are presented in
This study is based on financial statements and non- Appendix 1. We have selected the two sets of vari-
financial data provided by a credit information com- ables on the basis of relevant prior studies.
pany (Suomen Asiakastieto Oy, SA) as of the end of When selecting the set of financial variables, we
2003, with the following restrictions and selection paid special attention to Edmister (1972), Ohlson
criteria. As of the cross-section date (end of 2003), (1980), Beaver (1966), Altman (1968), and
the maximum acceptable age of the most recent Bellovary, Giacomino, and Akers (2007). The avail-
financial statement was set to 24 months. Private able set of variables includes 15 financial variables
proprietors, foundations, associations, financial classified into seven groups (profitability, liquidity,
firms, and housing companies were not included. If solvency, cash flow, size, growth, and changes in
the company age was not recorded or if turnover or ratios). It should be noted that almost all the firms
total asset figures were missing, the firm was dropped. in the data are private, limited companies.
Two consecutive most-recent financial statements Therefore, market-based ratios are not included
were needed to calculate the cash-flow and the annual in the analyses (as in Hillegeist et al. 2004; Chava
change-based variables, which left out very young and Jarrow 2004; Reisz and Perlich 2007). Since the
startups from the data. We have gathered follow-up basic data are cross-sectional, only one year’s
data regarding the future statuses and status change growth is considered as a measure of growth. The
years for the 2004–2013 period. distributions of most financial variables are skewed
 APPLIED ECONOMICS 5

so that medians provide better insight on average The wrapper method adopted in this study is the
values than do means. Variables containing stepwise logistic regression, which is naturally con-
obvious outlier observations have been winsorized, sistent and connected with LR in modelling. This
generally at 1% and/or 99%.1 method is performed in several successive steps.
The second set of variables is composed of 20 First, the method estimates the intercept and com-
non-financial variables with special reference to putes the chi-square statistic for each variable not in
Keasey and Watson (1987), Flagg, Giroux, and the model and examines the largest of these statis-
Wiggins (1991), Laitinen (1999), Back (2005), tics. If it is significant at the specified level, the
Altman, Sabato, and Wilson (2010), and Altman corresponding predictor is added to the model.
et al. (2016). These non-financial variables are also Then, the chi-square statistic for each variable not
classified into seven groups (firm type, age, indus- in the model is again calculated, and a new predictor
try risk, audit report, disclosure policy, payment is potentially entered into the model. However,
behaviour, and board members). Since almost all a variable already in the model does not necessarily
the firms are limited companies, the company remain but can be removed if its significance level is
legal form variable is not included in the data below the specified level after adding a new predictor
set. The variable set is discussed in detail in to the model. Effects are entered into and removed
Altman et al. (2016). from the model in such a way that each forward
selection step can be followed by one or more back-
ward elimination steps. The stepwise selection pro-
Variable selection methods
cess terminates if no further effect can be added to
In this study, we compare two commonly used vari- the model or if the current model is identical to
able selection methods: a filter and a wrapper. To a previously visited model.
this aim we use the SAS Enterprise Miner 14.1 soft-
ware (hereafter SAS EM).2 We apply these selection
Modelling methods
methods only with LR and NN, contrasting the
results with corresponding all-variables models. We compare the performance of five different
We use the R-squared method as representative of modelling methods: decision tree (DT), gradient
filter methods. This method is performed in two boosting (GB), logistic regression (LR), neural net-
successive steps. First, R-squares between each poten- work with multi-layer perceptron (NN), and sup-
tial predictor and the dummy target variable (bank- port vector machine (SVM) methods. In fact, SVM
ruptcy = 1, non-bankruptcy = 0) are calculated. The is a part of the family of NN algorithms (András
variables with a correlation above a specified thresh- 2002). These methods can be classified into three
old are selected in the first step. The variables which classes with respect to sophistication. DT and GB
are selected in the first step enter the second step of belong to the lowest level, while NN and SVM
variable selection in which a sequential forward selec- belong to the highest level of sophistication. LR
tion process is used. This process starts by selecting can be classified into the middle level of sophistica-
the predictor variable that has the highest correlation tion. The methods are only briefly described here
coefficient with the target variable. A regression because they have been reviewed in detail in recent
model is estimated with the selected predictor. studies (Chen 2011a, 2011b; Erdogan 2013; du
Then, at each successive step of the sequence, an Jardin 2015), and the exact descriptions are pre-
additional predictor that provides the largest incre- sented in the SAS EM Internet support pages. If not
mental contribution to the model R-square is added otherwise explicitly reported, the results were run
to the regression. If the lower bound for the incre- with default values of the SAS EM options.3
mental contribution to the model R-square is DT is a non-linear discrimination method which
reached, the selection process stops. uses a set of independent variables to split a sample

1
The descriptive statistics for financial and non-financial variables are available from the authors upon request.
2
Exact descriptions of the selected methods can be found in the SAS EM support pages ([Link] The filter
method is included in the variable selection node, whereas the wrapper method is located in the regression node.
3
NN and SVM are run using the high-performance data mining nodes of the SAS EM 14.1, software, also with default values.
6 E. I. ALTMAN ET AL.

into progressively smaller subgroups. The proce- NN is a collection of computational elements

dure is iterative at each branch of the tree, and it where neurons are interconnected. The basic com-
selects the independent variable that has the stron- putational structure is comprised of three layers of
gest association with the dependent variable neurons: the input layer (independent variables),
according to a specified criterion (Chen 2011b). the hidden layer, and the output (bankrupt or non-
Thus, an empirical tree represents a segmentation bankrupt) layer. In addition to these neurons, the
of the data that is created by applying a series of network is also composed of connections between
simple rules. Each rule assigns an observation to the layers. The number and patterns of these con-
a segment based on the value of one input. Then, nections determine the task a network is capable of
one rule is applied after another, resulting in performing. We apply here a version of NN called
a hierarchy of segments within segments (a tree). multilayer perceptron (MLP), which is
The hierarchy is called a tree, and each segment is a feedforward artificial neural network model.
called a node. The final nodes are called leaves. For MLP consists of multiple layers of nodes in
each leaf, a decision is made and applied to all a directed graph, with each layer fully connected
observations in the leaf. In predictive modelling to the next one. Except for the input nodes, each
such as bankruptcy prediction, the decision is the node is a neuron (or processing element) with
predicted value. a nonlinear activation function. MLP utilizes
Like decision trees, GB does not make assump- a supervised learning technique called backpropa-
tions about the distribution of the data. It is gation for training the network.
a machine learning technique for classification pro- Support Vector Machine (SVM) is a machine
blems that produces a prediction model in the form learning algorithm for classifying high-dimensional
of an ensemble of weak prediction models, typically data. SVM uses a linear model to implement non-
decision trees. It builds the model in a stagewise linear class boundaries by mapping input vectors
fashion and generalizes it by allowing optimization nonlinearly into a high-dimensional feature space
of an arbitrary differentiable loss function. It (Chen 2011a). SVM has also been shown to be very
resamples the data several times to generate results resistant to the overfitting problem, eventually
that form a weighted average of the resampled data achieving high-generalization performance in sol-
set. Tree boosting creates a series of decision trees ving various forecasting and classification problems.
that form a single predictive model. Boosting is less Training SVM is equivalent to solving a linearly-
exposed to overfitting the data than is a single constrained quadratic programming problem, so
decision tree. If a decision tree fits the data rela- that the SVM solution is always unique and globally
tively well, then boosting often improves the fit. optimal, unlike other training that carries the risk of
LR is based on the assumption that the probability getting into local optima.
of bankruptcy is related to the independent variables
through a logistic link function. It can be used to
Competing models and performance evaluation
predict a binary dependent variable (bankrupt or
criteria
non-bankrupt) and to determine the (pseudo) per-
centage of variance in the dependent variable The set of competing models includes three logistic
explained by the independent variables (predictors). regression variants, three neural network models,
This analysis does not require that the distributions SVM, DT and GB. The first model, denoted by LR-
of independent variables be multivariate normal or ALL, contains all 35 predictor variables in the study
that groups have equal covariance matrices, which (see Appendix 1). The second one, LR-W is based
are basic assumptions in linear discriminant analysis on the variables chosen by the wrapper method,
(Hosmer and Lemeshow 1989). LR creates a score and the third one, LR-F, has the variables selected
(logit) for every firm. It is assumed that the indepen- by the filter method. Correspondingly, NN-ALL
dent variables are linearly related to the logit. This receives all potential variables of the study, NN-W
(risk) score is used to determine the probability of gets variables selected by the preceding wrapper
membership in bankrupt firms through the logistic procedure, and NN-F has the variables given by
function. the filter method. SVM utlizes all available variables
 APPLIED ECONOMICS 7

and is therefore comparable to the all-variable ver- money to all the firms scoring below a specific cut-
sions LR-ALL and NN-ALL. Wrapper and filter off point, and we calculate the profits of the credit
versions of SVM are not reported, however, for decision (cost-return) for the sample firms for each
brevity. Finally, DT and GB initially receive all method and each period, contrasting the profits
available variables, but (in our data) the final mod- generated in each case and comparing them with
els do not contain all variables.4 the naive policy of lending to all the firms. If the
When interpreting the results, the multivariate potential customer’s score is above the cut-off, the
prediction (classification) accuracy for different credit is declined and no transactions are carried
horizons plays the key role. The models are esti- out, leading to zero cost or profit. Finally, we search
mated for four different horizons: short term for the cut-off point giving the highest profits
(1 year), middle term (2–3 years), long term (4– (profit maximization). We use all the data to calcu-
5 years), and very long term (6–10 years). Each of late these profits (estimation, validation, and test
the bankruptcy prediction models is estimated samples) to increase the dispersion of probability
using randomly selected estimation (training), vali- value points in the data. The absolute amounts of
dation, and test samples (40% + 30% + 30% of the cost and return are not relevant to our analysis
total sample). Classification accuracy of the models since only the cost-return ratio is important.
is measured by the AUC (Area Under Curve) mea-
sure extracted from the ROC (Receiver Operating IV. Empirical results
Characteristic curve). These profiles show the
AUC results
trade-offs between Type I and Type II errors and
statistically represent the cumulative probability Table 1 (Panel 1) presents the accuracy of the statis-
distribution of default events. We report AUCs tical methods in terms of AUC for each length of the
only for the test data. horizon. Comparing the three LR approaches, the
We estimate each model so that the bankrupt LR-ALL versions that are run without a preceding
and non-bankrupt portions are equally weighted in variable selection procedure (thus including all our
the estimations, whereby the cut-off pseudo- potential predictor variables) give the highest AUCs
probability of each model is located at 0.50. We for all the prediction horizons. In the same way, the
find this weighting procedure necessary in order to LR model based on the wrapper method (LR-W)
make the analysis of alternative cut-off points of outperforms the version employing the filter method
competing models across different horizons feasi- (LR-F), which supports P1a. The difference in AUCs
ble in the profit-maximization results section. In between the wrapping and filter methods also
practice, the accuracy and profitability of the increases with the length of the horizon, supporting
adopted prediction model depend on the choice P1b. For the NN models, the interpretation of the
of the specific cut-off point with associated Type results is similar with respect to the research propo-
I and Type II errors. Weiss and Capkun (2005) sitions. For each horizon, the wrapper method (NN-
stressed that the superiority of one model over W) gives a higher AUC than the filter method (NN-
another cannot be fully measured unless the costs F), thereby supporting P1a. Moreover, the difference
of errors are taken into account. in accuracy increases with the horizon, which is
It is assumed that the lender receives returns consistent with P1b. However, the all-variables ver-
from its non-bankrupt customers but suffers losses sion of NN (NN-ALL) outperforms the wrapper
from its bankrupt customers. Therefore, we assume version (NN-W) for only the one-year horizon.
that if the users of the model lend to a firm which The differences in AUCs among the best-
does not go bankrupt, they get 1 unit of money as performing methods are small. The one-year horizon
return. If credit is given to a firm which goes bank- AUCs of all LR and NN versions are nearly equal and
rupt, the users will lose from 1 to 100 units of comparable with the AUC of the support vector
money. Then, we assume that the users lend machine method (SVM). SVM and LR-W give almost

4
Considering the short term horizon versions, DT and GB use 20 out of the total number of 35 potential predictors. The wrapper versions have 11 variables,
whereas the filter method selects only 7 predictors, remarkably excluding the equity ratio.
8 E. I. ALTMAN ET AL.

Table 1. The classification accuracy of alternative methods as measured by AUC in the test data.
Panel 1. AUC in the test data.
Horizon Year 1 Years 2-3 Years 4-5 Years 6-10 Average
LR-ALL 0,9340 0,8380 0,7810 0,7390 0,8230
LR-W 0,9330 0,8250 0,7680 0,7360 0,8155
LR-F 0,9240 0,8220 0,6480 0,7100 0,7760
NN-ALL 0,9400 0,8380 0,7750 0,7350 0,8220
NN-W 0,9320 0,8430 0,7800 0,7360 0,8228
NN-F 0,9220 0,8230 0,6480 0,7150 0,7770
SVM 0,9350 0,8220 0,7620 0,7360 0,8138
GB 0,8930 0,7590 0,7730 0,6810 0,7765
DT 0,7040 0,7110 0,6830 0,6950 0,6983
Average 0,9019 0,8090 0,7353 0,7203 0,7916
Panel 2. Ranks of methods.
Horizon Year 1 Years 2-3 Years 4-5 Years 6-10 Sum of ranks Final rank
LR-ALL 3 2 1 1 7 1
LR-W 4 4 5 2 15 4
LR-F 6 6 8 7 27 7
NN-ALL 1 2 3 5 11 3
NN-W 5 1 2 2 10 2
NN-F 7 5 8 6 26 6
SVM 2 6 6 2 16 5
GB 8 8 4 9 29 8
DT 9 9 7 8 33 9
Legend:
LR-ALL = Logistic regression with all variables
LR-W = Logistic regression with wrapper method
LR-F = Logistic regression with filter method (R-Square based variable selection)
NN-ALL = Neural network with all variables
NN-W = Neural network with wrapper method
NN-F = Neural network with filter method (R-Square based variable selection)
SVM = High performance support vector machine
GB = Gradient boosting
DT = Decision tree

equal AUCs for the different horizons. Similarly, the performance order, but the difference compared to
AUCs given by the methods based on the filter meth- the AUC of the best method is negligible (0.008). It is
ods (LR-F and NN-F) differ only slightly. The AUCs somewhat surprising that the all-variables version of
of these methods for the horizon of 4–5 years are LR performs so well. A probable reason contributing
exceptionally low (0.6480), reflecting the poor perfor- to this is that some of those variables not selected by
mance of the filtering method for this horizon. On the stepwise procedure still contribute to prediction
average, DT gives low AUCs of around 0.70 for each (significance close to the level required to enter the
horizon. However, classification accuracy does not model) and that due to the winsorization procedures,
decline with the prediction horizon but rather is there are no strongly exceptional, outlying values in
almost stable. Both GB and DT clearly give the lowest the unselected variables. We emphasize, however, that
AUCs for the sample. These methods give low AUCs we do not propose abandoning variable selection
already in the one-year horizon, contradicting P2a. procedures in model development.
The most sophisticated methods (NN, SVM, and also The gradient boosting (GB) and decision tree
LR) give higher AUCs for the longer periods, support- (DT) methods have low rankings for all the horizons
ing P2c, but the difference in AUCs between the more and get ranked to the last positions in overall final
and less sophisticated methods does not increase with accuracy classification. The models based on the
the length of the horizon, contradicting P2b. filter variable selection method (LR-F and NN-F)
Panel 2 of Table 1 presents the ranks of the meth- are clearly outperformed by the all-variables models
ods based on AUCs for all horizons and the final ranks (LR-ALL and NN-ALL), the wrapper method ver-
(according to the sum of ranks) over all the horizons. sions (LR-W and NN-W), and by the SVM. On the
The final ranks show that the LR-ALL and NN-W basis of generic classification accuracy (AUC), there
methods perform well for each horizon. The rank of are both statistical and data mining method versions
NN-W for the one-year horizon is only fifth in the among the top ranks.
 APPLIED ECONOMICS 9

Profit-maximization results horizon. For this reason, the ‘credit to all’ decision
leads to negative profits for the period of 6–10 years,
Detailed results for the selection of the optimal
when the cost-return ratio is 50:1 or higher. The
(profit maximizing) cut-off scores are reported sepa-
lower the percentage of bankrupt firms, the smaller
rately for the four horizons in Appendices 2–5. The
the differences in earned profit percentage among
results (cut-off, profit, and percentage of bankrupt
different models. For the period of 4–5 years, these
and non-bankrupt firms correctly classified) are cal-
differences are very small, except for very high cost-
culated for six different combinations of cost (lend-
return ratios.
ing to bankrupt firms) and return (lending to non-
For simplicity, we can limit our analysis and
bankrupt firms) so that the cost to return ratio varies
assume that the cost to return ratio is in practice
from 1 to 100. Table 2 reports a summary of the
30:1, which is close to the value (35:1) suggested by
appendices, presenting the profit at each method’s
Altman, Haldeman, and Narayanan (1977). For the
optimal cut-off as a percentage relative to the
one-year horizon, all the statistical models in this
hypothetical perfect credit decision profit. The per-
case (30:1) clearly beat the naive ‘credit to all’ rule.
fect decision means here that credit would be
However, only LR-ALL and NN-W models are able
granted to each non-bankrupt firm but to none of
to return more than 90% of the maximum profit.
the bankrupt firms. For each horizon, the number of
GB is outperformed by DT but, in general, the
non-bankrupt firms is constant (46,949), which
differences in the profit (as a percentage of the
yields 46,949 units of money as return (maximum
hypothetical perfect model profit) among all the
profit). For example, Table 4 (Panel 1) shows that for
methods are quite small (in the 90.28–84.89
the one-year horizon model, with a cost-return ratio
range). Table 3 presents the percentage of profit
of 100:1, the profit yielded by the LR-ALL model is
that a method earns in comparison with the profit
79.02% of the maximum profit. This method yields
of the best method for the cost to return ratio and
37,100 units of money with its optimal cut-off score
the horizon in question. Thus, for the ratio of 30:1
of 0.560 (see Appendix 2), which makes just 79.02%
and the horizon of one year, GB earns 91.55% of
of the maximum profit. It clearly beats the ‘credit to
the profit (Panel 1 of Table 3) yielded by the best
all’ decision (last column), which only gets 24.6% of
method (LR-ALL) in that case. This percentage,
the maximum profit for this cost-return ratio.
compared to the best model profit, reflects the
When the cost-return ratio declines from 100:1
loss incurred by not using the best method at hand.
towards 1:1, the profit as a percentage of maximum
Considering the 2–3 year horizon with the cost
profit will approach almost 100%, irrespective of the
to return ratio of 30:1, the methods which give the
method and the horizon. At the same time, the
highest profit are the same as for the one-year
optimal cut-off score rises and approaches the
horizon (LR-ALL, NN-ALL, LR-W, and NN-W),
value of 1. Due to the low costs of Type I errors,
but here DT performs exceptionally well, earning
the ‘credit to all’ policy also yields a profit close to
97.34% of the profit given by LR-ALL (Panel 2). In
100% of the maximum when the cost-return ratio is
this case, DT clearly outperforms GB (89.58%). It
close to 1. The percentage of profit earned for dif-
also beats the profit yielded by the models based on
ferent horizons strongly depends on the number
the filter method (LR-F and NN-F). When the cost-
(percentage) of bankruptcies in the sample within
return ratio is 10:1, DT yields the highest profit.
the period. If there are plenty of bankrupt firms and
For the 4–5 year horizon with the cost to return
the cost-return ratio is high, the percentage of profit
ratio of 30:1, all methods except DT yield more than
earned will be low due to high costs of Type I errors.
99% of the profit of NN-W (100%) (see Panel 3). DT
The numbers of bankrupt firms in this study for the
here earns the same profit as the naive ‘credit to all’
horizons of 1, 2–3, 4–5, and 6–10 years are, respec-
rule. For this horizon, GB earns very high profits for
tively, 354, 438, 225, and 950, leading to percentages
each cost-return ratio. For the ratio of 100:1, GB
of 0.75, 0.93, 0.48, and 2.02. These percentages,
earns the highest profit of all methods.
together with the accuracies of the models, explain
Then, for the horizon of 6–10 years with the
why the profit (percentage) figures are so high for
ratio of 30:1, GB performs worst, yielding only
the 4–5 year horizon and so low for the 6–10 year
82.60% of the maximum profit earned by the best
10 E. I. ALTMAN ET AL.

Table 2. Profit of alternative methods as percentage of perfect credit decision profit.

Cost = 100, Cost = 70, Cost = 50, Cost = 30, Cost = 10, Cost = 1,
Cost/Return Return = 1 Return = 1 Return = 1 Return = 1 Return = 1 Return = 1
Panel 1. Horizon 1 year.
LR-ALL 79,02 83,23 86,55 90,28 95,32 99,29
LR-W 77,84 82,57 85,72 89,18 95,15 99,27
LR-F 76,02 81,13 84,73 88,56 94,96 99,26
NN-ALL 79,27 83,21 85,94 89,98 95,18 99,27
NN-W 79,73 83,69 86,51 90,14 95,06 99,29
NN-F 77,46 81,80 85,25 88,80 94,95 99,28
SVM 77,59 81,36 84,60 88,40 95,04 99,27
GB 69,11 74,20 79,23 84,89 93,39 99,25
DT 61,66 72,46 79,65 86,85 94,05 99,25
Credit to all 24,60 47,22 62,30 77,38 92,46 99,25
Panel 2. Horizon 2–3 years.
LR-ALL 58,90 65,52 71,93 80,39 91,56 99,07
LR-W 56,50 63,90 70,96 79,78 91,54 99,07
LR-F 52,14 60,35 68,26 77,68 91,13 99,07
NN-ALL 61,22 67,29 73,09 80,27 91,54 99,07
NN-W 58,47 65,69 71,73 79,77 91,64 99,07
NN-F 52,03 60,74 68,66 78,04 91,28 99,07
SVM 55,85 64,74 70,73 79,20 91,54 99,07
GB 49,07 55,46 62,96 72,01 92,80 99,07
DT 52,93 62,65 69,12 78,25 92,88 99,07
Credit to all 6,71 34,70 53,35 72,01 90,67 99,07
Panel 3. Horizon 4–5 years
LR-ALL 67,09 74,87 80,24 86,90 95,25 99,52
LR-W 66,47 73,88 80,06 87,15 95,34 99,52
LR-F 62,52 72,51 79,20 86,67 95,24 99,52
NN-ALL 63,88 72,73 79,58 86,97 95,21 99,52
NN-W 66,37 73,59 79,94 87,43 95,28 99,52
NN-F 62,91 72,51 79,20 86,65 95,21 99,52
SVM 64,58 72,76 79,32 86,65 95,23 99,52
GB 68,23 74,82 80,21 86,71 95,24 99,52
DT 66,95 72,81 78,47 85,62 95,21 99,52
Credit to all 52,08 66,45 76,04 85,62 95,21 99,52
Panel 4. Horizon 6–10 years.
LR-ALL 22,78 30,97 40,94 54,28 80,48 97,98
LR-W 21,68 30,24 39,29 53,11 80,28 97,98
LR-F 17,65 24,56 34,22 47,36 79,85 97,98
NN-ALL 22,71 31,69 40,47 55,20 80,66 97,98
NN-W 22,44 31,03 39,48 54,55 80,43 97,98
NN-F 18,59 26,85 35,60 48,35 80,08 97,98
SVM 17,02 28,51 38,73 54,24 80,37 97,98
GB 12,64 22,84 30,59 45,60 79,77 98,89
DT 12,39 25,25 36,87 52,74 80,20 99,43
Credit to all −102,35 −41,64 −1,17 39,30 79,77 97,98
Legend:
LR-ALL = Logistic regression with all variables
LR-W = Logistic regression with wrapper method
LR-F = Logistic regression with filter method (R-Square based variable selection)
NN-ALL = Neural network with all variables
NN-W = Neural network with wrapper method
NN-F = Neural network with filter method (R-Square based variable selection)
SVM = High performance support vector machine
GB = Gradient boosting
DT = Decision tree

method (NN-ALL). For this horizon, the models (Table 1). LR-ALL, NN-ALL, and NN-W share
based on the filter method (LR-F and NN-F) also the first places, and GB and DT take the last ones.
yield exceptionally low profits, whereas DT yields Moreover, the models based on the filter method
satisfactory results. (LR-F and NN-F) are outperformed by SVM. SVM
Table 4 presents the ranks of different prediction and DT rank highest for the horizons of 6–10 and
methods in earning profit for different horizons 2–3 years, when the percentage of bankrupt firms is
using the (justified) cost-return ratio of 30:1 as an the highest. However, GB’s best rank (5) is for the
exemplary value. The table shows that these ranks 4–5 year horizon, when the percentage is the low-
are consistent with the ranks based on AUCs est. LR-F and NN-F reach quite low ranks for each
 APPLIED ECONOMICS 11

Table 3. Profit of alternative methods as percentage of best method profit.

Cost = 100, Cost = 70, Cost = 50, Cost = 30, Cost = 10, Cost = 1,
Cost/Return Return = 1 Return = 1 Return = 1 Return = 1 Return = 1 Return = 1
Panel 1. Horizon 1 year.
LR-ALL 99,11 99,45 100,00 100,00 100,00 100,00
LR-W 97,63 98,66 99,05 98,78 99,83 99,98
LR-F 95,35 96,95 97,90 98,09 99,63 99,97
NN-ALL 99,43 99,43 99,30 99,66 99,85 99,98
NN-W 100,00 100,00 99,96 99,84 99,73 100,00
NN-F 97,15 97,75 98,51 98,36 99,61 99,99
SVM 97,32 97,22 97,75 97,91 99,71 99,98
GB 86,68 88,66 91,55 94,03 97,98 99,96
DT 77,33 86,58 92,04 96,20 98,67 99,95
Credit to all 30,85 56,42 71,98 85,71 97,00 99,95
Panel 2. Horizon 2–3 years.
LR-ALL 96,20 97,37 98,41 100,00 98,58 100,00
LR-W 92,29 94,96 97,08 99,25 98,56 100,00
LR-F 85,16 89,68 93,39 96,63 98,12 100,00
NN-ALL 100,00 100,00 100,00 99,85 98,56 100,00
NN-W 95,50 97,61 98,14 99,23 98,67 100,00
NN-F 84,98 90,26 93,94 97,08 98,28 100,00
SVM 91,22 96,21 96,77 98,52 98,56 100,00
GB 80,14 82,42 86,14 89,58 99,92 100,00
DT 86,46 93,09 94,57 97,34 100,00 100,00
Credit to all 10,96 51,56 73,00 89,58 97,62 100,00
Panel 3. Horizon 4–5 years
LR-ALL 98,33 100,00 100,00 99,39 99,90 100,00
LR-W 97,42 98,68 99,78 99,68 100,00 100,00
LR-F 91,64 96,85 98,71 99,13 99,89 100,00
NN-ALL 93,63 97,15 99,19 99,47 99,86 100,00
NN-W 97,27 98,29 99,63 100,00 99,93 100,00
NN-F 92,21 96,85 98,71 99,11 99,86 100,00
SVM 94,66 97,19 98,86 99,10 99,88 100,00
GB 100,00 99,93 99,97 99,17 99,89 100,00
DT 98,13 97,25 97,80 97,93 99,86 100,00
Credit to all 76,33 88,76 94,77 97,93 99,86 100,00
Panel 4. Horizon 6–10 years.
LR-ALL 100,00 97,71 100,00 98,33 99,77 100,00
LR-W 95,18 95,42 95,97 96,21 99,52 100,00
LR-F 77,48 77,49 83,59 85,79 98,99 100,00
NN-ALL 99,70 100,00 98,87 100,00 100,00 100,00
NN-W 98,52 97,91 96,44 98,81 99,72 100,00
NN-F 81,61 84,73 86,97 87,59 99,27 100,00
SVM 74,74 89,96 94,60 98,25 99,64 100,00
GB 55,48 72,06 74,73 82,60 98,89 100,00
DT 54,39 79,67 90,06 95,54 99,43 100,00
Credit to all −449,28 −131,40 −2,87 71,18 98,89 100,00
Legend:
LR-ALL = Logistic regression with all variables
LR-W = Logistic regression with wrapper method
LR-F = Logistic regression with filter method (R-Square based variable selection)
NN-ALL = Neural network with all variables
NN-W = Neural network with wrapper method
NN-F = Neural network with filter method (R-Square based variable selection)
SVM = High performance support vector machine
Boosting = Gradient boosting
Decision Tree = Decision tree

horizon, indicating the obvious inefficiency of the horizon, which is consistent with P1b. For the one-
filter method. year horizon, assuming a reasonable cost to return
In summary, these specific results for profit max- ratio (such as 30:1), the differences in profit earned
imization indicate that for any horizon, the wrapper among the statistical methods are quite small (the
method performs better than the filter method, con- worst model earns 94.03% of the highest profit). This
forming to P1a. For the shortest horizon, the differ- result lends support to P2a. The differences in profit
ences in profit earning among the methods are earned among the methods increase with the pre-
small, but they increase with the length of the diction horizon, supporting P2b. The most
12 E. I. ALTMAN ET AL.

Table 4. Ranks of methods according to earned profits for cost- a large number of variables, the LR model consist-
return ratio 30:1. ing of all the variables outperformed the LR ver-
Horizon
Year
1
Years
2-3
Years
4-5
Years
6-10
Sum of
ranks
Final
rank
sions where wrapper or filter variable selection
LR-ALL 1 1 4 3 9 1 methods were used. For both LR and NN, the
LR-W 4 3 2 5 14 4 wrapper method (logistic stepwise selection) gave
LR-F 6 8 6 8 28 7
NN-ALL 3 2 3 1 9 1 higher AUCs than the filter method (the R-squared
NN-W 2 4 1 2 9 1 method) for all the horizons. The longer the hor-
NN-F 5 7 7 7 26 6
SVM 7 5 8 4 24 5 izon, the larger was the difference in AUCs among
GB 9 9 5 9 32 9
DT 8 6 9 6 29 8
the selection methods. For NN, the naive all-
variables model and the version based on the wrap-
per method gave comparable AUCs, although on
sophisticated models, NN-ALL and NN-W, perform average, AUCs of the wrapper version slightly sur-
best with a long horizon, but LR-ALL and LR-W are passed those of the all-variables model. SVM was
comparable with SVM, which is more sophisticated very efficient with a very short or a very long
than the logistic regression models. GB and DT earn horizon. On average, SVM clearly outperformed
quite low profits in long-term prediction. Thus, the the filter versions of LR and NN. In our compar-
results lend support to P2c. ison, GB and DT clearly gave the lowest AUCs for
most horizons.
V. Summary and conclusions The AUC results appear to be in contrast with
those in Jones, Johnstone, and Wilson (2017), in
Failure prediction models are widely used by many which generalized boosting (which is equivalent to
stakeholders in predicting a failure emerging in our GB), AdaBoost, and random forests (RF)
terms of payment default, bankruptcy, or a similar showed superior performance compared to more
event. The higher the accuracy for a given prediction conventional methods, including NN and stepwise
horizon, the more valuable the model. Its value is LR. The same conclusions were reached by Barboza,
also dependent on the cost of Type I and II errors in Kimura, and Altman (2017). There are several
classifying bankrupt and non-bankrupt firms potential sources for these contradicting results.
(Altman, Haldeman, and Narayanan 1977; Weiss First of all, our data consist of small Finnish SMEs,
and Capkun 2005). whereas theirs are composed of public
The first purpose of our study was to assess the U.S. companies. Secondly, we used, except in the
performance of different estimation methods in variable selection, default options of the software.
a large cross-sectional sample of Finnish firms Thus, the differences in options (or in software) may
over a 10-year period. We applied five different have contributed to the disparities. Our poorest
methods for bankruptcy prediction using both results are for the DT. As GB leans on an ensemble
financial and non-financial variables: DT, GB, LR, of decision tree models, poor DT performance may
NN, and SVM. The majority of firms in our sample be a contributing factor to the modest performance
are small and medium-sized enterprises (SMEs). of GB in our study. It is worth noting that in three of
Thus, our study is a response to the call for small the four horizons examined, GB was still able to
business studies based on financial and non- produce remarkably better AUCs than DT.
financial variables. The findings contribute to the Thirdly, we used rather conventional financial state-
previous research on variable selection methods ment variables and many non-financial predictors,
(Acosta-González and Fernández-Rodríguez 2014; whereas they used mainly financial variables, of
Back, Laitinen, and Sere 1996a, 1996b; du Jardin which the best-performing ones (in the RF model)
2010) and estimation methods (Laitinen and were highly exceptional, the top three being annual
Kankaanpää 1999; Chen 2011a; du Jardin 2015, growth in capital expenditures, annual growth in
2016; Gordini 2014; Barboza, Kimura, and leverage-free cash flow, and earnings per share.
Altman 2017; Jones, Johnstone, and Wilson 2017). Furthermore, only conventional variables were sig-
We first analysed the accuracy of the methods in nificant in their standard logit model. It would be
terms of AUCs in the test data. Although we had very interesting to test whether these kinds of
 APPLIED ECONOMICS 13

exceptional variables can perform well in the bank- methods can be expanded to such methods as the
ruptcy prediction of SMEs, for which conventional chi-square method or a genetic algorithm
variables are even less stable than for larger public approach that is naturally connected with NN. In
companies. Unfortunately, such variables are not the estimation, more methods should be assessed,
available in most SME company databases. including such methods as survival analysis,
The second purpose of the study was to assess rough set approach, LARS (least-angle regres-
the value of methods using the cost-return ratio for sions), LASSO (least absolute shrinkage and selec-
classification. When this ratio is very small tion operator), AdaBoost, and other ‘new age’
(around 1), all the methods earn almost 100% of methods. Secondly, our analysis is based on data
the maximum profit for any horizon. In this in which the percentage of bankrupt firms
extreme case, the naive ‘credit to all’ rule is also strongly varies with the horizon, which affects
efficient due to the low relative cost of Type I error. the performance of some methods. In further
In more normal cost-return cases, we found that research, this effect should be eliminated using
the value of a method is strongly dependent on the equal percentages for each horizon. Finally, it
percentage of bankrupt firms in the sample. This is would be useful to assess the statistical signifi-
due to the high relative Type I error cost and to cance of the differences among different methods.
differences in error rates among different methods. Moreover, studies that screen the sensitivity of
For simplicity, we discussed in detail the empiri- performance to alterations in the options that
cally justified case with the 30:1 cost-return ratio. are available for various methods are needed to
We found that in this case, the LR and NN all- further improve the ability to predict bankruptcies
variables versions and the NN wrapper version of large firms, public companies, and SMEs.
shared the best overall rank, while GB, DT, and
the filter versions of LR and NN had the lowest
Disclosure statement
ranks. SVM and DT had their best ranks for the
horizons of 5–10 and 2–3 years, when the percen- No potential conflict of interest was reported by the authors.
tage of bankrupt observations was the highest.
However, GB reached its best rank for the
ORCID
4–5 year horizon, when the percentage was lowest.
Thus, our empirical findings imply that for small Małgorzata Iwanicz-Drozdowska [Link]
businesses, an efficient failure prediction model can 0002-8490-5178
be based on a very large set of both financial and
non-financial variables. Moreover, an LR model can Data availability statement
perform as efficiently as the more sophisticated NN
and even better than SVM. If a variable selection The data described in this article are openly available in the
method is used, the wrapper method seems to be Open Science Framework at DOI:10.17605/[Link]/TPA6U.
superior to the filter method for all the horizons. For
LR and NN, the wrapper method and the all- References
variables versions give comparable results.
Therefore, if the user of the model prefers a simple Acosta-González, E., and F. Fernández-Rodríguez. 2014.
“Forecasting Financial Failure of Firms via Genetic
version with few variables to the all-variables ver-
Algorithms.” Computational Economics 43:133–157.
sion, a wrapper variable selection method is strongly Altman, E. 1968. “Financial Ratios, Discriminant Analysis and
recommended. These results are not sensitive to the the Prediction of Corporate Bankruptcy.” Journal of
length of the prediction horizon in this study. Finance. 23:589–609.
In summary, our study has many important Altman, E., R. Haldeman, and P. Narayanan. 1977. “Zeta
implications for failure prediction in the small Analysis: A New Model to Identify Bankruptcy Risk of
Corporations.” Journal of Banking and Finance. 1:29–54.
business context. However, it also has limitations
Altman, E., M. Iwanicz-Drozdowska, E. Laitinen, and
which can be mitigated in further studies. Firstly, A. Suvas. 2016. “Financial and Nonfinancial Variables as
we limited our analysis to only a small set of Long-horizon Predictors of Bankruptcy.” Journal of Credit
methods. In further studies, the variable selection Risk. 12:49–78.
14 E. I. ALTMAN ET AL.

Altman, E., and G. Sabato. 2007. “Modelling Credit Risk for Do Prado, J., V. de Castro Alcantara, F. de Melo Carvalho,
SMEs: Evidence from the U.S. Market.” Abacus. K. Vieira, L. Machado, and D. Tonelli. 2016. “Multivariate
43:332–357. Analysis of Credit Risk and Bankruptcy Research Data:
Altman, E., G. Sabato, and N. Wilson. 2010. “The Value of A Bibliometric Study Involving Different Knowledge
Non-financial Information in Small and Medium-sized Fields (1968-2014).” Scientometrics. 106:1007–1029.
Enterprise Risk Management.” Journal of Credit Risk. Du Jardin, P. 2009. “Bankruptcy Prediction Models: How to
6:1–33. Choose the Most Relevant Variables?” Bankers, Markets &
András, P. 2002. “The Equivalence of Support Vector Investors. 98:39–46.
Machine and Regularization Neural Networks.” Neural Du Jardin, P. 2010. “Predicting Bankruptcy Using Neural
Processing Letters. 15:97–104. Networks and Other Classification Methods: The
Argenti, J. 1976. Corporate Collapse; the Causes and Influence of Variable Selection Techniques on Model
Symptoms. Wiley, New York: McGraw Hill. Accuracy.” Neurocomputing. 73:2047–2060.
Back, B., T. Laitinen, and K. Sere. 1996a. “Neural Networks Du Jardin, P. 2015. “Bankruptcy Prediction Using Terminal
and Bankruptcy Prediction: Funds Flows, Accrual Ratios Failure Processes.” European Journal of Operational
and Accounting Data.” Advances in Accounting. 14:23–37. Research. 242:286–303.
Back, B., T. Laitinen, and K. Sere. 1996b. “Neural Networks Du Jardin, P. 2016. “A Two-stage Classification Technique for
and Genetic Algorithms for Bankruptcy Predictions.” Bankruptcy Prediction.” European Journal of Operational
Expert Systems with Applications. 11:407–413. Research. 254:236–252.
Back, P. 2005. “Explaining Financial Difficulties Based on Du Jardin, P., and E. Severin. 2011. “Predicting Corporate
Previous Payment Behavior, Management Background Bankruptcy Using a Self-organizing Map: An Empirical
Variables and Financial Ratios.” European Accounting Study to Improve the Forecasting Horizon of
Review. 14:839–868. a Financial Failure Model”. Decision Support Systems.
Balcaen, S., and H. Ooghe. 2006. “35 Years of Studies on 51:701–711.
Business Failure: An Overview of the Classic Statistical Du Jardin, P., and E. Severin. 2012. “Forecasting Financial
Methodologies and Their Related Problems.” The British Failure Using a Kohonen Map: A Comparative Study to
Accounting Review. 38:63–93. Improve Model Stability over Time.” European Journal of
Barboza, F., H. Kimura, and E. Altman. 2017. “Machine Operational Research. 221:378–396.
Learning Models and Bankruptcy Prediction.” Expert Edmister, R. 1972. “An Empirical Test of Financial Ratio
Systems with Applications. 83:405–417. Analysis for Small Business Failure Prediction.” Journal of
Beaver, W. 1966. “Financial Ratios as Predictors of Failure. Financial and Quantitative Analysis. 7:1477–1493.
Empirical Research in Accounting, Selected Studies.” Erdogan, B. 2013. “Prediction of Bankruptcy Using Support
Journal of Accounting Research. 4:71–111. Vector Machines: An Application to Bank Bankruptcy.”
Bellovary, J., D. Giacomino, and M. Akers. 2007. “A Review of Journal of Statistical Computation and Simulation.
Bankruptcy Prediction Studies: 1930 to Present.” Journal of 83:1543–1555.
Financial Education. 1:3–41. Flagg, J., G. Giroux, and C. Wiggins. 1991. “Predicting
Beniwal, S., and J. Arora. 2012. “Classification and Feature Corporate Bankruptcy Using Failing Firms.” Review of
Selection Techniques in Data Mining.” International Financial Economics. 1:67–78.
Journal of Engineering Research & Technology. 6:1–6. Gordini, N. 2014. “A Genetic Algorithm Approach for SMEs
Chava, S., and R. Jarrow. 2004. “Bankruptcy Prediction with Bankruptcy Prediction: Empirical Evidence from Italy.”
Industry Effects.” Review of Finance. 8:537–569. Expert Systems with Applications. 41:6433–6445.
Chen, M. Y. 2011a. “Bankruptcy Prediction in Firms with Hambrick, D., and R. D’Aveni. 1988. “Large Corporate
Statistical and Intelligent Techniques and a Comparison Failures as Downward Spirals.” Administrative Science
of Evolutionary Computation Approaches.” Computers Quarterly. 33:1–23.
and Mathematics with Applications. 62:4514–4524. Hillegeist, S., E. Keating, D. Cram, and K. Lundstedt. 2004.
Chen, M. Y. 2011b. “Predicting Corporate Financial Distress “Assessing the Probability of Bankruptcy.” Review of
Based on Integration of Decision Tree Classification and Accounting Studies. 9:5–34.
Logistic Regression.” Expert Systems with Applications. Hosmer, D., and S. Lemeshow. 1989. Applied Logistic
38:11261–11272. Regression. New York: Wiley.
D’Aveni, R. 1989. “The Aftermath of Organizational Decline: Jones, S., D. Johnstone, and R. Wilson. 2017. “Predicting
A Longitudinal Study of the Strategic and Managerial Corporate Bankruptcy: An Evaluation of Alternative
Characteristics of Declining Firms.” Academy of Statistical Frameworks.” Journal of Business Finance and
Management Journal. 32:577–605. Accounting. 44:3–34.
Dimitras, A., S. Zanakis, and C. Zopounidis. 1996. “A Survey Keasey, K., and R. Watson. 1987. “Non-financial Symptoms
of Business Failures with an Emphasis on Prediction and the Prediction of Small Company Failure: A Test of
Methods and Industrial Applications.” European Journal Argenti’s Hypotheses.” Journal of Business Finance and
of Operational Research. 90:487–513. Accounting. 14:335–354.
 APPLIED ECONOMICS 15

Kumar, P., and V. Ravi. 2007. “Bankruptcy Prediction in Moulton, W., H. Thomas, and M. Pruett. 1996. “Business Failure
Banks and Firms via Statistical and Intelligent Path Ways: Environmental Stress and Organizational
Techniques – A Review.” European Journal of Operational Response.” Journal of Management. 22:571–595.
Research. 180:1–28. Ohlson, J. 1980. “Financial Ratios and the Probabilistic
Laitinen, E. 1991. “Financial Ratios and Different Failure Prediction of Bankruptcy.” Journal of Accounting
Processes.” Journal of Business Finance and Accounting. Research. 18:109–131.
18:649–674. Ooghe, H., and S. De Prijcker. 2008. “Failure Processes and
Laitinen, E. 1999. “Predicting a Corporate Credit Analyst’s Causes of Company Bankruptcy: A Typology.”
Risk Estimate by Logistic and Linear Models.” International Management Decision. 46:223–242.
Review of Financial Analysis. 8:97–121. Reisz, A., and C. Perlich. 2007. “A Market-based Framework for
Laitinen, T., and M. Kankaanpää. 1999. “Comparative Bankruptcy Prediction.” Journal of Financial Stability.
Analysis of Failure Prediction Methods: The Finnish 3:85–131.
Case.” European Accounting Review. 8:67–92. Weiss, L., and V. Capkun. 2005. “The Impact of Incorporating
Moro, S., P. Cortez, and P. Rita. 2015. “Business Intelligence in the Cost of Errors into Bankruptcy Prediction
Banking: A Literature Analysis from 2002 to 2013 Using Models (March 14, 2005).” Available at SSRN: https://
Text Mining and Latent Dirichlet Allocation.” Expert [Link]/abstract=651261 or [Link]
Systems with Applications. 42:1314–1324. ssrn.651261

Appendices

Appendix 1. List of variables.

Notation Definition
FINANCIAL VARIABLES
1. Profitability
ROA (Profit after Interest and Taxes but before Extraordinary Items)/Total
Assets *100
RETA Retained Earnings/Total Assets *100
2. Liquidity
SHTDEBTtoASSETS ShortTerm Debt to Assets * 100
QUICKRATIO (Current Assets – Inventories)/Current Liabilities
3. Solvency
EQRATIO Book value of Equity/Total Assets *100
DEBTtoEBITDA Debt/EBITDA *100
4. Cash flow
TradCFtoASSETS (Profit after Interest and Taxes +Depreciation)/Total Assets *100
CFLOWtoASSETS Cash Flow/Total Assets *100
5. Size
SIZEA LN(Total Assets)
SIZESQ LN(Total Assets) squared
LNSALES LN(Sales)
6. Growth
ASSETG1Y LN(Assets(t)/Assets(t-1))/(Accounting period length in months)/12); One-
year Total Asset growth
SALESG1Y (Sales(t)/Sales(t-1) – 1) * 100, One-year growth rate of Sales
7. Changes in ratios
SHTDEBTtoASSETSch Short Term Debt/Total Assets -Lag(Short Term Debt/Total Assets)
TOTALDEBTchToASSETS (Change in Debt)/Total Assets
NON-FINANCIAL VARIABLES
1. Firm type
DAUGHTER DUMMY
PARENT DUMMY
2. Industry
WRDUMMY Wholesale-Retail Dummy
CONSTRDUMMY Construction Firm Dummy
3. Age
LOGofAGE LN(Age in years)
AGE3to9 Age 3 to 9 years
4. Industry risk
IND_BR_RISK Industry Bankruptcy Risk (% of firms)
IND_PDEF_RISK Industry Payment Default Risk (% of firms)
5. Audit report and disclosure policy
(Continued)
16 E. I. ALTMAN ET AL.

Appendix 1. (Continued).
Notation Definition
AUDITQUALIF Auditors report is qualified (not severe)
AUDITQUALIF_SEVERE Auditors report is qualified (severe)
LateFiling Postponed publication of the financial statement
6. Payment behaviour
DELAYSDUM The firm has delays recorded within 9 months, DUMMY
DelaysOver60 Number of payment delays over 60 days late
DELAYStoASSETS Delays(EUR)/Total Assets
LNPOSPAYOBS LN(Positive payment observations+1)
PRIOR_PDEFAULTS Number of payment default events prior to cross-section – all
7. Board members
BOARD_SIZE Number of board members (No deputy members)
RESIGNED Number of resigned persons in charge (the last 6 months)
LNBOARD_OWN_PDEFS Number of board member’s own payment defaults
PDBRCONNECTS Number of associations with payment default firms (other than
bankruptcy) by persons in charge

Appendix 2. Model efficiency comparisons; one year before bankruptcy; all data combined.
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
Cost = 100, Return = 1
LR-ALL 37,100 0,560 83,33 91,59 91,03
LR-W 36,546 0,624 79,10 93,60 93,06
LR-F 35,691 0,534 77,40 93,06 92,53
NN-ALL 37,218 0,657 82,77 92,27 91,71
NN-W 37,432 0,471 86,16 90,17 89,59
NN-F 36,365 0,532 82,20 90,88 90,33
SVM 36,429 0,527 83,33 90,16 89,61
GB 32,446 0,599 78,81 85,08 84,61
DT 28,947 0,301 52,26 97,65 97,28
Credit to all 11,549 1 0 100 100
Cost = 70, Return = 1
LR-ALL 39,075 0,664 78,53 94,56 94,01
LR-W 38,766 0,624 79,10 93,60 93,06
LR-F 38,091 0,534 77,40 93,06 92,53
NN-ALL 39,067 0,670 82,20 92,60 92,04
NN-W 39,291 0,616 82,49 92,93 92,37
NN-F 38,406 0,693 77,12 93,88 93,35
SVM 38,199 0,527 83,33 90,16 89,61
GB 34,837 0,747 66,67 91,80 91,36
DT 34,017 0,301 52,26 97,65 97,28
Credit to all 22,169 1 0 100 100
Cost = 50, Return = 1
LR-ALL 40,633 0,696 76,84 95,28 94,74
LR-W 40,246 0,624 79,10 93,60 93,06
LR-F 39,778 0,594 74,58 94,31 93,80
NN-ALL 40,347 0,682 81,36 92,97 92,41
NN-W 40,616 0,722 79,10 94,39 93,84
NN-F 40,026 0,693 77,12 93,88 93,35
SVM 39,718 0,575 74,86 94,08 93,56
GB 37,200 0,752 66,10 92,01 91,58
DT 37,397 0,301 52,26 97,65 97,28
Credit to all 29,249 1 0 100 100
Cost = 30, Return = 1
LR-ALL 42,387 0,770 71,47 96,73 96,22
LR-W 41,868 0,657 76,84 94,42 93,88
LR-F 41,578 0,594 74,58 94,31 93,80
NN-ALL 42,245 0,828 71,75 96,37 95,86
NN-W 42,318 0,851 71,75 96,53 96,02
NN-F 41,690 0,753 73,45 94,81 94,29
SVM 41,502 0,600 70,06 95,17 94,68
GB 39,855 0,829 53,95 95,31 94,94
DT 40,777 0,301 52,26 97,65 97,28
Credit to all 36,329 1 0 100 100
Cost = 10, Return = 1
LR-ALL 44,750 0,910 50,00 98,79 98,42
LR-W 44,672 0,915 49,72 98,94 98,58
LR-F 44,583 0,940 43,22 99,24 98,92
NN-ALL 44,685 0,914 58,19 98,33 97,91
(Continued)
 APPLIED ECONOMICS 17

Appendix 2. (Continued).
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
NN-W 44,631 0,928 45,20 99,19 98,86
NN-F 44,576 0,956 42,66 99,27 98,96
SVM 44,622 0,706 51,41 98,71 98,33
GB 43,846 0,918 28,25 98,80 98,60
DT 44,157 0,301 52,26 97,65 97,28
Credit to all 43,409 1 0 100 100
Cost = 1, Return = 1
LR-ALL 46,616 0,996 11,02 99,96 99,88
LR-W 46,607 0,995 9,89 99,95 99,88
LR-F 46,601 0,997 5,08 99,74 99,94
NN-ALL 46,608 0,993 12,43 99,93 99,84
NN-W 46,615 0,993 14,69 99,93 99,82
NN-F 46,610 0,993 8,76 99,97 99,90
SVM 46,605 0,855 7,91 99,96 99,90
GB 46,596 0,975 0,28 100,00 100,00
DT 46,595 0,971 0,00 100,00 100,00
Credit to all 46,595 1 0 100 100

Appendix 3. Model efficiency comparisons; years 2–3 before bankruptcy; all data combined.
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
Cost = 100, Return = 1
LR-ALL 27,651 0,508 76,71 80,62 80,09
LR-W 26,527 0,471 77,63 77,38 76,87
LR-F 24,479 0,473 75,34 75,14 74,68
NN-ALL 28,744 0,552 78,31 81,46 80,91
NN-W 27,450 0,557 74,20 82,54 82,01
NN-F 24,427 0,527 74,66 75,67 75,21
SVM 26,220 0,372 68,49 85,24 84,74
GB 23,036 0,475 83,33 64,61 64,17
DT 24,852 0,332 65,30 85,31 84,84
Credit to all 3149 1 0 100 100
Cost = 70, Return = 1
LR-ALL 30,763 0,592 68,26 86,25 85,75
LR-W 30,002 0,614 63,24 87,91 87,44
LR-F 28,335 0,550 59,59 86,74 86,31
NN-ALL 31,594 0,552 78,31 81,46 80,91
NN-W 30,840 0,557 74,20 82,54 82,01
NN-F 28,517 0,669 59,13 87,43 87,00
SVM 30,397 0,372 68,04 85,62 85,12
GB 26,040 0,543 59,82 81,71 81,32
DT 29,412 0,332 65,30 85,31 84,84
Credit to all 16,289 1 0 100 100
Cost = 50, Return = 1
LR-ALL 33,771 0,634 64,16 88,65 88,16
LR-W 33,314 0,635 61,19 89,06 88,60
LR-F 32,048 0,565 56,16 88,71 88,29
NN-ALL 34,315 0,716 64,84 89,49 88,99
NN-W 33,678 0,659 65,07 88,03 87,54
NN-F 32,234 0,689 55,48 89,43 89,01
SVM 33,206 0,372 66,44 86,38 85,89
GB 29,560 0,543 59,82 81,71 81,32
DT 32,452 0,332 65,30 85,31 84,84
Credit to all 25,049 1 0 100 100
Cost = 30, Return = 1
LR-ALL 37,741 0,804 44,52 95,91 95,54
LR-W 37,458 0,763 46,80 94,67 94,29
LR-F 36,470 0,774 36,07 95,57 95,28
NN-ALL 37,686 0,778 55,71 92,67 92,22
NN-W 37,450 0,797 47,72 94,40 94,01
NN-F 36,639 0,804 39,04 95,10 94,78
SVM 37,182 0,379 46,12 94,28 93,90
GB 33,809 0,614 0,00 100,00 100,00
DT 36,738 0,716 45,21 93,59 93,23
Credit to all 33,809 1 0 100 100
Cost = 10, Return = 1
LR-ALL 42,987 0,940 16,44 99,36 99,21
(Continued)
18 E. I. ALTMAN ET AL.

Appendix 3. (Continued).
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
LR-W 42,979 0,899 24,89 98,55 98,33
LR-F 42,786 0,933 10,73 99,46 99,37
NN-ALL 42,976 0,907 21,00 98,91 98,72
NN-W 43,024 0,949 16,44 99,44 99,29
NN-F 42,853 0,906 14,61 99,24 99,11
SVM 42,979 0,388 22,15 98,81 98,61
GB 43,569 0,614 0,00 100,00 100,00
DT 43,605 0,869 27,85 97,48 97,23
Credit to all 42,569 1 0 100 100
Cost = 1, Return = 1
LR-ALL 46,511 0,998 0,00 100,00 100,00
LR-W 46,511 0,998 0,00 100,00 100,00
LR-F 46,511 0,993 0,00 100,00 100,00
NN-ALL 46,512 0,983 0,23 100,00 100,00
NN-W 46,511 0,976 0,00 100,00 100,00
NN-F 46,512 0,975 0,23 100,00 100,00
SVM 46,511 0,417 0,00 100,00 100,00
GB 46,511 0,614 0,00 100,00 100,00
DT 46,511 0,925 0,00 100,00 100,00
Credit to all 46,511 1 0 100 100

Appendix 4. Model efficiency comparisons; years 4–5 before bankruptcy; all data combined..
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
Cost = 100, Return = 1
LR-ALL 31,497 0,742 46,22 92,86 92,67
LR-W 31,206 0,695 48,44 91,18 90,99
LR-F 29,353 0,452 31,11 95,54 95,41
NN-ALL 29,992 0,739 45,33 90,08 89,91
NN-W 31,158 0,575 66,67 82,34 82,11
NN-F 29,535 0,444 31,11 95,54 95,41
SVM 30,320 0,002 60,89 83,32 83,11
GB 32,031 0,697 56,00 89,31 89,10
DT 31,433 0,453 61,33 85,48 85,26
Credit to all 24,449 1 0 100 100
Cost = 70, Return = 1
LR-ALL 35,150 0,759 44,00 93,65 93,48
LR-W 34,686 0,695 48,44 91,18 90,99
LR-F 34,043 0,459 30,22 95,92 95,79
NN-ALL 34,148 0,804 31,56 95,70 95,57
NN-W 34,549 0,738 40,89 93,42 93,25
NN-F 34,043 0,451 30,22 95,92 95,79
SVM 34,161 0,002 31,56 95,72 95,59
GB 35,126 0,715 53,78 90,32 90,11
DT 34,183 0,795 40,89 92,64 92,48
Credit to all 31,199 1 0 100 100
Cost = 50, Return = 1
LR-ALL 37,670 0,759 44,00 93,65 93,48
LR-W 37,587 0,822 28,89 97,10 96,98
LR-F 37,183 0,459 30,22 95,92 95,79
NN-ALL 37,364 0,831 24,89 97,58 97,48
NN-W 37,529 0,868 21,78 98,68 98,58
NN-F 37,183 0,451 30,22 95,92 95,79
SVM 37,241 0,002 31,56 95,72 95,59
GB 37,658 0,814 35,44 95,44 95,29
DT 36,843 0,795 40,89 92,64 92,48
Credit to all 35,699 1 0 100 100
Cost = 30, Return = 1
LR-ALL 40,800 0,867 23,11 97,96 97,86
LR-W 40,917 0,887 18,67 98,85 98,76
LR-F 40,691 0,815 15,56 98,81 98,74
NN-ALL 40,833 0,866 19,11 98,60 98,52
NN-W 41,049 0,868 21,78 98,68 98,58
NN-F 40,683 0,914 12,89 99,18 99,12
SVM 40,680 0,002 20,00 98,15 98,06
GB 40,709 0,839 31,56 96,55 96,42
DT 40,199 0,881 0,00 100,00 100,00
(Continued)
 APPLIED ECONOMICS 19

Appendix 4. (Continued).
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
Credit to all 40,199 1 0 100 100
Cost = 10, Return = 1
LR-ALL 44,720 0,975 3,56 99,87 99,86
LR-W 44,763 0,939 10,22 99,59 99,54
LR-F 44,713 0,966 4,89 99,80 99,77
NN-ALL 44,701 0,957 1,33 99,94 99,93
NN-W 44,731 0,961 6,22 99,77 99,74
NN-F 44,699 0,953 0,00 100,00 100,00
SVM 44,708 0,002 0,44 100,00 100,00
GB 44,713 0,944 2,22 99,92 99,91
DT 44,699 0,881 0,00 100,00 100,00
Credit to all 44,699 1 0 100 100
Cost = 1, Return = 1
LR-ALL 46,724 0,996 0,00 100,00 100,00
LR-W 46,724 0,996 0,00 100,00 100,00
LR-F 46,724 0,967 0,00 100,00 100,00
NN-ALL 46,724 0,980 0,00 100,00 100,00
NN-W 46,724 0,988 0,00 100,00 100,00
NN-F 46,724 0,952 0,00 100,00 100,00
SVM 46,724 0,002 0,00 100,00 100,00
GB 46,724 0,962 0,00 100,00 100,00
DT 46,724 0,881 0,00 100,00 100,00
Credit to all 46,724 1 0 100 100

Appendix 5. Model efficiency comparisons; years 6–10 before bankruptcy; all data combined.
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
Cost = 100, Return = 1
LR-ALL 10,695 0,356 88,32 46,42 45,73
LR-W 10,180 0,367 88,11 45,75 45,08
LR-F 8287 0,392 89,16 39,59 39,02
NN-ALL 10,663 0,356 86,32 50,40 49,67
NN-W 10,537 0,326 86,63 49,49 48,78
NN-F 8728 0,342 88,84 41,17 40,57
SVM 7993 0,000 89,05 39,18 38,62
GB 5934 0,478 90,21 32,45 32,00
DT 5817 0,259 78,95 54,99 54,32
Credit to all −48,051 1 0 100 100
Cost = 70, Return = 1
LR-ALL 14,538 0,383 85,47 51,54 50,81
LR-W 14,198 0,393 85,47 50,82 50,10
LR-F 11,530 0,441 78,32 55,27 54,61
NN-ALL 14,879 0,461 81,16 58,38 57,60
NN-W 14,568 0,342 85,37 51,75 51,02
NN-F 12,607 0,417 82,63 61,45 50,78
SVM 13,385 0,000 77,58 60,27 59,52
GB 10,722 0,489 80,84 49,97 49,36
DT 11,854 0,339 77,89 56,56 55,88
Credit to all −19,551 1 0 100 100
Cost = 50, Return = 1
LR-ALL 19,219 0,479 73,47 67,77 66,96
LR-W 18,444 0,472 73,58 66,02 65,23
LR-F 16,066 0,468 71,16 63,40 62,72
NN-ALL 19,002 0,514 70,95 69,87 69,06
NN-W 18,534 0,428 77,58 62,15 61,37
NN-F 16,714 0,497 74,63 61,27 60,55
SVM 18,182 0,000 73,05 65,99 65,22
GB 14,362 0,489 80,84 49,97 49,36
DT 17,308 0,419 67,89 69,35 68,61
Credit to all −551 1 0 100 100
Cost = 30, Return = 1
LR-ALL 25,485 0,607 50,11 84,57 83,88
LR-W 24,936 0,542 60,53 77,08 76,33
LR-F 22,235 0,540 41,37 82,95 82,47
NN-ALL 25,918 0,630 50,42 85,30 84,59
NN-W 25,610 0,593 57,05 80,62 79,87
NN-F 22,702 0,652 33,58 88,69 86,24
(Continued)
20 E. I. ALTMAN ET AL.

Appendix 5. (Continued).
Cost/Return Profit Optimal cut-off Bankr correct-% Non-br correct-% Credit given/All-%
SVM 25,465 0,000 52,95 82,80 82,09
GB 21,408 0,497 60,63 69,50 68,90
DT 24,761 0,468 58,21 78,11 77,39
Credit to all 18,449 1 0 100 100
Cost = 10, Return = 1
LR-ALL 37,783 0,867 9,68 98,75 98,58
LR-W 37,690 0,856 9,47 98,60 98,44
LR-F 37,488 0,850 5,79 98,91 98,82
NN-ALL 37,870 0,860 14,42 97,98 97,73
NN-W 37,763 0,852 13,16 98,01 97,78
NN-F 37,595 0,847 9,63 98,56 98,42
SVM 37,735 0,000 10,74 98,44 98,25
GB 37,449 0,531 0,00 100,00 100,00
DT 37,655 0,705 14,53 97,50 97,26
Credit to all 37,449 1 0 100 100
Cost = 1, Return = 1
LR-ALL 46,000 0,984 0,11 100,00 100,00
LR-W 46,000 0,981 0,11 100,00 100,00
LR-F 45,999 0,964 0,00 100,00 100,00
NN-ALL 46,000 0,904 0,11 100,00 100,00
NN-W 46,000 0,890 0,11 100,00 100,00
NN-F 45,999 0,864 0,00 100,00 100,00
SVM 46,000 0,000 0,11 100,00 100,00
GB 45,999 0,530 0,00 100,00 100,00
DT 45,999 0,883 0,00 100,00 100,00
Credit to all 45,999 1 0 100 100