Decision Support Systems 141 (2021) 113446

Contents lists available at ScienceDirect

Decision Support Systems

journal homepage: [Link]/locate/dss

Predicting donation behavior: Acquisition modeling in the nonprofit sector

using Facebook data
Lisa Schetgen , Matthias Bogaert *, Dirk Van den Poel
Ghent University, Department of Marketing, Tweekerkenstraat 2, 9000 Ghent, Belgium

A R T I C L E I N F O A B S T R A C T

Keywords: The purpose of this study is to demonstrate the value of Facebook data in predicting first-time donation behavior.
Facebook More specifically, we provide evidence that Facebook data can be used as a valuable data source for nonprofit
CRM organizations in acquiring new donors. To do so, we evaluate three different dimensionality reduction techniques
Customer acquisition
(i.e., singular value decomposition, non-negative matrix factorization, and latent Dirichlet allocation) over seven
Predictive analytics
Social media
classification techniques (i.e., logistic regression, k-nearest neighbors, bagged trees, random forest, adaboost,
extreme gradient boosting, and artificial neural networks) using five times twofold cross-validation. Next, we
assess what type of Facebook data and which predictors are most important. The results indicate that we can
predict first-time donation behavior based on Facebook data with high predictive performance. Our benchmark
indicates that the combination of singular value decomposition and logistic regression outperforms all other
analytical methodologies with an area under the receiver operating characteristic of 0.72 and a top decile lift of
3.33. The results show that Facebook pages and categories of Facebook pages are the most important data types.
The most important predictors are dimensions related to age, education, residence, materialism, responsible
consumption, and interest in nonprofits. The presented acquisition models can be used by nonprofit organiza­
tions to implement a one-to-one targeted marketing campaign towards Facebook fans. To the best of our
knowledge, our study is the first to determine the predictive value of Facebook data for nonprofits in a real-life
acquisition context.

1. Introduction an impact on CRM, yet they remain unsure on how to incorporate them
into their CRM strategies and activities [45].
Nonprofit organizations (NPOs) are characterized by a strong devo­ Traditionally, it is very hard for nonprofit organizations to find new
tion to their end users, namely those in need. To successfully fulfill their potential donors. Customer acquisition is inherently difficult, as orga­
social mission, acquiring potential donors and retaining existing donors nizations need information about donors whom they have no existing
is crucial. Hence, customer relationship management (CRM), which relationship with [57]. Therefore, in order to identify potential donors,
focuses on the relationship with donors, can be extremely valuable to NPOs often have no other choice but to acquire expensive external da­
them. For example, Verhaert and Van den Poel [59] demonstrate how tabases. However, the emergence of social media could empower or­
nonprofits can use transactional data in order to predict which in­ ganizations to identify potential customers. By liking a NPO’s Facebook
dividuals are likely to donate again. It goes without saying that suc­ page, users publicly display a certain interest in the organization and
cessful CRM initiatives can lead to increased loyalty of donors, can, therefore, be seen as new potential donors. Hence, by extracting the
significant cost reductions, and improved fundraising results [39]. In list of individuals who have liked their social media page, nonprofits can
recent years, the relationship between nonprofit organizations and do­ pinpoint a substantial base of prospects [57]. Furthermore, social media
nors has been dramatically transformed by the emergence of social contains a huge amount of behavioral data on these potential donors. In
media. Due to limited knowledge and resources, most nonprofit orga­ sum, instead of using external data sources, an organization’s own social
nizations use these platforms for communication purposes [26]. They media data could be used to construct an acquisition model, which
are rarely used as a source of valuable information or as a tool for identifies users that are most likely to donate. Subsequently, these users
decision-making. Nonprofit organizations realize that social media have can directly be targeted with tailored campaigns. Despite the

* Corresponding author.
E-mail addresses: [Link]@[Link] (L. Schetgen), [Link]@[Link] (M. Bogaert), [Link]@[Link] (D. Van den Poel).

[Link]
Received 20 February 2020; Received in revised form 27 September 2020; Accepted 2 November 2020
Available online 5 November 2020
0167-9236/© 2020 Elsevier B.V. All rights reserved.
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

opportunities of social media for donor acquisition, no study evaluates extensive, we focus our literature review on CRM studies in the
how social media data can be used for acquisition modeling in the nonprofit sector. In other words, we review literature that explores
nonprofit sector. Nevertheless, this could be worthwhile for a multitude donation behavior from a CRM point-of-view. Taking into account the
of reasons. First, even though academics and practitioners generally relevance of social media to our study, we also include studies about
focus on customer retention, acquisition of new donors is vital to any donation behavior using social media.
nonprofit organization. Considering the limited customer lifetime of Table 1 categorizes the existing literature according to three di­
donors [51], there will always be a certain number of donors leaving the mensions: (1) research type, (2) CRM domain, and (3) type of data. First,
organization and the necessity to acquire new ones. Second, the (added) two research types can be distinguished. Diagnostic research typically
value of social media data in prospecting, and CRM in general, has investigates underlying reasons and potential determinants of the
already been proven in several applications [8,48]. Because of the behavior of interest, in our case donation behavior. Specifically, these
shrinking donor pool and the fierce competition amongst organizations, studies examine how different factors (e.g., personal values, satisfaction,
it is important for nonprofits to make their acquisition models as accu­ and motivating factors) impact donor return [31,53], recency and fre­
rate as possible [35,52]. Finally, social media data can be a valuable quency of donation behavior [27], and the choice of charity [7]. The
source for mining information with regards to personal characteristics derived results are inherently historical as behavior is described and
and behavior [37,57]. The latter are relevant to nonprofit organizations explained based on information from the past. On the other hand, the
as they can be employed to improve targeted marketing and fundraising goal of predictive research is to determine how current information can
efforts. Given the lack of studies focusing on predicting donation be used to predict future behavior. For example, Althoff and Leskovec
behavior with social media data, several questions remain unanswered: [2] use logistic regression models to predict whether donors on online
(1) “Is it possible to predict donation behavior using solely social media crowdfunding platforms will donate again. Second, literature can be
data, and if so, what data analytical methodologies are required?”, (2) divided according to the application in the CRM domain. Studies that
“What type of social media data matters the most?”, and (3) “Which focus on customer retention investigate potential reasons for churn or
variables are most important?” repeated donation behavior, and explain how their results can be
To fill this gap in literature, this paper investigates whether social incorporated into retention efforts (e.g., [33]). Studies focusing on
media data (i.e., Facebook data) can be used to predict donation customer acquisition highlight how their analyses and results can be
behavior. More specifically, we construct an acquisition model aimed at used to identify potential donors and work out specific marketing stra­
predicting which potential donors (i.e., Facebook fans of the organiza­ tegies (e.g., [59]). Finally, research on donation behavior can be cate­
tion) are most likely to become actual donors. To do so, we worked gorized according to the type of data. Traditional data consists of past
together with a well-known European nonprofit organization and donation behavior (e.g., [35]), socio-demographic data (e.g., [32]),
gathered publicly available Facebook data of their potential donors (i.e., personality traits (e.g. [27]), and intentions (e.g., [15]). Social media
liked Facebook pages, categories of liked Facebook pages, joined Face­ data includes self-reported information about social media usage, as
book groups, frequency variables, and gender). To effectively predict well as information directly retrieved from social media websites. Most
donation behavior, we come up with a decision support system to build studies that use social media data use the former. For example, Brown
acquisition models based on social media data. The goal of this decision and Taylor [12] employ self-reported information to determine whether
support system is to provide a data collection and analytical framework there is a relationship between presence on social media and donation
such that NPOs and/or researchers can easily replicate and implement behavior. Enjolras et al. [24], as well as Farrow and Yuan [26], take it
our approach. Our data collection framework describes the steps to one step further by investigating the underlying reasons of the positive
gather the prospect list from the NPO’s Facebook page and collect the impact of using social media on donating time and money. Chell and
behavioral data from these prospects on Facebook. Our analytical Mortimer [15] take a CRM point-of-view and explore the value of
framework consists of a specific two-stage process. First, we evaluate granting online recognition on social media to motivate individuals to
different dimensionality reduction techniques (i.e., singular value donate again. With regards to the type of data, Courtois and Verdegem
decomposition (SVD), non-negative matrix factorization (NMF), and [18] are the only ones who use directly gathered social media data in
latent Dirichlet allocation (LDA)) to cope with the great number of addition to self-reported data. In their study, they retrieve information
Facebook pages and groups. To increase the robustness of our results, we related to Facebook pages and groups (e.g., the specific goal and the
benchmark these different dimensionality reduction techniques over number of page likes) in order to investigate the determinants of con­
seven prediction algorithms (i.e., logistic regression (LR), k-nearest nective action on Facebook.
neighbors (KNN), bagged trees (BT), random forest (RF), adaboost (AB), From Table 1, it is clear that no study has built a predictive model on
extreme gradient boosting (XGB), and artificial neural networks (NN)). the basis of solely social media data for acquisition purposes in the
We contribute to existing literature by determining the optimal combi­ nonprofit sector. Our contribution to existing literature is twofold. First,
nation of data reduction and modeling techniques when using social none of the predictive studies in Table 1 are performed in a real-life
media data. This optimal combination then serves as the input of the acquisition context. In the study by Lee and Chang [41], the depen­
second stage of our process, in which we determine the most valuable dent variable (i.e., whether someone is likely to donate) is derived from
Facebook features. To determine the (added) value of different variable self-reported information. Furthermore, no distinction is made between
types, we construct several models on different subsets of variables. For first-time donations and redonations. In contrast to Lee and Chang [41],
instance, we compare the performance of the most complete model to a Verhaert and Van den Poel [59] model actual donation behavior as their
model that excludes variables related to Facebook page categories. dependent variable. However, they focus on existing donors only, which
Finally, we also assess variable importances to uncover the driving implies that they model redonation instead of first-time donation. They
forces of predictive performance. extrapolate their results to an acquisition context by suggesting that
The remainder of this paper is organized as follows. First, we provide empathy could be a valuable predictor for new donors. Hence, we are
an overview of existing literature. Second, we explain our methodology the first to investigate the prediction of donation behavior in a real-life
in more detail. Next, we present an overview of our results, followed by a acquisition context. Second, the studies taking into account social media
conclusion and discussion of their practical implications. Finally, we data, even though they have evolved from self-reported to crawled data,
formulate recommendations for future research. inherently perform diagnostic research. For example, Brown and Taylor
[12] and Enjolras et al. [24] perform a regression analysis to determine
2. Literature overview the effects of specific individual attributes on charitable behavior.
Farrow and Yuan [26] use structural equation modeling to test causal
Since an elaborate overview of social CRM literature would be too hypotheses and study the relationship between using a NPO’s Facebook

2
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 1
Overview of CRM literature concerning the nonprofit sector, as well as studies regarding donation behavior using social media.
Study Research type CRM domain Data type

Diagnostic Predictive Retention Acquisition Traditional Social media

Sargeant [52] x x x
Bennett [7] x x x x
Hsu et al. [35] x x x x
Germain et al. [32] x x x
Ferguson [27] x x x x
Schlumpf et al. [53] x x x
Garner & Garner [31] x x x
Enjolras et al. [24] x x x
Chell & Mortimer [15] x x x x
Brown & Taylor [12] x x x
Courtois & Verdegem [17] x x x
Warren et al. [60] x x
Farrow & Yuan [26] x x x
Godin et al. [33] x x x
Althoff & Leskovec [2] x x x
Lee & Chang [41] x x x x
Verhaert & Van den Poel [59] x x x x
Our study x x x

group and actual behavior. Furthermore, only one of these studies is Facebook allows individuals to find and interact with people supporting
situated in the CRM domain. Chell and Mortimer [15] use Pearson the same good cause and can consequently engender feelings of positive
correlation and regression analysis techniques to test whether online group acceptance and identification. Farrow and Yuan [26] later
recognition has an impact on someone’s intention to donate again. Their confirmed these findings for Facebook groups. Hence, considering
results can assist NPOs to develop online retention strategies that Facebook makes it possible to identify a novel pool of interested in­
leverage self-interest over altruism but they do not assess whether their dividuals (i.e., Facebook fans) and has the potential of solidifying the
findings can be used to predict future donations. Hence, there are no relationship with these individuals, it would be a missed opportunity not
predictive studies on acquisition modeling using social media data in the to investigate this new source of potential donors. Second, we believe
nonprofit sector. This is a missed opportunity since Facebook has over 2 that Facebook data is valuable for the prediction of first time donation
billion active monthly users [25] and social media data is available for behavior as previous research on social media data for prospecting
(almost) everyone. Hence, this would allow organizations to access an confirms the potential value of Facebook data [8,48]. On the basis of
enormous number of potential donors. Moreover, when exclusively Facebook data, Bogaert et al. [8] are able to accurately predict which
using social media data, nonprofit organizations do not need to have a individuals practice a certain hobby (i.e., soccer). Sports brands can then
pre-existing relationship with potential donors to collect information use this information to identify new potential customers and decide
and work out acquisition initiatives. In other words, social media allows which ones to target. Whereas Bogaert et al. [8] focus on a B2C setting,
decision makers to perform real-life acquisition experiments. Meire et al. [48] investigate the added value of Facebook data for
To fill this gap in literature, we create a decision support system for customer acquisition in a B2B setting. They find that models incorpo­
constructing an acquisition model on the basis of Facebook data. The rating Facebook data are considerably better at predicting good pros­
aim of this model is to predict whether a fan of a nonprofit organiza­ pects. Hence, as Facebook data has proven to be a valuable predictor in
tion’s Facebook page is likely to become an actual donor. Nowadays, several acquisition situations, we believe it could be equally valuable for
nonprofit organizations are forced to use their own databases in order to acquisition modeling in the nonprofit sector. Third, previous work on
build predictive models. Consequently, they can only focus on their own predicting user behavior with social media data has shown that indi­
customer base. Leveraging the power of their Facebook pages (and social vidual Facebook features can be used to accurately predict personal
media data in general) allows them to identify new potential donors (i. characteristics such as age, gender, relationship status, and personality
e., users who like their Facebook page), target those Facebook fans who traits [3,36,37]. Considering that these characteristics are also signifi­
are most likely to become a donor, and gain insights into their charac­ cantly related to the act of giving [6,12,14,51], we believe that Facebook
teristics. To deliver a decision support system that allows NPOs to pre­ data can similarly be used for the prediction of donation behavior.
dict acquisition of their Facebook fans, we present a framework for the Specifically, there are two types of Facebook features that we suspect to
collection of relevant Facebook data (Facebook pages, categories of be particularly valuable for this task. Kosinski et al. [37] demonstrate
Facebook pages, Facebook groups, and socio-demographics) and employ the value of Facebook pages in predicting a wide range of private at­
a specific data analytical process. In the first step of this process, we tributes that are related to charitable giving (e.g., age, personality traits,
benchmark three dimensionality reduction techniques over seven clas­ political viewpoints). The value of Facebook likes results from the fact
sification algorithms to find the optimal combination of both methods. that an ever-increasing proportion of human activities takes place on­
Second, we use this optimal combination to evaluate different subsets of line. Hence, Facebook likes represent digital records of human behavior
variables. This allows us to establish the (added) value of the different and are used by users to express their positive association with online
types of Facebook features. Furthermore, we compute variable impor­ content [37]. Besides Facebook pages, we believe that the categories of
tances to discover which features are most important in predicting these liked pages could be valuable for the prediction of donation
donation behavior. behavior as well. Zhang and Pennacchiotti [61] find that categories of
There are several reasons why we believe Facebook can be valuable liked Facebook pages hold enough information to predict users’ pur­
for the development of donor acquisition models. First, Facebook can chase behavior. This could be explained by the fact that Facebook users
represent an important acquisition platform as it enables new opportu­ like specific categories and as such express personal interests [61]. Based
nities of gaining and sharing information [24], which can strengthen the on the findings in existing literature, there are strong indications that
relationship with prospective donors [26] and ultimately the decision to Facebook pages and categories of Facebook pages will be amongst the
donate [35]. Furthermore, Courtois and Verdegem [17] state that top variables when predicting donation behavior.

3
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

To summarize, our study does not only allow us to determine Facebook users in our sample allow public access to information related
whether it is possible to predict donation behavior with Facebook data, to pages and groups, this reduces our sample to 8246 observations.
but also enables us to gain insights into the most important data types Furthermore, we exclusively consider Facebook pages and groups that
and predictors. Moreover, on the basis of previous literature, we find have been liked or joined prior to liking the NPO’s Facebook page. This
strong indications that donation behavior (i.e., becoming a new donor) is crucial for the correctness of our predictive model. When developing a
can be accurately predicted with individual Facebook data. To the best predictive model, the predictors should be computed based on data from
of our knowledge this is the first study to investigate the feasibility of the independent period [40]. Whereas the independent period has no
acquisition modeling on the basis of Facebook data in the nonprofit predefined start date, it ends at the time of liking the NPO’s Facebook
sector. page, as shown in Fig. 1. As such, all the Facebook pages (groups) that an
individual has liked (joined) up until the moment of liking the NPO’s
3. Methodology Facebook page are included in the computation of the predictors. The
dependent variable (i.e., donor or not) should be computed based on
3.1. Data data from the dependent period, meaning the period following after
liking the Facebook page (see Fig. 1). Hence, the dependent period starts
To gather our Facebook data, a data collection framework similar to from the moment a Facebook fan likes the NPO’s Facebook page and
van Dam and van de Velden [57] is used. The first step consists of ends on the 8th of April 2018 (i.e., the date we received a copy of the
identifying users who liked the NPO’s Facebook page. In what follows, internal donor database of the NPO). As the end of the independent
we will refer to these users as fans. A list of fans can be viewed by going period and the start of the dependent period are different for every
to [Link] Facebook fan in our sample, the length of the dependent period ranges
er_pages, where 12345 should be replaced by the NPO’s Facebook from 39 to 715 days.
page ID. It is important to note that this URL is exclusively accessible to Besides Facebook data we also possess transactional data from the
administrators of the concerned Facebook page. In February 2018, a list NPO, which will be used to identify actual donors amongst the Facebook
of 8646 unique fans was retrieved by scraping the aforementioned fans. The data is merged via names, as this is the only piece of infor­
Facebook page using the RSelenium R-package [34]. Fans’ Facebook mation common to both datasets [49]. To sum up, for each individual in
names, along with their Facebook ID and the date on which they started our sample we have the following data (if available): (i) name, (ii) date
to like the NPO’s Facebook page, were extracted from the webpage’s of becoming a donor (if applicable), (iii) date of liking the NPO’s
source code using regular expressions. The second step of the data Facebook page, (iv), list of the names of liked Facebook pages, (v) list of
collection framework proposed by van Dam and van de Velden [57] the categories of liked Facebook pages, (vi) list of the names of joined
involves the retrieval of relevant Facebook data for each of the identified Facebook groups, and (vii) socio-demographics (gender and birthdate).
fans. In the context of this research, the relevant data consists of liked To guarantee correct matching, names occurring more than once are
Facebook pages, joined Facebook groups, and socio-demographics (i.e., deleted from our sample. The resulting sample consists of 8167 Face­
gender and age). As this data cannot be collected using Facebook’s book fans who have liked 667,740 unique Facebook pages, corre­
Graph API, an alternative manner was pursued. Individuals’ pages and sponding to 1738 unique page categories, and joined 213,633 unique
groups were collected by visiting and scraping the following webpages: Facebook groups. Of those 8167 Facebook fans, we matched 477 in­
[Link] dividuals to the NPO’s internal database.
=Pagina+die+ik+leuk+vind&starttime=0&endtime=6789 and htt
ps://[Link]/search/12345/groups&starttime=0&endt
ime=6789. In these URL’s, 12345 should be replaced by a fan’s Face­ 3.2. Variables
book ID and 6789 should represent the date on which this particular fan
started to like the NPO’s Facebook page. The first URL leads to a web­ 3.2.1. Dependent variable
page displaying the Facebook pages that an individual has liked up until Our dependent variable is a binary variable that indicates whether a
the moment of liking the NPO’s Facebook page. Similarly, the second Facebook fan has become a donor (1) or not (0). To make sure that we do
URL gives access to a webpage that consists of a list of Facebook groups not violate the time window of our predictive model, the dependent
joined by a particular individual. In essence, for each of the 8646 unique variable is computed based on information from the period after liking
Facebook ID’s, the aforementioned webpages were accessed and scraped the NPO’s Facebook page (see Fig. 1). Furthermore, an acquisition
between February and May 2018. By applying regular expressions to the model is aimed at prospective donors. This means that the individuals in
source code of the respective webpages, we were able to extract a list of our sample cannot be donors at the end of the independent period.
liked Facebook pages, containing the names of the pages as well as the Hence, our data sample should consist of the Facebook fans that are
corresponding categories, and a list of joined Facebook groups, con­ targetable. These are either Facebook fans that are not donors at all, or
sisting of the groups’ names. For the collection of gender and age, a Facebook fans that became a donor after liking the NPO’s Facebook
similar approach was employed. Between May and June 2020, we page.
scraped the following page for every fan in our sample and extracted In practice, the selection of relevant observations and the computa­
information on gender and birthdate using regular expressions: https tion of the dependent variable are based on the date of becoming a donor
://[Link]/12345/about?section=contact-info, where and the date of liking the NPO’s Facebook page. If the first variable is
12345 should be replaced by the fan’s Facebook ID. It is important to missing, this means that the Facebook user could not be matched to the
mention that only publicly available information is collected.1 In other internal data and thus is not a donor. Hence, the dependent variable is
words, the collected data exclusively concerns users who allow public set to 0. In case the date of first donation takes place between the date of
access to their Facebook information. liking the Facebook page and the 8th of April 2018, the user is identified
Since it is imperative that most information is available for all ob­ as a donor and the dependent variable is set to 1. Finally, users that were
servations in our sample, we delete Facebook fans for whom no data is
available about Facebook pages and/or groups. As over 95% of

1
Facebook’s privacy regulations state that public information can be viewed,
opened, shared, and downloaded by third parties. See [Link]
com/full_data_use_policy for more information. Fig. 1. Time window.

4
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

a donor before liking the NPO’s Facebook page, are excluded from our 3.2.3. Models
sample. One of the goals of this study is to assess the value of the different
Our final data sample consists of 7795 observations. From these, only types of Facebook features. To do so, we create different models sum­
96 became a donor in the dependent period (i.e., after liking the NPO’s marized in Table 2. Models 1 (M1), 2 (M2), and 3 (M3) are, respectively,
Facebook page). In other words, merely 1.23% (96 out of 7795) of the built on Facebook pages, Facebook page categories, and Facebook
individuals in our sample are donors, whereas 98.77% (7699 out of groups only. This is done to investigate whether any of these types of
7795) are not. This class imbalance is inherent to the issue at hand. Facebook features on their own are sufficient to create an accurate
Namely, on social media, the number of fans and followers of nonprofit prediction model. Model 5 (M5) consists of the frequency variables, the
organizations is ever increasing but only very few of them actually variables related to Facebook pages, as well as those related to Facebook
donate. Hence, identifying these few potential donors is of great value page categories. In comparison with Model 8 (i.e., the most complete
and is exactly what our predictive model is aimed at. To cope with the model), the variables related to Facebook groups have been left out in
class imbalance problem, we follow the recommendation of Bogaert M5. By comparing the performance measures of Model 5 and Model 8,
et al. [10] to use random oversampling as their study finds that it is we are able to judge the added value of considering Facebook features
superior to other data sampling methods on social media data (i.e., related to groups. The same rationale can be applied to Model 4 (M4),
undersampling and SMOTE). Following their recommendations, we Model 6 (M6), and Model 7 (M7). Note that the variable gender is added
performed oversampling on the training set until a 50/50 distribution as a control variable to all models.
was reached. From Table 2, it is clear that we do not consider all possible com­
binations of Facebook features. This is because we are specifically
3.2.2. Independent variables interested in the (added) predictive value of the different types of
For the construction of the independent variables, we start by Facebook data.
creating matrices corresponding to the different Facebook features: (1)
user-page matrix, (2) user-category matrix, and (3) user-group matrix. 3.3. Dimensionality reduction
The rows of the matrices correspond to the Facebook fans, whereas the
columns correspond to the liked Facebook pages, the categories, and the In this section we discuss the different dimensionality reduction (DR)
joined Facebook groups. For example, element xij of the user-page ma­ techniques that we apply to our input data (i.e., user-page, user-cate­
trix is set to 1 if user i likes Facebook page j. The same rationale can be gory, and user-group). The following three DR techniques are used:
applied to the other matrices. 68.59% of the Facebook pages (78.33% of singular value decomposition (SVD), non-negative matrix factorization
the Facebook groups) are liked (joined) by only one individual in our (NMF), and latent Dirichlet allocation (LDA). SVD is the most popular
sample. Based on the recommendation of Kosinki et al. [37,38], we dimensionality reduction technique in existing literature [37]. Its
decide to keep only those pages, categories, and groups that have been popularity can be attributed to its simplicity and computational speed
liked by at least 20 users in our sample. This reduces the number of [38]. Furthermore, Clark and Provost [16] compared the impact of
unique Facebook pages, page categories, and Facebook groups to several DR techniques on the performance of predictive models and
respectively 11,408, 1098, and 1402. found SVD to be the best technique. Just as SVD, NMF is a matrix
Nevertheless, the dimensionalities and sparsity levels of our matrices factorization-based DR technique. However, in contrast to SVD, NMF
remain high. Therefore, we apply several dimensionality reduction (DR) requires the resulting components to be non-negative. Consequently, the
techniques to these matrices. The main reason for using DR in predictive number of components is often sparse and easy to interpret [16,50].
modeling is to increase predictive performance while reducing sampling Finally, LDA is traditionally used as a clustering and topic modeling
variance [16]. Furthermore, DR could uncover latent information pre­ technique but can be equally useful for dimensionality reduction pur­
sent in our Facebook data. We decide to perform DR on each of the poses. It is easy to interpret but can be computationally expensive and
matrices separately, as this allows us to distinguish between the (pre­ requires the original data to be non-negative [38]. A description of the
dictive value of the) different types of Facebook features in the second different DR techniques is given in Table 3.
stage of our research. The different dimensionality reduction techniques Choosing the optimal number of dimensions (k) is crucial when
are presented in Section 3.3. applying dimensionality reduction techniques. If k is too small, the
Furthermore, we also include frequency variables into our predictive concepts related to the resulting dimensions will be too broad. If k is too
model: the number of liked Facebook pages and the number of joined large, this may lead to similar concepts [50]. Several authors highlight
Facebook groups. For each Facebook fan, these are calculated as the sum the fact that there is no single correct way of identifying the optimal
of the corresponding row in the user-page matrix and the user-group number of dimensions [16,38,50]. Kosinski et al. [38] note that this
matrix respectively. Finally, we consider key socio-demographical var­ choice should depend on the application. For example, when using DR to
iables, namely gender and age. The variable gender was successfully build predictive models, it could be useful to choose a larger number of
collected for 6571 out of the 7795 Facebook users in our final data dimensions, as these will preserve more information from the original
sample. The remaining 1224 values are imputed based on the fans’ first data. However, when k is set too high, the benefits of DR could be lost,
names using the [Link] API.2 With regards to the variable age, leading to a decrease in accuracy [38].
information on birthdate was only available for 7.99% of the Facebook Similarly to Praet et al. [50], we decide on the optimal value of k by
fans in our sample. Another option could be to deduce age by linking iterating over several values and inspecting the resulting predictive
users’ Facebook ZIP codes with census data. Unfortunately, similarly to performances. Specifically, we constructed a model for every combina­
age, this information is available for very few Facebook users. Given the tion of algorithm, DR technique and level of k (50, 100, and 200) (see
large number of missing values, we discarded the variable age. In sum, Section 3.5). The optimal number of dimensions is chosen according to
the independent variables for our predictive model consist of k di­ the value of k that results in the best performing model, measured by the
mensions related to Facebook pages, page categories, and Facebook AUC.
groups, which result from the application of a DR technique on the To assess whether using DR techniques such as SVD, NMF, and LDA,
respective matrices (see Section 3.3), as well as the number of Facebook improves the predictive performance, it is imperative to compare these
pages, the number of Facebook groups, and the user’s gender. DR techniques with a dataset that has not undergone any feature engi­
neering [16]. Hence, we also create a baseline model for which we do
not apply any DR techniques to our input data. In this model the pre­
diction variables are equal to the columns in the user-page, user-cate­
2
[Link] gory, and user-group matrices. The columns represent binary variables

5
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 2
Description of the models.
Description M1 M2 M3 M4 M5 M6 M7 M8

Pages
100 dimensions related to Facebook pages, resulting from the application of a dimensionality reduction x x x x x
technique on the user-page matrix
Categories
100 dimensions related to categories of Facebook pages, resulting from the application of a dimensionality x x x x x
reduction technique on the user-category matrix
Groups
100 dimensions related to Facebook groups, resulting from the application of a dimensionality reduction x x x x x
technique on the user-group matrix
Frequency
#Liked pages (total number of Facebook pages) and #Groups (total number of Facebook groups) x x x x

(i.e., least absolute shrinkage and selection operator) approach [56].

Table 3 Whereas LR is a parametric algorithm (i.e., it simplifies the learning
Description of dimensionality reduction techniques.
function to a known form and learns the related coefficients from the
DR technique Description training data), KNN is nonparametric. KNN makes no assumptions about
Singular Value SVD represents the user-page matrix (A) as the product the form of the mapping function other than that data points that are
Decomposition of three matrices (A = UΣVT): user-to-concept matrix close are likely to have a similar outcome. It is equally a popular
(U), page-to-concept matrix (V), and a square matrix nonparametric method in social media applications [10]. Finally, NN is
containing the singular values on the diagonal (Σ). The
former two respectively represent how much a given
a semi-parametric method, as it requires to select a specific functional
user and Facebook page correspond to a given concept. form a priori. The parameters of the model are estimated using nonlinear
The singular values in the latter are sorted in decreasing optimization [44]. NN has often been employed in social media ana­
order of explained variance in the original data. By lytics [9] and has achieved superior performance in large-scale bench­
retaining the k first singular values, the k most
mark studies [44] due to its ability to uncover complex nonlinear
important dimensions are identified [38].
Non-negative Matrix NMF factorizes a given matrix V into two matrices W relationships between the predictors and the response variable. For our
Factorization and H, under the constraint that all three matrices are study, we train a NN with only one hidden layer and a logistic activation
positive: V ≈ WH. As the goal is to approximate V, W function using backpropagation since it has been proven that NNs with
and H are computed such that the Frobenius norm of the one hidden layer serve as universal approximators [4].
difference V-WH is minimized. The columns in H, which
contain weights, indicate how to reconstruct an
The ensemble techniques in this study are bagged trees (BT), random
approximation of the original data vectors as a forest (RF), adaboost (AB), and extreme gradient boosting (XGB). These
combination of the basis elements in W [42]. classifiers are often found to be top performers in both CRM [43] and
Latent Dirichlet LDA is based on the idea that Facebook users like a social media studies [5]. Ensemble algorithms aggregate a set of indi­
Allocation certain distribution of latent topics and that these topics
vidual classifiers by combining their individual predictions [23]. Even
are described by distributions of Facebook pages. The
application of LDA results in the computation of two though the considered algorithms are all tree-based ensembles, they
matrices ϕ and θ. ϕ represents the importance of differ in how they generate and aggregate the different decision trees. BT
Facebook pages per topic. θ indicates the importance of generates multiple classifiers by manipulating the training examples (i.
topics for the Facebook users. The probability pij that e., taking bootstrap sample of the training data and aggregating the
user i likes page j can be computed as [54]: pij =
∑T bootstrap samples). It is identified as one of the top performing en­
t=1ϕjtθti
sembles in customer churn [43]. RF adds an extra layer of randomness to
BT by selecting only a random subset of the predictors at each tree split
that indicate whether a Facebook fan has liked a certain page, category [11]. RF was found to be the best performing algorithm in a number of
or group (see Section 3.2). To ensure that we only keep the Facebook social media applications, such as B2B acquisition [48]. Whereas BT and
pages, categories and groups that are meaningful to the classification RF are parallel ensemble methods (i.e., the individual trees are gener­
task (i.e., donor or not), we follow the advice of van Dam and van de ated in parallel), AB and XGB generate the individual learners sequen­
Velden [57] to only maintain the top 100 Facebook pages, categories tially. AB iteratively gives more weight to misclassified observations
and groups in our baseline model. The top pages and page categories [28], whereas XGB minimizes the gradient of the loss function in each
(top groups) are determined based on the number of likes (group iteration [29]. Both AB [8–10] and XGB [19] have proven to be suc­
members) in our sample. cessful in social media applications. Table 4 summarizes the settings of
the hyperparameters of the different prediction algorithms.

3.4. Prediction algorithms

3.5. Experimental set-up
Since we are interested in classifying Facebook fans into donors and
non-donors, we employ several prediction algorithms. We chose algo­ To evaluate predictive performance we use two distinct measures
rithms that have been proven to yield superior performance in analytical that are widely accepted as appropriate evaluation metrics for the per­
CRM and social media (e.g., [10,48]) as this ensures the comparability of formance of classification algorithms, namely AUC (Area Under ROC
our results. The algorithms can be divided into two broad categories: Curve) and top decile lift (TDL) [47,48].
single classifiers and ensemble techniques. The single classifiers are lo­ The advantage of the AUC, in comparison with other performance
gistic regression (LR), k-nearest neighbors (KNN), and an artificial measures, is that it does not depend on a specific threshold [13]. The
neural network (NN). LR is one of the most popular single classifiers and receiver operating characteristic (ROC) curve graphically represents the
is often used as a benchmark model in analytical CRM [18,21]. It esti­ relation between the true positive (TP) rate and the false positive (FP)
mates the conditional probability p(y | x) by maximizing the likelihood rate [10]. The AUC can be interpreted as the probability that a random
function [39]. One downside of LR is that it is prone to overfitting when positive observation gets a higher score than a random negative obser­
introduced with a lot of variables [8]. Therefore we applied the LASSO vation [13]. An AUC of 0.5 indicates that the predictive model does not

6
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 4 4. Results
Hyperparameter settings of the prediction algorithms.
Prediction Hyperparameter(s) Settings 4.1. Dimensionality reduction and prediction algorithms
algorithm

Logistic Regression Regularization parameter (lambda)a Fold-based: 0 < Table 5 provides an overview of the cross-validated model perfor­
(LR) lambda <0.25 mance measures (i.e., AUC and TDL) for all possible combinations of DR
K-Nearest Number of nearest neigbors (k)a 10, 100, 200, …, techniques and prediction algorithms. The optimal level of k is reported
Neighbors (KNN) 1000 between brackets3 and varies across the different combinations of al­
Bagged Trees (BT) Number of trees 500
Random Forest Number of trees 500
gorithms and DR techniques. The results indicate that acquisition of
(RF) Number of predictor variables in the sqrt(number of NPOs’ Facebook fans can be accurately predicted on the basis of indi­
random subset at each node of the tree variables) vidual Facebook features: the AUC ranges from 0.55 to 0.72, depending
AdaBoost (AB) Number of iterations 500 on the employed techniques. The best performing models per algorithm
Number of terminal nodes in the base 8
are marked in bold in Table 5. We note that these do not differ signifi­
classifiers
Extreme Gradient Maximum depth of a tree (max_depth)a 2, 3, 5, 7, 10, 100 cantly in terms of AUC. The top performer (underlined and in bold) is the
Boosting (XGB) Learning rate (eta)a 0.025, 0.05, 0.1, combination of SVD and LR and results in an AUC of 0.72 and a TDL of
0.2, 0.3 3.33. This means that the probability that a random donor gets a higher
The minimum loss reduction required 0, 0.1, 0.2, 0.3, score than a random non-donor is equal to 0.72. In other words, the
to create a partition on a leaf (gamma)a 0.4, 1.0, 1.5, 2.0
Neural Network Number of hidden nodes (size)a 2, 3, …, 20
added value of our top-performing model over a random model is 0.22.
(NN) Regularization parameter (decay)a 10^(–4, –3.5, …, Focusing on the 10% highest predictions, our model identifies 3.33
0) times more donors than a random model. The fact that the combination
a
Hyperparameter was tuned.
of LR and SVD is found to be the top performer could be due to the
specificities of SVD. In the context of this study, SVD decomposes the
user-page matrix into a user-to-concept matrix (displaying how much a
perform any better than a random classifier. On the other hand, a value
given user corresponds to a given concept), a matrix containing the
equal to 1 stipulates that the predictive model is perfect [9].
singular values on the diagonal (representing the strength of each
The top decile lift (TDL) measures how much better a predictive
concept), and a page-to-concept matrix (displaying to which extend a
model is at identifying positives, in comparison to simply selecting in­
given Facebook page corresponds to a given concept). The singular
stances at random. It is defined as the ratio of the percentage of positives
values are positive and sorted in decreasing order of the amount of
(i.e., donors) in the top 10% of potential donors that have been attrib­
variance they account for in the original data. SVD operates as a DR
uted the highest scores and the overall percentage of positives [48].
technique by selecting the k first singular values (i.e., the dimensions
The AUC and TDL measure two different aspects of model perfor­
that capture most of the variation in the original data) [38]. By
mance. Whereas the AUC measures the performance over the entire
excluding the low-variance components, SVD overcomes the multi­
range of predictions, TDL focuses on the predictions with the highest
collinearity problem [46]. In other words, SVD creates new dimensions
likelihood to become a donor. Inspecting both of them, allows for a more
that are linear combinations of the original features and that are
nuanced evaluation of the predictive performance of the models [47].
orthogonal and uncorrelated by definition [46]. The absence of multi­
To guarantee the robustness of our results, we perform five times
collinearity, which is one of the conditions of LR, is as such fulfilled.
twofold cross-validation (5x2fcv) [22]. The fist step is to randomly split
Moreover, applying SVD will also make sure that there is an approxi­
the data into two distinct non overlapping samples of equal size (i.e., 50/
mate linear relationship between the predictors [46]. Hence, SVD will
50 distribution). Subsequently, the first sample is used to compute the
tackle one of the major downsides of regression techniques, namely their
new dimensions (i.e., resulting from the application of a specific DR
sensitivity to multicollinearity and nonlinearities in the data. Perhaps
technique for a pre-specified level of k) and to train the model (i.e.,
this is even more the case in the context of this research as liking a
training set), whereas the second sample is used to evaluate its perfor­
specific Facebook page is most likely strongly correlated with liking
mance (i.e., test set), and vice versa. In case the algorithm requires
another page and the relationships are inherently nonlinear [5]. By
hyperparameter tuning, the training set is again equally split into a
applying SVD and taking away these correlations and nonlinearities, LR
training and a validation set, and a grid search is performed to select the
is able to estimate the true function.
optimal parameter(s). Subsequently, the model is re-trained on the full
Furthermore, based on the results in Table 5, we can state that, in
training set with the optimal parameter settings. By repeating this pro­
comparison to the baseline model, the use of a DR technique generally
cess 5 times, we obtain 10 performance values [5]. As mentioned in
leads to an increase in overall predictive performance (AUC). Across
Section 3.3, 5x2fcv is repeated for every combination of algorithm, DR
several prediction algorithms, SVD appears to be the best performing DR
technique and level of k (50, 100, and 200) separately. In other words,
technique, which is in line with the findings of Clark and Provost [16]. In
700 models are compared in total: 7 algorithms and 3 DR techniques
terms of TDL, the application of a DR technique only leads to an
cross-validated 10 times and repeated for 3 levels of k, as well as 7 binary
important increase in performance in case of LR and XGB. Nevertheless,
baseline models equally cross-validated 10 times. Finally, the reported
our results demonstrate the importance of dimensionality reduction as
results are the best combination of algorithm and DR, by taking the
the top performer for both AUC and TDL can only be identified by
median AUC and TDL over the 10 cross-validation runs and selecting the
evaluating all possible combinations of DR and prediction algorithms.
optimal level of k.
To sum up, the top performer would have never been found if we had
To determine whether the performances of the different models are
only considered binary variables (i.e., variables that have not undergone
significantly different from each other, we make use of the combined
any feature engineering). Furthermore, additional analysis4 revealed
Alpayadin F-test [1] with Holm corrections for family-wise error [30].
that applying DR techniques is necessary to make our models more
Existing literature covers a wide range of statistical tests for comparison
stable and robust to overfitting. The binary models woefully overfit the
of prediction models. However, they generally focus on comparing
data, which explains their suboptimal performance. Next, the optimal
classifiers across different data sets [20]. As the aim of this study is to
compare several models based on the same data set, these tests are not
appropriate and we use the combined Alpayadin F-test instead.
3
The optimal level of k is only included for the AUC in Table 5. The same
levels of k were used for the TDL and therefore not reported.
4
The results are available upon request.

7
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 5
Cross-validated median AUC and TDL for all combinations of DR techniques and prediction algorithms.
AUC TDL

Binary SVD NMF LDA Binary SVD NMF LDA

LR 0.6658 0.7239 (k = 100) 0.6960 (k = 100) 0.6989 (k = 100) 2.6265 3.3305 2.9500 2.7250
KNN 0.6326 0.6078 (k = 100) 0.6408 (k = 100) 0.6263 (k = 50) 2.0770 1.5125 1.9980 1.7215
BT 0.5898 0.6155 (k = 100) 0.6114 (k = 50) 0.6532 (k = 100) 1.7130 1.6020 1.5550 1.7700
RF 0.6663 0.6820 (k = 100) 0.6735 (k = 50) 0.6757 (k = 100) 2.4185 1.8185 2.0020 2.1515
AB 0.6387 0.6639 (k = 100) 0.5496 (k = 50) 0.5985 (k = 50) 2.3645 1.8595 1.1920 1.5935
XGB 0.5978 0.6630 (k = 100) 0.5833 (k = 50) 0.6463 (k = 50) 1.7685 1.8300 1.2970 2.0145
NN 0.5854 0.6793 (k = 200) 0.6624 (k = 200) 0.7009 (k = 50) 2.2985 2.1365 1.9790 2.2080

combination is used for the construction of different models and the worse than the most complete model (M8). Moreover, we can state that
computation of variable importances in the second part of this study. Facebook pages alone (M1) are sufficient to create an accurate acqui­
Considering only Facebook data were included, as well as the diffi­ sition model. The same conclusion can be made for the categories of
culty of acquisition modeling, our model performance is very satisfying. Facebook pages (M2). Besides inspecting the overall performance of
Specifically, the performance of our best model (i.e., the combination of these models, we also evaluate how well they perform in terms of TDL.
SVD and LR) is highly competitive compared to existing acquisition The resulting TDLs are presented in Fig. 2(b). We find that, just as in
literature. Meire et al. [48], who modeled acquisition in a B2B setting, terms of AUC, M1 and M2 perform as well as M8 (see Table 6). M3,
obtained AUCs ranging from 0.54 to 0.61. Similarly, D’Haen et al. [21] which performs significantly worse than M8 at the 1% significance level
and Thorleuchter et al. [55] reached maximum AUCs of 0.62 and 0.61 in terms of AUC, only performs worse than M8 at the 10% significance
respectively. Also in terms of TDL, our best model performs well in level in terms of TDL (see Table 6). When comparing M1, M2, and M3,
comparison to previous acquisition studies. For example, Thorleuchter we find that M1 an M2 perform significantly better than M3 in terms of
et al. [55] report TDL values ranging from 1.35 to 1.65. AUC, but not in terms of TDL. Even though the difference with M3 is not
As we consider Facebook data only, our results can be compared to statistically significant, Fig. 2(b) shows that M1 and M2 are also superior
existing literature in social media analytics. Specifically, we examine in terms of TDL. These results highlight the individual predictive power
studies that are concerned with predicting personal characteristics based of Facebook pages and page categories for the prediction of donation
on social media data. In a study by Kosinski et al. [37], the AUCs vary behavior.
between 0.6 and 0.95 depending on the characteristic that is being Second, we evaluate the added value of the different types of Face­
predicted. The best performing models correspond to the prediction of book features. By creating a model that excludes one type of features,
personal attributes such as gender or origin. On the other hand, the and comparing its performance to that of the most complete model (M8),
prediction of increasingly tricky characteristics, such as relationship the added value of those features can be determined. Fig. 3(a) displays
status or use of drugs, consistently generate lower AUCs [37]. Consid­ the AUCs of the resulting models in comparison to the AUC of M8. We
ering the fact that donation behavior is a complex personal attribute, our find that M6 (i.e., the model leaving out the Facebook page categories)
results are in line with previous research and can be perceived as good. has the lowest AUC, followed by M7 (i.e., the model leaving out the
Facebook pages). In other words, categories of Facebook pages have an
added value of 0.09 (= 0.7239–0.6362). Similarly, including features
4.2. Models related to Facebook pages leads to an increase in AUC of 0.04 (=
0.7239–0.6863). The added value of Facebook groups and frequency
To assess the value of the distinct types of Facebook data, we con­ variables appears to be smaller. It is important to note that M6 is the only
structed several models on different subsets of features (see Section 3.2). model that performs significantly worse than M8 in terms of AUC (see
First, we evaluate the individual predictive value of each subset. Fig. 2(a) Table 6), ratifying the added value of Facebook page categories. With
shows the AUCs of the models build on Facebook pages, Facebook page regards to the TDL, even though the differences between the models are
categories, and Facebook groups only (M1, M2, and M3 respectively), as not statistically significant (see Table 6), Fig. 3(b) shows that similar
well as the AUC of the most complete model (M8). In terms of AUC, M1 conclusions as for the AUC can be made.
and M2 appear to be comparable to M8. To determine whether this is Moreover, Table 6 shows that, with an AUC of 0.58, M3 (i.e., model
statistically true, we inspect the results of the combined Alpayadin F-test including only Facebook groups) is by far the worst performing model in
with Holm corrections for family-wise error in Table 6. The lower di­ terms of AUC, as it is significantly different from all other models, except
agonal of Table 6 presents the adjusted p-values for the differences in M6. For both AUC and TDL, M3 performs significantly worse than M4 at
AUC, whereas the upper diagonal displays the p-values for the differ­ the 1% significance level. Hence, adding features related to Facebook
ences in TDL. The significant differences are marked in italic. The F-test pages and page categories leads to a significant increase in TDL from
confirms that, in terms of AUC, M1 and M2 do not perform significantly

Fig. 2. Cross-validated median (a) AUC and (b) TDL of M1, M2, M3, and M8.

8
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 6
Alpayadin F-test with Holm corrections.
Adj. p TDL

M1 M2 M3 M4 M5 M6 M7 M8

AUC M1 0.7641 0.5710 0.7641 0.7641 0.7641 0.7641 0.7641

M2 0.6832 0.3873 0.7641 0.7641 0.4459 0.7641 0.7641
M3 0.0138** 0.0105** 0.0096*** 0.1670 0.4349 0.4353 0.0844*
M4 0.6832 0.6832 0.0031*** 0.7641 0.5710 0.7641 0.7641
M5 0.6832 0.6832 0.0001*** 0.6832 0.4873 0.7641 0.7641
M6 0.5025 0.6832 0.4763 0.6443 0.3448 0.7641 0.2676
M7 0.6832 0.6283 0.0099*** 0.6832 0.6832 0.6832 0.7641
M8 0.6832 0.6832 0.0007*** 0.6832 0.6832 0.0973* 0.6832
*
p < 0.1
**
p < 0.05
***
p < 0.01

Fig. 3. Cross-validated median (a) AUC and (b) TDL of M4, M5, M6, M7, and M8.

1.34 to 3.40 (see Figs. 2(b) and 3(b)). These findings suggest that, in
comparison to Facebook groups, Facebook pages and their categories
are especially valuable for the prediction of donation behavior of
Facebook fans.
The importance of Facebook pages and categories of Facebook pages
is in line with existing literature regarding social media analytics.
Kosinski et al. [37] have previously proven the predictive value of
Facebook pages for a wide range of personal characteristics. With
regards to the categories of liked Facebook pages, Zhang and Pennac­
chiotti [61] found that these features solely are enough to accurately
predict users’ online purchase behavior. They state that this can be
explained by the fact that Facebook users like specific categories in line
with their personal interests.

4.3. Predictors

The final goal of this research is to determine which specific features

are most important in the prediction of donation behavior. To do so, we
examine variable importances for our most complete model (i.e., M8).
For a regularized logistic regression, the importance of each variable can Fig. 4. Variable importance plot of M8.
be measured by its corresponding coefficient (i.e., β coefficient). In
contrast to other approaches, LASSO regression allows to determine
which features exhibit the strongest effects [56], as coefficients that do star). Hence, Fig. 4 allows us to confirm what type of Facebook data is
not add any value to the performance of the model are shrunken towards most important for the prediction of donation behavior.
zero. Hence, the smaller the coefficient, the smaller the importance of Fig. 4 shows that the top predictor (i.e., the variable with rank 1) is
the corresponding variable. Fig. 4 shows a scree plot of the 25 most related to categories of Facebook pages. Furthermore, we find that
important variables of M8. Specifically, this plot depicts the absolute amongst the 25 most important variables 11 are related to Facebook
value of the β coefficients in descending order on the y-axis, against their pages, 10 to page categories, and only 4 to Facebook groups. These re­
rank on the x-axis. The variable with the highest absolute coefficient sults confirm the importance of variables related to Facebook pages and
receives rank 1, the one with the second highest absolute coefficient page categories for the prediction of donation behavior. Frequency
receives rank 2 and so on. The variables in our model are the SVD di­ variables do not appear in the top 25 and we can, therefore, conclude
mensions related to the user-page, user-category, and user-group that these are less important. Similarly, gender is not one of the 25 most
matrices. The shape of the points in the variable importance plot rep­ important variables.
resents the underlying type of Facebook data (i.e., pages are rhombuses, Finally, we take a closer look at the underlying pages, categories and
categories squares, groups triangles, frequency circles, and gender a

9
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

Table 7 groups of the top ten most important variables. Recall that most of our
Top ten most important variables (i.e., SVD dimensions) of M8, along with the variables are represented by the dimensions of SVD. Table 7 then dis­
most related Facebook pages/categories/groups. plays the underlying Facebook pages/categories/groups that make up
1. Category 2. Group 3. Group the top ten SVD dimensions, measured by the absolute value of the β
Bar Namur, Belgium Police Control Belgiumb
coefficients for M8. By inspecting the former, we can gain insights into
Dance- and nightclub 2nd hand clothes Info radars, police what distinguishes donors from non-donors and what aspects are most
NIVELLESb controls Liègeb important for the prediction of donation behavior of Facebook fans.
Pub Selling, buying services in You are a real Liégois…b Most pages and groups in Table 7 have been translated from Dutch or
Brusselsb
French. If applicable, the original language is indicated, as this may hold
Publisher 2nd hand and new items for Recommended
sale, Antwerpa restaurants in Liègeb valuable information for the interpretation of the dimensions. Further­
Public service Bart’s waya Liégeoises Liégeoisb more, when deemed necessary for interpretation, extra information is
Café Halle Selling, Free or Radars Liègeb given between brackets in italic.
Trading groupa Table 7 shows that the most important variable is strongly related to
Education For sale in the region of Most beautiful pictures of
Monsb Liègeb
categories describing aspects of nightlife (e.g., Bar, Dance- and night­
Higher education and For sale in Mons and its Bob from Liègeb club, and Pub) as well as education (e.g., Education, and Higher edu­
university environsb cation and university). In other words, this dimension is most likely
Business service All cars for sale in Belgiumb Gastronomy from Liègeb associated with students and young adults. These findings are in line
Jewelry/watches [Link] Police, Accidents, Road
with existing literature as it is generally accepted that the decision, as
Conditionsb
well as the amount to donate, are related to characteristics such as age
4. Page 5. Group and education (e.g., [12,51]). The second most important variable in our
model is a dimension related to second hand and sales groups and could,
Studio Brussel (radio VIRTUAL FLEA MARKET
station) LIEGEb therefore, describe users that want to save money and/or consume
Stromae Zoo of Antwerpa responsibly. The third most important dimension clearly describes
[Link] - Be the For sale Brusselsb people from Liège, a city in the French-speaking part of Belgium. The
first to discover fourth most important variable in the prediction of donation behavior is
De Ideale Wereld (tv Looking for apartment(− sharing) Brusselsb
show)
strongly related to Facebook pages of left-wing and humoristic media
Doctors Without Border JOBS BELGIUM channels. The sixth most important dimension consists of Facebook
(Belgium)a pages of several well-known brands (e.g., CLUSE, Loavies, Coca-cola).
Humo (magazine) Info radars and controls- Hence, this variable could be a measure for materialism and describe
Charleroib
users that appreciate brands. Previous research by Bennett [7] shows
Nutella Green bio zero wasteb
De Morgen (newspaper) Bouddhism that, in contrast to what one would expect, materialism positively in­
UNICEF Belgium Second hand books fluences the generosity of donations. Finally, the seventh most impor­
Brusselsb tant variable of M8 is related to NPOs (e.g. Doctors Without Borders,
UNILAD 2ND HAND CLOTHES Amnesty International, and UNICEF).
CHARLEROIb
In summary, Facebook fans that become donors differ from non-
6. Page 7. Page 8. Category donors in terms of their life stage, which part of Belgium they come
from, the media channels they like, but also in terms of materialism,
Sneaker District Doctors Without Borders Touring company
(Belgium)b
responsible consumption and interest in (other) NPOs. Our findings are
[Link] - Be the Amnesty International Business service in line with previous research on donation behavior, where de­
first to discover Belgiumb mographics and socio-economic profiles [12,51], but also personal
Guts & Gusto UNICEF Belgium Café values and personality traits [7], are found to have an impact on the
CLUSE Amnesty International Magazine
decision to donate.
Loavies GuiHome vous détend Hairdresser
(humorist)
Colourful Rebel Doctors Without Bordersb Publisher 5. Conclusion and practical implications
Coca-Cola UNICEF Performance Art
TravelBird Barack Obama Higher education and
This research shows how NPOs can harness the power of social media
university
LADbible Doctors Without Border Medical company
in CRM. Specifically, our aim is to demonstrate how nonprofit organi­
(Belgium)a zations can use social media (data) for acquisition modeling. First, we
EF Belgium - Study Doctors of the World Telecommunication build a predictive model to determine whether it is possible to accurately
abroad Belgiumb company predict donation behavior using Facebook data. Given the considerable
number of Facebook pages, categories and groups, we evaluate different
9. Category 10. Category
combinations of DR techniques (i.e., SVD, NMF, and LDA) and predic­
Organization Event planner tion algorithms (LR, KNN, BT, RF, AB, XGB, NN) to determine which
Public organization Hairdresser
combination of techniques is best suited for this task. Second, we build
Society- and Society- and culturewebsite
culturewebsite several models based on different subsets of variables and computed
Cosmetics store Education variable importances to gain insights into the (most important) pre­
Health- and Youth organization dictors of donation behavior.
wellnesswebsite
The results indicate that acquisition of Facebook fans can be pre­
Magazine Library
Public service Non-governmental
dicted on the basis of Facebook data with high predictive accuracy. With
organization (NGO) an AUC of 0.72 and a TDL of 3.33, the combination of SVD and LR is
Other Dance- and nightclub found to be the top performer. Interestingly, the combination of SVD and
Movie− /tv-studio Sports LR outperformed more advanced ensemble modeling techniques. This
Entrepreneur Fastfoodrestaurant
emphasizes that data preparation and feature engineering mainly drive
a
Translated from Dutch. predictive performance and not the algorithm [18]. Our results are
b
Translated from French. important to NPOs as they represent an alternative way to current
acquisition practices. Traditionally, donor acquisition is extremely

10
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

expensive because nonprofit organizations have to purchase external data set (i.e., only 1.23% of the Facebook fans are donors). To solve this
databases. This study shows that it is possible to build an accurate problem we performed random oversampling on the training set until a
acquisition model using only Facebook data and, therefore, presents a 50/50 distribution was reached. However, considering the limited
cheaper and effective way of identifying new donors. Furthermore, our number of events (i.e., 96 donors) and the great number of features (i.e.,
results serve as a decision support system to NPOs who want to use social 303), there will be a lot of similarity amongst the training samples,
media data for acquisition. Our data collection framework provides which could lead to an overestimation of the results. Since a rigorous
guidance as to how to gather the prospect list and the relevant social experimental set-up with a strict separation between training and test set
media data. Our analytical framework recommends which combination on each fold was followed, we believe that our results have value and
of data analytical techniques to use to construct an accurate predictive can be interpreted. Additional analyses proved that the models based on
model. Based on the predictions of the resulting acquisition model, as DR techniques generally do not overfit, whereas the binary models (i.e.,
well as the available budget, NPOs can directly determine which Face­ the models not using DR) do exhibit overfitting. Furthermore, the class
book fans to target during acquisition campaigns. imbalance is inherent to social media: the number of fans and followers
The second objective of this study is to determine what type of of NPOs is ever increasing but only very few of them actually donate,
Facebook features and what predictors are most important for the pre­ which makes resampling inevitable. Since a large benchmark study of
diction of donation behavior. The results reveal that variables related to data resampling methods is not within the scope of this study, we
Facebook pages and page categories are most valuable. Variables related decided not to implement other strategies. One avenue for future
to Facebook groups, as well as frequency variables and gender, are found research can be to study the impact of other resampling methods (e.g.,
to be less important. These insights are valuable to NPOs as they provide SMOTE) on our results.
guidance as to which type of predictors to include in acquisition models. Third, our aim is to present a decision support system that allows
Because collecting and preprocessing data is expensive, it could be NPOs to predict acquisition based on solely social media data. Hence,
worthwhile to implement acquisition models using only Facebook pages the acquisition models in this study are built on Facebook data only. As a
or Facebook page categories. With AUCs (TDLs) of 0.70 (2.92) and 0.68 topic for future research, the predictive value of social media data could
(3.00) respectively, these models do not perform significantly worse be compared to that of traditional data sources (e.g., external data). In
than the most complete model. Hence, our results allow NPOs to create their study, Meire et al. [48] evaluate the added value of social media
predictive acquisition models that are not only accurate but also as data over commercially purchased and website data for acquisition
efficient as possible. Finally, we provide insight into the individual models in a B2B setting. A similar study can be set up in the nonprofit
characteristics that differentiate donors from non-donors. Inspection of sector. In summary, whereas we focused on the value of different types
the top dimensions shows that aspects such as life stage, residence, of Facebook features, future research could investigate the (added)
materialism, responsible consumption, and interest in (other) NPOs are value of different data sources for acquisition modeling in the nonprofit
most important in predicting first-time donation behavior. Insights into sector.
these characteristics can help NPOs gain a better understanding of their
online donor base and tailor appropriate marketing campaigns. References
In summary, this study proves the value of social media to non­
profits. Specifically, we demonstrate how nonprofits can use Facebook [1] E. Alpaydin, Combined 5× 2 cv F test for comparing supervised classification
learning algorithms, Neural Comput. 11 (8) (1999) 1885–1892.
(data) to identify a pool of interested individuals (i.e., Facebook fans) [2] T. Althoff, J. Leskovec, Donor retention in online crowdfunding communities: A
and predict which ones are most likely to become a donor. As such, we case study of donorschoose. org, in: Proceedings of the 24th International
offer a potential solution to the current expensive acquisition practices. Conference on World Wide Web, International World Wide Web Conferences
Steering Committee, 2015, May, pp. 34–44.
Furthermore, our analyses can easily be reproduced and implemented [3] Y. Bachrach, M. Kosinski, T. Graepel, P. Kohli, D. Stillwell, Personality and patterns
by NPOs. Our results offer guidance regarding which methodology to of Facebook usage, in: Proceedings of the 4th Annual ACM Web Science
use, as well as which Facebook features to consider. Moreover, we hope Conference, ACM, 2012, June, pp. 24–32.
[4] B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, J. Suykens, J. Vanthienen,
that this study encourages nonprofit organizations to use Facebook as a Benchmarking state-of-the-art classification algorithms for credit scoring, J. Oper.
tool for predictive targeted marketing rather than for simple Res. Soc. 54 (6) (2003) 627–635.
communication. [5] M. Ballings, D. Van den Poel, CRM in social media: predicting increases in
Facebook usage frequency, Eur. J. Oper. Res. 244 (1) (2015) 248–260.
[6] R. Bekkers, Traditional and health-related philanthropy: the role of resources and
6. Limitations and future research personality, Soc. Psychol. Q. 69 (4) (2006) 349–366.
[7] R. Bennett, Factors underlying the inclination to donate to particular types of
charity, Int. J. Nonprofit Volunt. Sect. Mark. 8 (1) (2003) 12–29.
This study contributes to research in donor acquisition in the
[8] M. Bogaert, M. Ballings, M. Hosten, D. Van den Poel, Identifying soccer players on
nonprofit sector by creating a predictive model using social media data. Facebook through predictive analytics, Decis. Anal. 14 (4) (2017) 274–297.
Nevertheless, as this domain is relatively unexplored, several limitations [9] M. Bogaert, M. Ballings, D. Van den Poel, The added value of Facebook friends data
in event attendance prediction, Decis. Support. Syst. 82 (2016) 26–34.
and opportunities for further research arise.
[10] M. Bogaert, M. Ballings, D. Van den Poel, Evaluating the importance of different
A first limitation of this study is related to our data collection communication types in romantic tie prediction on social media, Ann. Oper. Res.
framework. Since we exclusively collected Facebook data of public 263 (1–2) (2018) 501–527.
Facebook profiles, this study may suffer from selection effects. It might [11] L. Breiman, Random forests, Mach. Learn. 45 (1) (2001) 5–32.
[12] S. Brown, K. Taylor, Charitable Behaviour and the Big Five Personality Traits:
be that some Facebook fans are unwilling to publicly share their data Evidence from UK Panel Data, 2015.
and that these Facebook fans have a different behavior than the fans in [13] J. Burez, D. Van den Poel, Handling class imbalance in customer churn prediction,
our sample. An alternative to our data collection framework would be to Expert Syst. Appl. 36 (3) (2009) 4626–4636.
[14] G. Carlo, M.A. Okun, G.P. Knight, M.R.T. de Guzman, The interplay of traits and
create a Facebook app that collects individual data with users’ consent motives on volunteering: agreeableness, extraversion and prosocial value
(e.g., [5]). However, due to recent scandals surrounding Facebook [58], motivation, Personal. Individ. Differ. 38 (6) (2005) 1293–1305.
very few users will allow this. Also, the possibilities of such an app have [15] K. Chell, G. Mortimer, Investigating online recognition for blood donor retention:
an experiential donor value approach, Int. J. Nonprofit Volunt. Sect. Mark. 19 (2)
been restricted considerably. Hence, organizations that want to harness (2014) 143–163.
the power of social media will have no other choice but to use a data [16] J. Clark, F. Provost, Matrix-Factorization-Based Dimensionality Reduction in the
collection framework similar to ours and will, therefore, be confronted Predictive Modeling Process: A Design Science Perspective, 2016.
[17] C. Courtois, P. Verdegem, Like to engage: A multi-level analysis of connective
to the same limitation. As such, this study is valuable to both researchers
action on Facebook, in: ICA (International Communication Association), Annual
and practitioners, as it demonstrates how Facebook data can be used in Conference, 2015.
this time and age.
A second limitation results from the strong class imbalance in our

11
L. Schetgen et al. Decision Support Systems 141 (2021) 113446

[18] K. Coussement, S. Lessmann, G. Verstraeten, A comparative analysis of data [44] S. Lessmann, B. Baesens, H.V. Seow, L.C. Thomas, Benchmarking state-of-the-art
preparation algorithms for customer churn prediction: a case study in the classification algorithms for credit scoring: an update of research, Eur. J. Oper. Res.
telecommunication industry, Decis. Support. Syst. 95 (2017) 27–36. 247 (1) (2015) 124–136.
[19] R. Cui, S. Gallino, A. Moreno, D.J. Zhang, The operational value of social media [45] E. Lucas, Reinventing the rattling tin: how UK charities use Facebook in
information, Prod. Oper. Manag. 27 (10) (2018) 1749–1769. fundraising, Int. J. Nonprofit Volunt. Sect. Mark. 22 (2) (2017).
[20] A. De Caigny, K. Coussement, K.W. De Bock, S. Lessmann, Incorporating textual [46] J. Mandel, Use of the singular value decomposition in regression analysis, Am. Stat.
information in customer churn prediction models based on a convolutional neural 36 (1) (1982) 15–24.
network, Int. J. Forecast. 36 (4) (2020) 1563–1578. [47] D. Martens, F. Provost, J. Clark, E.J. de Fortuny, Mining massive fine-grained
[21] J. D’Haen, D. Van den Poel, D. Thorleuchter, D.F. Benoit, Integrating expert behavior data to improve predictive analytics, MIS Q. 40 (4) (2016).
knowledge and multilingual web crawling data in a lead qualification system, [48] M. Meire, M. Ballings, D. Van den Poel, The added value of social media data in
Decis. Support. Syst. 82 (2016) 69–78. B2B customer acquisition systems: a real-life experiment, Decis. Support. Syst. 104
[22] T.G. Dietterich, Approximate statistical tests for comparing supervised (2017) 26–37.
classification learning algorithms, Neural Comput. 10 (7) (1998) 1895–1923. [49] M. Meire, K. Hewett, M. Ballings, V. Kumar, D. Van den Poel, The role of marketer-
[23] T.G. Dietterich, Ensemble methods in machine learning, in: International generated content in customer engagement marketing, J. Mark. 83 (6) (2019)
Workshop on Multiple Classifier Systems, Springer, Berlin, Heidelberg, 2000, June, 21–42.
pp. 1–15. [50] S. Praet, P. Van Aelst, D. Martens, I Like, Therefore I Am: Predictive Modeling to
[24] B. Enjolras, K. Steen-Johnsen, D. Wollebæk, Social media and mobilization to Gain Insights in Political Preference in a Multi-Party System, 2018.
offline demonstrations: transcending participatory divides? New Media Soc. 15 (6) [51] A. Sargeant, Charitable giving: towards a model of donor behavior, J. Mark.
(2013) 890–908. Manag. 15 (4) (1999) 215–238.
[25] Facebook, Newsroom – Key facts. [Link] 2020. [52] A. Sargeant, Relationship fundraising: how to keep donors loyal, Nonprofit Manag.
[26] H. Farrow, Y.C. Yuan, Building stronger ties with alumni through Facebook to Leader 12 (2) (2001) 177–192.
increase volunteerism and charitable giving, J. Comput.-Mediat. Commun. 16 (3) [53] K.S. Schlumpf, S.A. Glynn, G.B. Schreiber, D.J. Wright, W. Randolph Steele, Y Tu,
(2011) 445–464. National Heart, Lung, and Blood Institute Retrovirus Epidemiology Donor Study,
[27] E. Ferguson, Conscientiousness, emotional stability, perceived control and the Factors influencing donor return, Transfusion 48 (2) (2008) 264–272.
frequency, recency, rate and years of blood donor behaviour, Br. J. Health Psychol. [54] N. Schröder, A. Falke, H. Hruschka, T. Reutterer, Analyzing the browsing basket: a
9 (3) (2004) 293–314. latent interests-based segmentation tool, J. Interact. Mark. 47 (2019) 181–197.
[28] J. Friedman, T. Hastie, R. Tibshirani, Additive logistic regression: a statistical view [55] D. Thorleuchter, D. Van den Poel, A. Prinzie, Analyzing existing customers’
of boosting (with discussion and a rejoinder by the authors), Ann. Stat. 28 (2) websites to improve the customer acquisition process as well as the profitability
(2000) 337–407. prediction in B-to-B marketing, Expert Syst. Appl. 39 (3) (2012) 2597–2605.
[29] J.H. Friedman, Greedy function approximation: a gradient boosting machine, Ann. [56] R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser.
Stat. (2001) 1189–1232. B Methodol. 58 (1) (1996) 267–288.
[30] S. García, A. Fernández, J. Luengo, F. Herrera, Advanced nonparametric tests for [57] J.W. van Dam, M. van de Velden, Online profiling and clustering of Facebook users,
multiple comparisons in the design of experiments in computational intelligence Decis. Support. Syst. 70 (2015) 60–72.
and data mining: experimental analysis of power, Inf. Sci. 180 (10) (2010) [58] M. Vengattil, Facebook Suspends Tens of Thousands of Apps in Response to
2044–2064. Cambridge Analytics Row. Reuters, Retrieved from, [Link]
[31] J.T. Garner, L.T. Garner, Volunteering an opinion: organizational voice and 2019, September 20.
volunteer retention in nonprofit organizations, Nonprofit Volunt. Sect. Q. 40 (5) [59] G.A. Verhaert, D. Van den Poel, Empathy as added value in predicting donation
(2011) 813–828. behavior, J. Bus. Res. 64 (12) (2011) 1288–1295.
[32] M. Germain, S.A. Glynn, G.B. Schreiber, S. Gélinas, M. King, M. Jones, Y. Tu, [60] A.M. Warren, A. Sulaiman, N.I. Jaafar, Understanding civic engagement behaviour
Determinants of return behavior: a comparison of current and lapsed donors, on Facebook from a social capital theory perspective, Behav. Inform. Technol. 34
Transfusion 47 (10) (2007) 1862–1870. (2) (2015) 163–175.
[33] G. Godin, M. Conner, P. Sheeran, A. Bélanger-Gravel, M. Germain, Determinants of [61] Y. Zhang, M. Pennacchiotti, Predicting purchase behaviors from social media, in:
repeated blood donation among new and experienced blood donors, Transfusion Proceedings of the 22nd International Conference on World Wide Web, ACM,
47 (9) (2007) 1607–1615. 2013, May, pp. 1521–1532.
[34] J. Harrison, J.Y. Kim, RSelenium: R Bindings for “Selenium WebDriver.”, 2020.
[35] J.L. Hsu, G.Y. Liang, C.P. Tien, Social concerns and willingness to support charities,
Lisa Schetgen (PhD) is a PhD researcher at Ghent University. She has received her B.S. in
Soc. Behav. Personal. Int. J. 33 (2) (2005) 189–200.
Business Engineering and M.S. in Business Engineering: Data Analytics from Ghent Uni­
[36] M. Kosinski, Y. Bachrach, P. Kohli, D. Stillwell, T. Graepel, Manifestations of user
versity. Her research interests are applications of analytics in social media and CRM.
personality in website choice and behaviour on online social networks, Mach.
Learn. 95 (3) (2014) 357–380.
[37] M. Kosinski, D. Stillwell, T. Graepel, Private traits and attributes are predictable Matthias Bogaert (PhD) is an Assistant Professor of Data Analytics at Ghent University,
from digital records of human behavior, Proc. Natl. Acad. Sci. 110 (15) (2013) where he teaches Social Media and Web Analytics and Predictive and Prescriptive Ana­
5802–5805. lytics. He is also a Visiting Professor at the University of Namur, where he teaches Business
[38] M. Kosinski, Y. Wang, H. Lakkaraju, J. Leskovec, Mining big data to extract Analytics and Big Data. His research focuses on applications of descriptive, predictive, and
patterns and predict real-life outcomes, Psychol. Methods 21 (4) (2016) 493. prescriptive analytics in social media and CRM. He has published in journals such as the
[39] L. Kristoffersen, S. Singh, Successful application of a customer relationship European Journal of Operational Research, Decision Support Systems, and OMEGA: the
management program in a nonprofit organization, J. Mark. Theory Pract. 12 (2) International Journal of Management Science.
(2004) 28–42.
[40] B. Larivière, D. Van den Poel, Investigating the role of product features in
Dirk Van den Poel (PhD) is a Senior Full Professor of Data Analytics/Big Data at Ghent
preventing customer churn, by using survival analysis and choice modeling: the
University, Belgium. He teaches courses such as Big Data, Apache Spark, and Analytical
case of financial services, Expert Syst. Appl. 27 (2) (2004) 277–285.
Customer Relationship Management. His major research interests are in the field of
[41] Y.K. Lee, C.T. Chang, Who gives what to charity? Characteristics affecting donation
analytical CRM: customer acquisition, churn, upsell/cross-sell, and win-back modeling.
behavior, Soc. Behav. Personal. Int. J. 35 (9) (2007) 1173–1180.
His methodological interests include ensemble classification methods and big data ana­
[42] D.D. Lee, H.S. Seung, Algorithms for non-negative matrix factorization, in:
lytics. He has graduated 15+ PhD students, and co-authored 100+ international peer-
Advances in Neural Information Processing Systems, 2001, pp. 556–562.
reviewed papers in journals such as the Journal of Statistical Software, Journal of Mar­
[43] A. Lemmens, C. Croux, Bagging and boosting classification trees to predict churn,
keting, and IEEE Transactions on Power Systems.
J. Mark. Res. 43 (2) (2006) 276–286.

12