19-1

Factor Analysis
 Discuss factor analysis and its difference with
respect to other data analysis techniques
including Anova, Regression, and Discriminant
Analysis.
 Give some examples of applications of Factor
Analysis.
 Discuss the formulation of the model in Factor
Analysis
 Steps associated with conducting Factor
Analysis
 Two common methods of doing Factor
Analysis
 Rotation of Factor
 Determination of fit of a model.
 19-2

Factor Analysis
 Factor analysis is a general name denoting a class
of procedures primarily used for data reduction and
summarization.
 Factor analysis is an interdependence technique
in that an entire set of interdependent relationships is
examined without making the distinction between
dependent and independent variables.
 Factor analysis is used in the following
circumstances:
 To identify underlying dimensions, or factors, that
explain the correlations among a set of variables.
 To identify a new, smaller, set of uncorrelated
variables to replace the original set of correlated
variables in subsequent multivariate analysis
 Factor analysis assumes that the observed
variables are linear combinations of some
underlying (hypothetical or
observable)factors.
Some of the factors are assumed to be
common to 2 or more variables and some are
assumed to be unique to each variable.

Only common factors (which are assumed

much smaller in number than the number of
observed variable) contribute to the
covariation among observed variables.
 Why to use Factor Analysis
What makes consumer to buy a product.
There may be 20 purchasing criteria.
How to Identify latent or underlying factors from an
array of seemingly important criteria?

How to identify smaller set of salient variables from

larger set for use in subsequent multivariariate analysis.

Factor analysis is a set of technique which , by analyzing

co-relation of variables, reduces their number in to fewer
factors which explains much of the original data more
economically.
 Validity
Construct Validity : A scale possesses validity
when it actually measures what it claims to measure.
When some trait is to be measured and it is not manifested
in any simple and specific behavior, then it is difficult to
measure that trait.
 19-6

How to select construct Indicators

Construct indicators should be internally consistent

There are optimal magnitudes of correlations between

items

Within-construct correlation must be greater than

between constructs
 19-7

Assessment Effect Indicator models

Intelligence Trust
Effect Indicator Model

11 12 13 14 21 22 23 24

GP Cl Ver GI H H2 B1 B2

€ €2 €3 €4 € €2 €3 €4
 19-8

Effect Vs Causal Indicator Measurement Model


F1=Trust F2=SES
Effect Indicator Model

Causal Indicator Model

11 12 13 14  11  12  13 14

H1 H2 B1 B2 Ed Occ Inc Ne

€ €2 €3 €4

Y1= ij F1+ €1 F2=  11 X1+  12 X2+  13 X3+  14 X4+ξ1

 19-9

Effect Vs Causal Indicator Measurement Model

F1 F2
Effect Indicator Model

Causal Indicator Model


11 12 13 14  11
14
 12  13

Y Y Y Y X X X X

€ €2 €3 €4

Y1= ij F1+I F2=  11 X1+  12 X2+  13 X3+ 14 X4+ξ1

Causal Indicators
Socioeconomic Status:
Education; Income; Occupational Prestige; Neighborhood.

Exposure to Discrimination:
Race, Sex, Age and Disabilities.

Life Stress:
Job Loss, divorce, recent bodily injury, death in family

Structural Bonding
Economic; legal; Strategic; Technical; Formal relationship.

Dependence:
Cost;Benefit;Available of alterantives
 Effect Indicators
Trust:
Honest, Benevolence

Intelligence:
GPA, In-class behavior;test-taking ability
under anxiety;Verbal.
Social Bonding:
Personal; Social; family; personal interaction.
Structural Bonding:“degree to which certain ties and link hold a buyer
and seller together in a relationship as a result of some mutually
beneficial economic, strategic, technological, or organizational
objective”. (Williams et. al., 1998)
Scale Used:1= Very unimportant----- 7=very important.
1. Economic benefits associated with your relationship with your bank.
2. Formal contracts or agreements between your firm and Bank
3. Sharing of industry or competitive information between your firm
and bank
4. Electronic ties or linkages between your business firm and bank
(such as electronics mail, access to computerized banking transaction
system etc.)
5. Exchange cards, gifts and the like on birthdays, holidays or on
special events between your firm and bank
Trust:Trust is the perceived credibility and benevolence
of a target of trust (cf.Ganesan 1994; Kumar et al 1995;
Doney and Canon 1997))
Scale Used:1= Strongly disagree----- 7=Strongly agree
1. Our bank rep. has been frank in dealing with us.
2. Our bank can always be counted on to do right thing
3. Our bank does not make false claims.
4. Though circumstances may change, I believe that
our bank will always be ready and willing to offer us
assistance and support.
5. While taking important decision our bank is
concerned about our welfare
6. Based on past experience I can say our bank is
trustworthy
Social Bonding:“is the bonding that takes place between
the individuals of the buyer and seller organization on a
personal and social basis.(Williams et. al., 1998)

Scale Used:
1= Very unimportant----- 7=very important.
1. Establishing personal relationship with you and your
bank representative.
2. Sharing of personal advice or support with your bank
representative.
3.Feelings of belongingness or acceptance between you
and your bank representative
4. Friendship between you and your bank
representative.
 19-15

Statistics Associated with Factor Analysis

 Communality. Communality is the amount of
variance a variable shares with all the other variables
being considered. This is also the proportion of
variance explained by the common factors.
 Eigenvalue. The eigenvalue represents the total
variance explained by each factor.
 Factor loadings. Factor loadings are simple
correlations between the variables and the factors.
 Factor loading plot. A factor loading plot is a plot
of the original variables using the factor loadings as
coordinates.
 Factor matrix. A factor matrix contains the factor
loadings of all the variables on all the factors
extracted.
 19-16

Statistics Associated with Factor Analysis

 Factor scores. Factor scores are composite scores
estimated for each respondent on the derived factors.
 Percentage of variance. The percentage of the total
variance attributed to each factor.
 Residuals are the differences between the observed
correlations, as given in the input correlation matrix, and
the reproduced correlations, as estimated from the factor
matrix.
 Scree plot. A scree plot is a plot of the Eigenvalues
against the number of factors in order of extraction.
 19-17

Factor Analysis Model

Mathematically, each variable is expressed as a linear combination
of underlying factors. The covariation among the variables is
described in terms of a small number of common factors plus a
unique factor for each variable. If the variables are standardized,
the factor model may be represented as:

Xi = Ai 1F1 + Ai 2F2 + Ai 3F3 + . . . + AimFm + ViUi

where

Xi = i th standardized variable
Aij = standardized multiple regression coefficient of
variable i on common factor j
F = common factor
Vi = standardized regression coefficient of variable i on
unique factor i
Ui = the unique factor for variable i
m = number of common factors
 19-18

Factor Analysis Model

The unique factors are uncorrelated with each other and with the
common factors. The common factors themselves can be
expressed as linear combinations of the observed variables.

Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk

where

Fi = estimate of i th factor
Wi = weight or factor score coefficient
k = number of variables
 19-19

Conducting Factor Analysis

Fig 19.1 Problem formulation

Construction of the Correlation Matrix

Method of Factor Analysis

Determination of Number of Factors

Rotation of Factors

Interpretation of Factors

Calculation of Selection of
Factor Scores Surrogate Variables

Determination of Model Fit

Conducting Factor Analysis
19-20

Formulate the Problem

 The objectives of factor analysis should be identified.

 The variables to be included in the factor analysis
should be specified based on past research, theory,
and judgment of the researcher. It is important that
the variables be appropriately measured on an
interval or ratio scale.
 An appropriate sample size should be used. As a
rough guideline, there should be at least four or five
times as many observations (sample size) as there
are variables.
Conducting Factor Analysis
19-21

Determine the Number of Factors

 A Priori Determination. Sometimes, because of

prior knowledge, the researcher knows how many
factors to expect and thus can specify the number of
factors to be extracted beforehand.

 Determination Based on Eigenvalues. In this

approach, only factors with Eigenvalues greater than
1.0 are retained. An Eigenvalue represents the
amount of variance associated with the factor.
Hence, only factors with a variance greater than 1.0
are included. Factors with variance less than 1.0 are
no better than a single variable, since, due to
standardization, each variable has a variance of 1.0.
If the number of variables is less than 20, this
approach will result in a conservative number of
factors.
Conducting Factor Analysis
19-22

 The analytical process is based on a matrix of

correlations between the variables.
 Bartlett's test of sphericity. Bartlett's test of

sphericity is a test statistic used to examine the

hypothesis that the variables are uncorrelated in the
population. In other words, the population
correlation matrix is an identity matrix; each variable
correlates perfectly with itself (r = 1) but has no
correlation with the other variables (r = 0).
 Kaiser-Meyer-Olkin (KMO) measure of sampling
adequacy. The Kaiser-Meyer-Olkin (KMO) measure
of sampling adequacy is an index used to examine
the appropriateness of factor analysis. High values
(between 0.5 and 1.0) indicate factor analysis is
appropriate. Values below 0.5 imply that factor
analysis may not be appropriate.
Conducting Factor Analysis
19-23

Determine the Method of Factor Analysis

 In principal components analysis, the total variance in
the data is considered. The diagonal of the correlation
matrix consists of unities, and full variance is brought into
the factor matrix. Principal components analysis is
recommended when the primary concern is to determine
the minimum number of factors that will account for
maximum variance in the data for use in subsequent
multivariate analysis. The factors are called principal
components.

 In common factor analysis, the factors are estimated

based only on the common variance. Communalities are
inserted in the diagonal of the correlation matrix. This
method is appropriate when the primary concern is to
identify the underlying dimensions and the common
variance is of interest. This method is also known as
principal axis factoring.
 19-24

Conducting Factor Analysis

RESPONDENT
Table 19.1 NUMBER V1 V2 V3 V4 V5 V6
1 7.00 3.00 6.00 4.00 2.00 4.00
2 1.00 3.00 2.00 4.00 5.00 4.00
3 6.00 2.00 7.00 4.00 1.00 3.00
4 4.00 5.00 4.00 6.00 2.00 5.00
5 1.00 2.00 2.00 3.00 6.00 2.00
6 6.00 3.00 6.00 4.00 2.00 4.00
7 5.00 3.00 6.00 3.00 4.00 3.00
8 6.00 4.00 7.00 4.00 1.00 4.00
9 3.00 4.00 2.00 3.00 6.00 3.00
10 2.00 6.00 2.00 6.00 7.00 6.00
11 6.00 4.00 7.00 3.00 2.00 3.00
12 2.00 3.00 1.00 4.00 5.00 4.00
13 7.00 2.00 6.00 4.00 1.00 3.00
14 4.00 6.00 4.00 5.00 3.00 6.00
15 1.00 3.00 2.00 2.00 6.00 4.00
16 6.00 4.00 6.00 3.00 3.00 4.00
17 5.00 3.00 6.00 3.00 3.00 4.00
18 7.00 3.00 7.00 4.00 1.00 4.00
19 2.00 4.00 3.00 3.00 6.00 3.00
20 3.00 5.00 3.00 6.00 4.00 6.00
21 1.00 3.00 2.00 3.00 5.00 3.00
22 5.00 4.00 5.00 4.00 2.00 4.00
23 2.00 2.00 1.00 5.00 4.00 4.00
24 4.00 6.00 4.00 6.00 4.00 7.00
25 6.00 5.00 4.00 2.00 1.00 4.00
26 3.00 5.00 4.00 6.00 4.00 7.00
27 4.00 4.00 7.00 2.00 2.00 5.00
28 3.00 7.00 2.00 6.00 4.00 3.00
29 4.00 6.00 3.00 7.00 2.00 7.00
30 2.00 3.00 2.00 4.00 7.00 2.00
 19-25

Correlation Matrix
Table 19.2

Variables V1 V2 V3 V4 V5 V6
V1 1.000
V2 -0.530 1.000
V3 0.873 -0.155 1.000
V4 -0.086 0.572 -0.248 1.000
V5 -0.858 0.020 -0.778 -0.007 1.000
V6 0.004 0.640 -0.018 0.640 -0.136 1.000

High correlation among V1(Prevention of cavities),

V3(strong gums) and V5(prevention of tooth decay):Health
Benefit

High correlation among V2 (shiney teeth), v4(fresh bredth)

and v6(attractive teeth): (Social Benefit)
 19-26

Results of Principal Components Analysis

Table 19.3

The %ge of variance is calculated as the eigenvalue

divided by sum of all eigen value.

Initial Eigen values

Factor Eigen value % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488
3 0.442 7.360 89.848
4 0.341 5.688 95.536
5 0.183 3.044 98.580
6 0.085 1.420 100.000
 19-27

Results of Principal Components Analysis

Table 19.3 cont.
Extraction Sums of Squared Loadings
Factor Eigen value % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488
Factor Matrix
Variables Factor 1 Factor 2
V1 0.928 0.253
V2 -0.301 0.795
V3 0.936 0.131
V4 -0.342 0.789
V5 -0.869 -0.351
V6 -0.177 0.871
Rotation Sums of Squared Loadings
Factor Eigenvalue % of variance Cumulat. %
1 2.688 44.802 44.802
2 2.261 37.687 82.488
 19-28

Results of Principal Components Analysis

Table 19.3 cont.
Rotated Factor Matrix
Variables Factor 1 Factor 2
V1 0.962 -0.027
V2 -0.057 0.848
V3 0.934 -0.146
V4 -0.098 0.845
V5 -0.933 -0.084
V6 0.083 0.885

Factor Score Coefficient Matrix

Variables Factor 1 Factor 2
V1 0.358 0.011
V2 -0.001 0.375
V3 0.345 -0.043
V4 -0.017 0.377
V5 -0.350 -0.059
V6 0.052 0.395
Conducting Factor Analysis
19-29

Calculate Factor Scores

The factor scores for the ith factor may be estimated

as follows:

Fi = Wi1 X1 + Wi2 X2 + Wi3 X3 + . . . + Wik Xk

Conducting Factor Analysis
19-30

Determine the Number of Factors

 Determination Based on Scree Plot. A scree

plot is a plot of the Eigenvalues against the number
of factors in order of extraction. Experimental
evidence indicates that the point at which the scree
begins denotes the true number of factors.
Generally, the number of factors determined by a
scree plot will be one or a few more than that
determined by the Eigenvalue criterion.

 Determination Based on Percentage of

Variance. In this approach the number of factors
extracted is determined so that the cumulative
percentage of variance extracted by the factors
reaches a satisfactory level. It is recommended that
the factors extracted should account for at least 60%
of the variance.
 19-31

Scree Plot
Fig 19.2
3.0

2.5

2.0
Eigenvalue

1.5

1.0

0.5

0.0
1 2 3 4 5 6
Component Number
Conducting Factor Analysis
19-32

Rotate Factors

 Although the initial or unrotated factor matrix

indicates the relationship between the factors and
individual variables, it seldom results in factors that
can be interpreted, because the factors are
correlated with many variables. Therefore, through
rotation the factor matrix is transformed into a
simpler one that is easier to interpret.
 In rotating the factors, we would like each factor to
have nonzero, or significant, loadings or coefficients
for only some of the variables. Likewise, we would
like each variable to have nonzero or significant
loadings with only a few factors, if possible with only
one.
 The rotation is called orthogonal rotation if the
axes are maintained at right angles.
Conducting Factor Analysis
19-33

Rotate Factors

 The most commonly used method for rotation is the

varimax procedure. This is an orthogonal method
of rotation that minimizes the number of variables
with high loadings on a factor, thereby enhancing the
interpretability of the factors. Orthogonal rotation
results in factors that are uncorrelated.
 The rotation is called oblique rotation when the
axes are not maintained at right angles, and the
factors are correlated. Sometimes, allowing for
correlations among factors can simplify the factor
pattern matrix. Oblique rotation should be used
when factors in the population are likely to be
strongly correlated.
Conducting Factor Analysis
19-34

Interpret Factors

 A factor can then be interpreted in terms of the

variables that load high on it.
 Another useful aid in interpretation is to plot the
variables, using the factor loadings as coordinates.
Variables at the end of an axis are those that have
high loadings on only that factor, and hence describe
the factor.
 19-35

Factor Loading Plot

Fig 19.3
Rotated Component Matrix

Component
Variable 1 2
Component Plot in Rotated Space
Component 1 V1 0.962 -2.66E-02
V2 -5.72E-02 0.848
1.0  
V4 V6
V3 0.934 -0.146
V2
V4 -9.83E-02 0.854
Component 2
0.5
V5 -0.933 -8.40E-02

0.0
V1 V6 8.337E-02 0.885
 V5 V3
-0.5

-1.0

1.0 0.5 0.0 -0.5 -1.0

Conducting Factor Analysis
19-36

Select Surrogate Variables

 By examining the factor matrix, one could select for

each factor the variable with the highest loading on
that factor. That variable could then be used as a
surrogate variable for the associated factor.
 However, the choice is not as easy if two or more
variables have similarly high loadings. In such a
case, the choice between these variables should be
based on theoretical and measurement
considerations.
Conducting Factor Analysis
19-37

Determine the Model Fit

 The correlations between the variables can be

deduced or reproduced from the estimated
correlations between the variables and the factors.
 The differences between the observed correlations
(as given in the input correlation matrix) and the
reproduced correlations (as estimated from the factor
matrix) can be examined to determine model fit.
These differences are called residuals.
 19-38

Results of Principal Components Analysis

Table 19.3 cont.

The lower left triangle contains the reproduced

correlation matrix; the diagonal, the communalities;
the upper right triangle, the residuals between the
observed correlations and the reproduced
correlations.

Factor Score Coefficient Matrix

Variables V1 V2 V3 V4 V5 V6
V1 0.926 0.024 -0.029 0.031 0.038 -0.053
V2 -0.078 0.723 0.022 -0.158 0.038 -0.105
V3 0.902 -0.177 0.894 -0.031 0.081 0.033
V4 -0.117 0.730 -0.217 0.739 -0.027 -0.107
V5 -0.895 -0.018 -0.859 0.020 0.878 0.016
V6 0.057 0.746 -0.051 0.748 -0.152 0.790
 19-39

Results of Common Factor Analysis

Table 19.4

Communalities
Barlett test of sphericity
Variables Initial Extraction
V1 0.859 0.928 • Approx. Chi-Square = 111.314
V2 0.480 0.562 • df = 15
V3 0.814 0.836
• Significance = 0.00000
V4 0.543 0.600
V5 0.763 0.789 • Kaiser-Meyer-Olkin measure of
V6 0.587 0.723 sampling adequacy = 0.660

Initial Eigenvalues
Factor Eigenvalue % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488
3 0.442 7.360 89.848
4 0.341 5.688 95.536
5 0.183 3.044 98.580
6 0.085 1.420 100.000
 19-40

Results of Common Factor Analysis

Table 19.4 cont.
Extraction Sums of Squared Loadings
Factor Eigenvalue % of variance Cumulat. %
1 2.570 42.837 42.837
2 1.868 31.126 73.964
Factor Matrix
Variables Factor 1 Factor 2
V1 0.949 0.168
V2 -0.206 0.720
V3 0.914 0.038
V4 -0.246 0.734
V5 -0.850 -0.259
V6 -0.101 0.844

Rotation Sums of Squared Loadings

Factor Eigenvalue % of variance Cumulat. %
1 2.541 42.343 42.343
2 1.897 31.621 73.964
 19-41

Results of Common Factor Analysis

Table 19.4 cont.

Rotated Factor Matrix

Variables Factor 1 Factor 2
V1 0.963 -0.030
V2 -0.054 0.747
V3 0.902 -0.150
V4 -0.090 0.769
V5 -0.885 -0.079
V6 0.075 0.847

Factor Score Coefficient Matrix

Variables Factor 1 Factor 2
V1 0.628 0.101
V2 -0.024 0.253
V3 0.217 -0.169
V4 -0.023 0.271
V5 -0.166 -0.059
V6 0.083 0.500
 19-42

Results of Common Factor Analysis

Table 19.4 cont.

The lower left triangle contains the reproduced

correlation matrix; the diagonal, the communalities;
the upper right triangle, the residuals between the
observed correlations and the reproduced correlations.

Factor Score Coefficient Matrix

Variables V1 V2 V3 V4 V5 V6
V1 0.928 0.022 -0.000 0.024 -0.008 -0.042
V2 -0.075 0.562 0.006 -0.008 0.031 0.012
V3 0.873 -0.161 0.836 -0.005 0.008 0.042
V4 -0.110 0.580 -0.197 0.600 -0.025 -0.004
V5 -0.850 -0.012 -0.786 0.019 0.789 0.003
V6 0.046 0.629 -0.060 0.645 -0.133 0.723
 19-43

SPSS Windows

To select this procedures using SPSS for Windows click:

Analyze>Data Reduction>Factor …
 19-44

Factor Rotation
19-45
19-46