Factor Analysis
SPSS for Windows® Intermediate & Advanced Applied Statistics
Zayed University Office of Research SPSS for Windows® Workshop
Series
Presented by
Dr. Maher Khelifa
Associate Professor
Department of Humanities and Social Sciences
College of Arts and Sciences
© Dr. Maher Khelifa
Understanding Factor Analysis
2
This workshop discusses factor analysis as an
exploratory and confirmatory multivariate
technique.
© Dr. Maher Khelifa
Understanding Factor Analysis
3
Factor analysis is commonly used in:
Data reduction
Scale development
The evaluation of the psychometric quality of a
measure, and
The assessment of the dimensionality of a set of
variables.
© Dr. Maher Khelifa
Understanding Factor Analysis
4
Regardless of purpose, factor analysis is used in:
the determination of a small number of factors based on a
particular number of inter-related quantitative variables.
Unlike variables directly measured such as
speed, height, weight, etc., some variables such
as egoism, creativity, happiness, religiosity,
comfort are not a single measurable entity.
They are constructs that are derived from the
measurement of other, directly observable
variables .
© Dr. Maher Khelifa
Understanding Factor Analysis
5
Constructs are usually defined as unobservable latent
variables. E.g.:
motivation/love/hate/care/altruism/anxiety/worry/stress/product
quality/physical aptitude/democracy /reliability/power.
Example: the construct of teaching effectiveness. Several
variables are used to allow the measurement of such
construct (usually several scale items are used) because
the construct may include several dimensions.
Factor analysis measures not directly observable constructs
by measuring several of its underlying dimensions.
The identification of such underlying dimensions (factors)
simplifies the understanding and description of complex
constructs.
© Dr. Maher Khelifa
Understanding Factor Analysis
6
Generally, the number of factors is much smaller
than the number of measures.
Therefore, the expectation is that a factor
represents a set of measures.
From this angle, factor analysis is viewed as a data-
reduction technique as it reduces a large number
of overlapping variables to a smaller set of factors
that reflect construct(s) or different dimensions of
contruct(s).
© Dr. Maher Khelifa
Understanding Factor Analysis
7
The assumption of factor analysis is that
underlying dimensions (factors) can be used to
explain complex phenomena.
Observed correlations between variables result
from their sharing of factors.
Example: Correlations between a person’s test
scores might be linked to shared factors such
as general intelligence, critical thinking and
reasoning skills, reading comprehension etc.
© Dr. Maher Khelifa
Ingredients of a Good Factor Analysis
Solution
8
A major goal of factor analysis is to represent
relationships among sets of variables
parsimoniously yet keeping factors
meaningful.
A good factor solution is both simple and
interpretable.
When factors can be interpreted, new
insights are possible.
© Dr. Maher Khelifa
Application of Factor Analysis
9
This workshop will examine three common
applications of factor analysis:
Defining indicators of constructs
Defining dimensions for an existing measure
Selecting items or scales to be included in a measure.
© Dr. Maher Khelifa
Application of Factor Analysis
10
Defining indicators of constructs:
Ideally 4 or more measures should be chosen to represent
each construct of interest.
The choice of measures should, as much as possible, be
guided by theory, previous research, and logic.
© Dr. Maher Khelifa
Application of Factor Analysis
11
Defining dimensions for an existing measure:
In this case the variables to be analyzed are chosen by
the initial researcher and not the person conducting
the analysis.
Factor analysis is performed on a predetermined set
of items/scales.
Results of factor analysis may not always be
satisfactory:
The items or scales may be poor indicators of the
construct or constructs.
There may be too few items or scales to represent each
underlying dimension.
© Dr. Maher Khelifa
Application of Factor Analysis
12
Selecting items or scales to be included in a measure.
Factor analysis may be conducted to determine what
items or scales should be included and excluded from a
measure.
Results of the analysis should not be used alone in
making decisions of inclusions or exclusions. Decisions
should be taken in conjunction with the theory and what
is known about the construct(s) that the items or scales
assess.
© Dr. Maher Khelifa
Steps in Factor Analysis
13
Factor analysis usually proceeds in four
steps:
1st Step: the correlation matrix for all variables is
computed
2nd Step: Factor extraction
3rd Step: Factor rotation
4th Step: Make final decisions about the number of
underlying factors
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
14
1st Step: the correlation matrix
Generate a correlation matrix for all variables
Identify variables not related to other variables
If the correlation between variables are small, it is
unlikely that they share common factors (variables
must be related to each other for the factor model to
be appropriate).
Think of correlations in absolute value.
Correlation coefficients greater than 0.3 in absolute
value are indicative of acceptable correlations.
Examine visually the appropriateness of the factor
model.
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
15
Bartlett Test of Sphericity:
used to test the hypothesis the correlation matrix is an
identity matrix (all diagonal terms are 1 and all off-diagonal
terms are 0).
If the value of the test statistic for sphericity is large and the
associated significance level is small, it is unlikely that the
population correlation matrix is an identity.
If the hypothesis that the population correlation matrix is
an identity cannot be rejected because the observed
significance level is large, the use of the factor model
should be reconsidered.
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
16
The Kaiser-Meyer-Olkin (KMO) measure of sampling
adequacy:
is an index for comparing the magnitude of the observed
correlation coefficients to the magnitude of the partial
correlation coefficients.
The closer the KMO measure to 1 indicate a sizeable sampling
adequacy (.8 and higher are great, .7 is acceptable, .6 is
mediocre, less than .5 is unaccaptable ).
Reasonably large values are needed for a good factor analysis.
Small KMO values indicate that a factor analysis of the
variables may not be a good idea.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
17
2nd Step: Factor extraction
The primary objective of this stage is to determine the factors.
Initial decisions can be made here about the number of factors
underlying a set of measured variables.
Estimates of initial factors are obtained using Principal
components analysis.
The principal components analysis is the most commonly used
extraction method . Other factor extraction methods include:
Maximum likelihood method
Principal axis factoring
Alpha method
Unweighted lease squares method
Generalized least square method
Image factoring.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
18
In principal components analysis, linear
combinations of the observed variables are formed.
The 1st principal component is the combination that
accounts for the largest amount of variance in the
sample (1st extracted factor).
The 2nd principle component accounts for the next
largest amount of variance and is uncorrelated with
the first (2nd extracted factor).
Successive components explain progressively smaller
portions of the total sample variance, and all are
uncorrelated with each other.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
19
To decide on how many
factors we need to represent
the data, we use 2 statistical Total Variance Explained
criteria: Extraction Sums of Squared
Eigen Values, and Initial Eigenvalues Loadings
The Scree Plot. Comp % of Cumulativ % of Cumulativ
onent Total Variance e% Total Variance e%
1 3.046 30.465 30.465 3.046 30.465 30.465
The determination of the
2 1.801 18.011 48.476 1.801 18.011 48.476
number of factors is usually 3 1.009 10.091 58.566 1.009 10.091 58.566
done by considering only
factors with Eigen values 4 .934 9.336 67.902
greater than 1. 5 .840 8.404 76.307
6 .711 7.107 83.414
7 .574 5.737 89.151
Factors with a variance less
8 .440 4.396 93.547
than 1 are no better than a 9 .337 3.368 96.915
single variable, since each 10 .308 3.085 100.000
variable is expected to have a Extraction Method: Principal Component Analysis.
variance of 1.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
20
The examination of the Scree plot
provides a visual of the total variance
associated with each factor.
The steep slope shows the large factors.
The gradual trailing off (scree) shows
the rest of the factors usually lower
than an Eigen value of 1.
In choosing the number of factors, in
addition to the statistical criteria, one
should make initial decisions based on
conceptual and theoretical grounds.
At this stage, the decision about the
number of factors is not final.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
21
Component Matrix using Principle Component Analysis
Component Matrixa
Component
1 2 3
I discussed my frustrations and feelings with person(s) in school .771 -.271 .121
I tried to develop a step-by-step plan of action to remedy the problems .545 .530 .264
I expressed my emotions to my family and close friends .580 -.311 .265
I read, attended workshops, or sought someother educational approach to correct the .398 .356 -.374
problem
I tried to be emotionally honest with my self about the problems .436 .441 -.368
I sought advice from others on how I should solve the problems .705 -.362 .117
I explored the emotions caused by the problems .594 .184 -.537
I took direct action to try to correct the problems .074 .640 .443
I told someone I could trust about how I felt about the problems .752 -.351 .081
I put aside other activities so that I could work to solve the problems .225 .576 .272
Extraction Method: Principal Component Analysis.
a. 3 components extracted.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
22
3rd Step: Factor rotation.
In this step, factors are rotated.
Un-rotated factors are typically not very
interpretable (most factors are correlated with
may variables).
Factors are rotated to make them more
meaningful and easier to interpret (each variable
is associated with a minimal number of factors).
Different rotation methods may result in the
identification of somewhat different factors.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
23
The most popular rotational method is Varimax rotations.
Varimax use orthogonal rotations yielding uncorrelated
factors/components.
Varimax attempts to minimize the number of variables that
have high loadings on a factor. This enhances the
interpretability of the factors.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
24
Other common rotational method used include Oblique
rotations which yield correlated factors.
Oblique rotations are less frequently used because their
results are more difficult to summarize.
Other rotational methods include:
Quartimax (Orthogonal)
Equamax (Orthogonal)
Promax (oblique)
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
25
A factor is interpreted or named by examining the largest values
linking the factor to the measured variables in the rotated factor
matrix. Rotated Component Matrix a
Component
1 2 3
I discussed my frustrations and feelings with person(s) in school .803 .186 .050
I tried to develop a step-by-step plan of action to remedy the problems .270 .304 .694
I expressed my emotions to my family and close friends .706 -.036 .059
I read, attended workshops, or sought someother educational approach to .050 .633 .145
correct the problem
I tried to be emotionally honest with my self about the problems .042 .685 .222
I sought advice from others on how I should solve the problems .792 .117 -.038
I explored the emotions caused by the problems .248 .782 -.037
I took direct action to try to correct the problems -.120 -.023 .772
I told someone I could trust about how I felt about the problems .815 .172 -.040
I put aside other activities so that I could work to solve the problems -.014 .155 .657
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
© Dr. Maher Khelifa a. Rotation converged in 5 iterations.
Steps in Factor Analysis:
Making Final Decisions
26
4th Step: Making final decisions
The final decision about the number of factors to choose is the
number of factors for the rotated solution that is most
interpretable.
To identify factors, group variables that have large loadings for the
same factor.
Plots of loadings provide a visual for variable clusters.
Interpret factors according to the meaning of the variables
This decision should be guided by:
A priori conceptual beliefs about the number of factors from past
research or theory
Eigen values computed in step 2.
The relative interpretability of rotated solutions computed in step 3.
© Dr. Maher Khelifa
Assumptions Underlying Factor
Analysis
27
Assumption underlying factor analysis include.
The measured variables are linearly related to the factors +
errors.
This assumption is likely to be violated if items limited response
scales (two-point response scale like True/False, Right/Wrong
items).
The data should have a bi-variate normal distribution for
each pair of variables.
Observations are independent.
The factor analysis model assumes that variables are
determined by common factors and unique factors. All
unique factors are assumed to be uncorrelated with each
other and with the common factors.
© Dr. Maher Khelifa
Obtaining a Factor Analysis
28
Click:
Analyze and
select
Dimension
Reduction
Factor
A factor
Analysis Box
will appear
© Dr. Maher Khelifa
Obtaining a Factor Analysis
29
Move
variables/scal
e items to
Variable box
© Dr. Maher Khelifa
Obtaining a Factor Analysis
30
Factor
extractio
n
When
variable
s are in
variable
box,
select:
Extracti
on
© Dr. Maher Khelifa
Obtaining a Factor Analysis
31
When the
factor
extraction
Box appears,
select:
Scree Plot
keep all
default
selections
including:
Principle component
Analysis
Based on Eigen
Value of 1, and
Un-rotated factor
solution
© Dr. Maher Khelifa
Obtaining a Factor Analysis
32
During
factor
extractio
n keep
factor
rotation
default
of:
None
Press
continue
© Dr. Maher Khelifa
Obtaining a Factor Analysis
33
During Factor
Rotation:
Decide on the
number of
factors based on
actor extraction
phase and enter
the desired
number of
factors by
choosing:
Fixed number of
factors and
entering the
desired number
of factors to
extract.
Under Rotation
Choose Varimax
Press continue
Then OK
© Dr. Maher Khelifa
Bibliographical References
34
Almar, E.C. (2000). Statistical Tricks and traps. Los Angeles, CA: Pyrczak Publishing.
Bluman, A.G. (2008). Elemtary Statistics (6th Ed.). New York, NY: McGraw Hill.
Chatterjee, S., Hadi, A., & Price, B. (2000) Regression analysis by example. New York:
Wiley.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the
behavioral sciences (2nd Ed.). Hillsdale, NJ.: Lawrence Erlbaum.
Darlington, R.B. (1990). Regression and linear models. New York: McGraw-Hill.
Einspruch, E.L. (2005). An introductory Guide to SPSS for Windows (2nd Ed.). Thousand
Oak, CA: Sage Publications.
Fox, J. (1997) Applied regression analysis, linear models, and related methods. Thousand
Oaks, CA: Sage Publications.
Glassnapp, D. R. (1984). Change scores and regression suppressor conditions. Educational
and Psychological Measurement (44), 851-867.
Glassnapp. D. R., & Poggio, J. (1985). Essentials of Statistical Analysis for the Behavioral
Sciences. Columbus, OH: Charles E. Merril Publishing.
Grimm, L.G., & Yarnold, P.R. (2000). Reading and understanding Multivariate statistics.
Washington DC: American Psychological Association.
Hamilton, L.C. (1992) Regression with graphics. Belmont, CA: Wadsworth.
Hochberg, Y., & Tamhane, A.C. (1987). Multiple Comparisons Procedures. New York: John
Wiley.
Jaeger, R. M. Statistics: A spectator sport (2nd Ed.). Newbury Park, London: Sage
Publications.
© Dr. Maher Khelifa
Bibliographical References
35
Keppel, G. (1991). Design and Analysis: A researcher’s handbook (3 rd Ed.). Englwood
Cliffs, NJ: Prentice Hall.
Maracuilo, L.A., & Serlin, R.C. (1988). Statistical methods for the social and behavioral
sciences. New York: Freeman and Company.
Maxwell, S.E., & Delaney, H.D. (2000). Designing experiments and analyzing data:
Amodel comparison perspective. Mahwah, NJ. : Lawrence Erlbaum.
Norusis, J. M. (1993). SPSS for Windows Base System User’s Guide. Release 6.0. Chicago,
IL: SPSS Inc.
Norusis, J. M. (1993). SPSS for Windows Advanced Statistics. Release 6.0. Chicago, IL:
SPSS Inc.
Norusis, J. M. (2006). SPSS Statistics 15.0 Guide to Data Analysis. Upper Saddle River,
NJ.: Prentice Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Guide to Data Analysis. Upper Saddle River,
NJ.: Prentice Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Statistical Procedures Companion. Upper
Saddle River, NJ.: Prentice Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Advanced Statistical Procedures Companion.
Upper Saddle River, NJ.: Prentice Hall.
Pedhazur, E.J. (1997). Multiple regression in behavioral research, third edition. New York:
Harcourt Brace College Publishers.
© Dr. Maher Khelifa
Bibliographical References
36
SPSS Base 7.0 Application Guide (1996). Chicago, IL: SPSS Inc.
SPSS Base 7.5 For Windows User’s Guide (1996). Chicago, IL: SPSS Inc.
SPSS Base 8.0 Application Guide (1998). Chicago, IL: SPSS Inc.
SPSS Base 8.0 Syntax Reference Guide (1998). Chicago, IL: SPSS Inc.
SPSS Base 9.0 User’s Guide (1999). Chicago, IL: SPSS Inc.
SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc.
SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc.
SPSS Interactive graphics (1999). Chicago, IL: SPSS Inc.
SPSS Regression Models 11.0 (2001). Chicago, IL: SPSS Inc.
SPSS Advanced Models 11.5 (2002) Chicago, IL: SPSS Inc.
SPSS Base 11.5 User’s Guide (2002). Chicago, IL: SPSS Inc.
SPSS Base 12.0 User’s Guide (2003). Chicago, IL: SPSS Inc.
SPSS 13.0 Base User’s Guide (2004). Chicago, IL: SPSS Inc.
SPSS Base 14.0 User’s Guide (2005). Chicago, IL: SPSS Inc..
SPSS Base 15.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
SPSS Base 16.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
SPSS Statistics Base 17.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
Tabachnik, B.G., & Fidell, L.S. (2001). Using multivariate statistics (4 th Ed). Boston, MA:
Allyn and Bacon.
© Dr. Maher Khelifa
