Lecture 13

Principal Components Analysis and Factor

Analysis

Prof. Dr. Svetlozar Rachev

Institute for Statistics and Mathematical Economics
University of Karlsruhe

Financial Econometrics, Summer Semester 2007

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Copyright

These lecture-notes cannot be copied and/or distributed

without permission.
The material is based on the text-book:
Financial Econometrics: From Basics to Advanced
Modeling Techniques
(Wiley-Finance, Frank J. Fabozzi Series)
by Svetlozar T. Rachev, Stefan Mittnik, Frank Fabozzi, Sergio
M. Focardi,Teo Jaši c̀.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Outline

I Factor models.
I Principal components analysis.
I Factor analysis.
I PCA and factor analysis compared.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I Factor models are statistical models that try to explain

complex phenomena through a small number of basic causes
or factors.
I Factor models serve two main purposes:
1. They reduce the dimensionality of models to make
estimation possible;
2. They find the true causes that drive data.
I Factor models were introduced by Charles Spearman in 1904.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I The Spearman model explains intellectual abilities through one

common factor, the famous ”general intelligence” g factor,
plus another factor s which is specific to each distinct ability.
I Louis Leon Thurstone developed the first true multifactor
model of intelligence, where were identified the following
seven primary mental abilities:
Verbal Comprehension Word Fluency
Number Facility Spatial Visualization
Associative Memory Perceptual Speed
Reasoning.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I In the early applications of factor models to psychometrics,

the statistical model was essentially a conditional multivariate
distribution. The objective was to explain psychometric tests
as probability distributions conditional on the value of one or
more factors. In this way, one can make predictions of, for
example, the future success of young individuals in different
activities.
I In economics, factor models are typically applied to time
series. The objective is to explain the behavior of a large
number of stochastic processes, typically price, returns, or rate
processes, in terms of a small number of factors. These
factors are themselves stochastic processes.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I In order to simplify both modeling and estimation, most factor

models employed in financial econometrics are static models.
This means that time series are assumed to be sequences of
temporally independent and identically distributed (IID)
random variables so that the series can be thought as
independent samples extracted from one common distribution.
I In financial econometrics, factor models are needed not only
to explain data but to make estimation feasible. Factor
models able to explain all pairwise correlations in terms of a
much smaller number of correlations between factors.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Linear Factor Models Equations

Linear factor models are regression models of the following type:

K
X
Xi = αi + βij fj + εi
j=1

where
Xi = a set of N random variables
fj = a set of K common factors
εi = the noise terms associated with each variable Xi
βij ’s are the factor loadings or factor sensitivities, which express
the influence of the j-th factor on the i-th variable.
Note: In this formulation, factor models are essentially static
models, but it is possible to add a dynamics to both the variables
and the factors.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I One of the key objectives of factor models is that the

covariances between the variables Xi is determined only by the
covariances between factors.
I Suppose that the noise terms are mutually uncorrelated, so
that 
0 i 6= j
E (εi εj ) =
σi2 i = j
and that the noise terms are uncorrelated with the factors,
that is, E (εi fj ) = 0, ∀i, j.
I Suppose also that both factors and noise terms have a zero
mean, so that E (Xi ) = αi .
Factor models that respect the above constraints are called strict
factor models.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

Lets compute the covariances of a strict factor model:

K
 X K
 X 
E ((Xi − αi )(Xj − αj )) = E βis fs + εi βjt ft + εj
s=1 t=1

K
 X K
 X  K
 X  
=E βis fs βjt ft +E βis fs (εj )
s=1 t=1 s=1

 K
X 
+E (εi ) βjt ft + E (εi εj )
t=1
X
= βis E (fs ft )βjt + E (εi εj )
s,t

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

We can express the above compactly in matrix form. Lets write a

factor model in matrix form as follows:

X = α + βf + ε

where
X = (X1 , . . . , XN )0 = the N-vector of variables
α = (α1 , . . . , αN )0 = the N-vector of means
ε = (ε1 , . . . , εN )0 = the N-vector of idiosyncratic noise terms
f = (f1 , . . . , fK )0 = the K -vector of factors
 
β11 · · · β1K
β =  ... .. ..  = the N × K matrix of factor loadings.

. . 
βN1 · · · βNK

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models
I Lets define the following:
Σ = the N × N variance-covariance matrix of the variables X
Ω = the K × K variance-covariance matrix of the factors
Ψ = N × N variance-covariance matrix of the error terms ε.
I If we assume that our model is a strict factor model, the
matrix Ψ will be a diagonal matrix with the noise variances on
the diagonal, that is,
 2 
Ψ1 · · · 0
Ψ =  ... .. .. 

. . 
0 ··· Ψ2N
I We can express the variance-covariance matrix of the variables
in the following way:

Σ = βΩβ 0 + Ψ
Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical
Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I In applied work, factor models will often be approximate

factor models. They allow idiosyncratic terms to be weakly
correlated among themselves and with the factors.
I As many different factor models have been proposed for
explaining stock returns, an important question is whether a
factor model is fully determined by the observed time series.
I An estimation procedure cannot univocally determine the
hidden factors and the factor loadings from the observable
variables X.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

In fact, suppose that we multiply the factors by any nonsingular

matrix R. We obtain other factors

g = Rf

with a covariance matrix

Ωg = RΩR−1

and we can write a new factor model:

X = α + βf + ε = α + βR−1 Rf + ε = α + βg g + ε

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

I In order to solve this indeterminacy, we can always choose the

matrix R so that the factors g are a set of orthonormal
variables, that is, uncorrelated variables (the orthogonality
condition) with unit variance (the normality condition).
I In order to make the model uniquely identifiable, we can
stipulate that factors must be a set of orthonormal variables
and that, in addition, the matrix of factor loadings is diagonal.
I Under this additional assumption, a strict factor model is
called a normal factor model. The model is still undetermined
under rotation, that is multiplication by any nonsingular
matrix such that RR0 = I .

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models

Summary:
I A set of variables has a normal factor representation if it is
represented by the following factor model:

X = α + βf + ε

where factors are orthonormal variables and noise terms are

such that the covariance matrix can be represented as follows:

Σ = ββ 0 + Ψ

where β is the diagonal matrix of factor loadings and Ψ is a

diagonal matrix.
I Approximate factor models are uniquely identifiable only in
the limit of an infinite number of series.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models: Types of Factors and Their Estimation

In financial econometrics, the factors used in factor models can

belong to three different categories:
I Macroeconomic factors
I Fundamental factors
I Statistical factors
Macroeconomic factors are macroeconomic variables that are
believed to determine asset returns (Example: GNP, the inflation
rate, the unemployment rate, or the steepness of the yield curve).
Fundamental factors are variables that derive from financial
analysis.
Statistical factors are factors that derive from a mathematical
process.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models: Types of Factors and Their Estimation

I Macroeconomic factors are exogenous factors that must be

estimated as variables exogenous to the factor model. They
influence the model variables but are not influenced by them.
I A factor model is estimated as a linear regression model,
means that there is indeed a linear relationship between the
factors and the model variables.
I However, such a model will have no explanatory power. The
variance of each variable that is not explained by common
factors appears as noise.
I Adding factors might improve the explanatory power of the
model but, in general, worsens the ability to estimate the
model because there are more parameters to estimate.
There is a trade-off between adding explanatory factors and
the ability to estimate them.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Models: Types of Factors and Their Estimation

I Statistical factors are obtained through a logical process of

analysis of the given variables.
I Statistical factors are factors that are endogenous to the
system. They are typically determined with one of two
statistical processes; namely, principal component analysis or
factor analysis.
I Note that factors defined through statistical analysis are linear
combinations of the variables.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I Principal components analysis (PCA) was introduced in 1933

by Harold Hotelling as a way to determine factors with
statistical learning techniques when factors are not
exogenously given.
I Given a variance-covariance matrix, one can determine factors
using the technique of PCA.
I The concept of PCA is the following.
Consider a set of n stationary time series Xi .
Consider next a linear combination of these series, that is, a
portfolio of securities. Each portfolio P is identified by an
n-vector of weights ωP and is characterized by a variance σP2 .
Lastly, consider a normalized portfolio, which has the largest
possible variance. In this context, a normalized portfolio is a
portfolio such that the squares of the weights sum to one.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis
I If we assume that returns are IID sequences, jointly normally
distributed with variance-covariance matrix σ, a lengthy direct
calculation demonstrates that each portfolios return will be
normally distributed with variance

σP2 = ωPT σωP

I The normalized portfolio of maximum variance can therefore

be determined in the following way: Maximize ωpT σωP
subject to the normalization condition

ωPT ωP = 1

where the product is a scalar product.

I It can be demonstrated that the solution of this problem is the
eigenvector ω2 corresponding to the largest eigenvalue λ2 of
the variance-covariance matrix σ.
Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical
Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I Consider next the set of all normalized portfolios orthogonal

to ω1 , that is, portfolios completely uncorrelated with ω1 .
These portfolios are identified by the following relationship:

ω1T ωP = ωPT ω1 = 0

Among this set, the portfolio of maximum variance is given by

the eigenvector ω2 corresponding to the second largest
eigenvalue λ2 of the variance-covariance matrix σ.
I If there are n distinct eigenvalues, we can repeat this process
n times. In this way, we determine the n portfolios Pi of
maximum variance. The weights of these portfolios are the
orthonormal eigenvectors of the variance- covariance matrix σ.
I These portfolios of maximum variance are all mutually
uncorrelated.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I It can be demonstrated that we can recover all the original

return time series as linear combinations of these portfolios:
n
X
Xj = αj,i Pi
i=1

I Thus far we have succeeded in replacing the original n

correlated time series Xj with n uncorrelated time series Pi
with the additional insight that each Xj is a linear
combination of the Pi .
I Suppose now that only p of the portfolios Pi have a significant
variance, while the remaining n-p have very small variances.
We can then implement a dimensionality reduction by
choosing only those portfolios whose variance is significantly
different from zero. Lets call these portfolios factors F.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I It is clear that we can approximately represent each series Xi

as a linear combination of the factors plus a small
uncorrelated noise. In fact we can write
p
X n
X p
X
Xj = αj,i Fi + αj,i Pi = αi,j Fi + εj
i=1 i=p+1 i=1

where the last term is a noise term.

I Therefore to implement PCA one computes the eigenvalues
and the eigenvectors of the variance-covariance matrix and
chooses the eigenvalues significantly different from zero.
I The corresponding eigenvectors are the weights of portfolios
that form the factors. Criteria of choice are somewhat
arbitrary.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I Suppose, however, that there is a strict factor structure, which

means that returns follow a strict factor model as defined
earlier in this chapter:

r = a + βf + ε

I The matrix β can be obtained diagonalizing the

variance-covariance matrix.
I In general, the structure of factors will not be strict and one
will try to find an approximation by choosing only the largest
eigenvalues.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis

I PCA works either on the variance-covariance matrix or on the

correlation matrix. The technique is the same but results are
generally different.
I PCA applied to the variance-covariance matrix is sensitive to
the units of measurement, which determine variances and
covariances. If PCA is applied to prices and not to returns,
the currency in which prices are expressed matters; one
obtains different results in different currencies. In these cases,
it might be preferable to work with the correlation matrix.
I PCA is a generalized dimensionality reduction technique
applicable to any set of multidimensional observations. It
admits a simple geometrical interpretation which can be easily
visualized in the three-dimensional case.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

I Performing PCA is equivalent to determining the eigenvalues

and eigenvectors of the covariance matrix or of the correlation
matrix. The two matrices yield different results. We perform
both exercises.
I We estimate the covariance with the empirical covariance
matrix. Recall that the empirical covariance σij between
variables (Xi , Xj ) is defined as follows:

T
1 X
σ̂ij = (Xi (t) − X̄i )(Xj (t) − X̄j )
T
t=1

T T
1 X 1 X
X̄i = Xi (t), X̄j = Xj (t)
T T
t=1 t=1

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

For the whole exhibit look at page 441.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

For the whole exhibit look at page 442.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

For the whole exhibit look at page 443.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

I If we form portfolios whose weights are the eigenvectors, we

can form 10 portfolios that are orthogonal (i.e., uncorrelated).
These orthogonal portfolios are called principal components.
I The variance of each principal component will be equal to the
corresponding eigenvector. Thus the first principal component
will have the maximum possible variance and the last principal
component will have the smallest variance.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

I The 10 principal components thus obtained are linear

combinations of the original series, X = (X1 , . . . , XN )0 that is,
they are obtained by multiplying X by the matrix of the
eigenvectors.
I If the eigenvalues and the corresponding eigenvectors are all
distinct we can apply the inverse transformation and recover
the X as linear combinations of the principal components.
I PCA is interesting if, in using only a small number of principal
components, we nevertheless obtain a good approximation.
So we regress the original series X onto a small number of
principal components. By choosing as factors the components
with the largest variance, we can explain a large portion of the
total variance of X.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

For the whole exhibit look at page 448.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Principal Components Analysis: Illustration

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
PCA and Factor Analysis with Stable Distributions

I PCA and factor analysis can be applied provided that all

variances and covariances exist. Applying them does not
require that distributions are normal, but only that they have
finite variances and covariances.
I Variances and covariances are not robust but are sensitive to
outliers. Robust equivalent of variances and covariances exist.
I In order to make PCA and factor analysis insensitive to
outliers, one could use robust versions of variances and
covariances and apply PCA and factor analysis to these robust
estimates.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
PCA and Factor Analysis with Stable Distributions

I In many cases, however, distributions might exhibit fat tails

and infinite variances. In this case, large values cannot be
trimmed but must be taken into proper consideration.
However, if variances and covariances are not finite, the least
squares methods used to estimate factor loadings cannot be
applied. In addition, the concept of PCA and factor analysis
cannot be applied.
I In fact, if distributions have infinite variances, it does not
make sense to determine the portfolio of maximum variance
as all portfolios will have infinite variance and it will be
impossible, in general, to determine an ordering based on the
size of variance.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis

I Here we consider an alternative technique for determining

factors: factor analysis (FA).
I Suppose we are given T independent samples of a random
vector X = (X1 , . . . , XN )0 , N time series of stock returns at T
moments, as in the case of PCA.
I Assuming that the data are described by a strict factor model
with K factors, the objective of factor analysis (FA) consists
of determining a model of the type

X = α + βf + ε

with covariance matrix

Σ = ββ 0 + Ψ

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis

I The estimation procedure is performed in two steps:

1. We estimate the covariance matrix and the factor
loadings.
2. We estimate factors using the covariance matrix and the
factor loadings.
I Assuming that the variables are jointly normally distributed
and temporally independently and identically distributed (IID),
we can estimate the covariance matrix with maximum
likelihood methods. Iterative methods such as the expectation
maximization (EM) algorithm are generally used for the
estimation of factor models.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis

I After estimating the matrices β and Ψ factors can be

estimated as linear regressions.
I In fact, assuming that factors are zero means, we can write
the factor model as

X − α = βf + ε

which shows that, at any given time, factors can be estimated

as the regression coefficients of the regression of (X α) onto β.
I Using the standard formulas of regression analysis we can now
write factors, at any given time, as follows:

fˆt = (β̂ 0 Ψ̂−1 β̂)−1 β̂ 0 Ψ̂−1 (Xt − α̂)

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis

I Maximum likelihood estimates implies that the number of

factors is known. The correct number of factors is determined
basing on the heuristic procedure when estimates of q factors
stabilize and cannot be rejected on the basis of p probabilities.
I The factor loadings matrix can also be estimated with
ordinary least squares (OLS) methods.
I OLS estimates of the factor loadings are inconsistent when
the number of time points goes to infinity but the number of
series remains finite, unless we assume that the idiosyncratic
noise terms all have the same variance.
I The OLS estimators remain consistent under the assumption
when both the number of processes and the time go to infinity.
So, to perform factor analysis, we need to estimate only the factor
loadings and the idiosyncratic variances of noise terms.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Applying PCA to Bond Portfolio Management

There are two applications in bond portfolio management where

PCA has been employed.
I The first application is explaining the movement or dynamics
in the yield curve and then applying the resulting principal
components to measure and manage yield curve risk.
I The second application of PCA is to identify risk factors
beyond changes in the term structure.
For example, given historical bond returns and factors that are
believed to affect bond returns, PCA can be used to obtain
principal components that are linear combinations of the variables
that explain the variation in returns.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Using PCA to Control Interest Rate Risk

I Using PCA, several studies have investigated the factors that

have affected the historical returns on Treasury portfolios.
I Robert Litterman and Jose Scheinkman found that three
factors explained historical bond returns for U.S. Treasuries
zero-coupon securities:
1. The changes in the level of rates;
2. The changes in the slope of the yield curve;
3. The changes in the curvature of the yield curve.
I After identifying the factors, Litterman and Scheinkman use
regression analysis to assess the relative contribution of these
three factors in explaining the returns on zero-coupon
Treasury securities of different maturities.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Using PCA to Control Interest Rate Risk

I On average, the first principal component explained about

90% of the returns, the second principal component 8%, and
the third principal component 2%.
I Thus, only three principal components were needed to fully
explain the dynamics of the yield curve.
I There have been several studies that have examined the yield
curve movement using PCA and reported similar results.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Using PCA to Control Interest Rate Risk

Once yield curve risk is described in terms of principal components,

the factor loadings can be used to:
I Construct hedges that neutralize exposure to changes in the
direction of interest rates.
I Construct hedges that neutralize exposure to changes in
nonparallel shifts in the yield curve.
I Structure yield curve trades.
PCA of the dynamics of the yield curve have lead to the use of
what is now referred to as principal component duration.
Moreover, PCA can be used to estimate the probability associated
with a given hypothetical interest rate shock so that a bond
portfolio manager can better analyze the interest rate risk of a
bond portfolio and traders can better understand the risk exposure
of a bond trading strategy.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis: Bond Risk Factors

I For a bond index that includes nongovernment securities,

there are risk factors other than term structure factors.
I Using PCA, Gauthier and Goodman have empirically identified
the risk factors that generate nominal excess returns for the
Salomon Smith Barney Broad Investment Grade Index (SSB
BIG Index) for the period January 1992 to March 2003.
I The results of their PCA for the first six principal components
for each bond sector of the SSB BIG Index are presented on
the next slide.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis: Bond Risk Factors

I The first principal component explains 92.7% of the variation.

I The second principal component explains 3.1% of nominal
excess returns. Gauthier and Goodman identify this factor as
the credit specific factor because of the high negative factor
loadings on the credit index combined with a high positive
weighting on Treasuries.
I Gauthier and Goodman identify the third principal component
as an optionality factor. This can be supported by noting that
the factor loadings on the assets classes that have some
optionality is positive, while the factor loading on the
noncallable series is negative.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Factor Analysis: Bond Risk Factors

I This third principal component, which represents optionality,

is consistent with studies of the movements of the yield curve.
I Gauthier and Goodman show that there is a high positive
correlation between the optionality factor and the slope of the
yield, but a negative relationship with 5-year cap volatility.
This suggests
1. the steeper the yield curve slope, the better a callable series
should do;
2. the higher the volatility, the lower the return on the callable
series.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
PCA and Factor Analysis Compared

I The two illustrations of PCA and FA are relative to the same

data and will help clarify the differences between the two
methods.
I Lets first observe that PCA does not imply any specific
restriction on the process. Given a nonsingular covariance
matrix, we can always perform PCA as an exact linear
transformation of the series. When we consider a smaller
number of principal components, we perform an
approximation which has to be empirically justified.
I Factor analysis, on the other hand, assumes that the data
have a strict factor structure in the sense that the covariance
matrix of the data can be represented as a function of the
covariances between factors plus idiosyncratic variances.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
PCA and Factor Analysis Compared

I PCA tends to be a dimensionality reduction technique that

can be applied to any multivariate distribution and that yields
incremental results. This means that there is a trade-off
between the gain in estimation from dimensionality reduction
and the percentage of variance explained.
I Factor analysis, on the other hand, tends to reveal the exact
factor structure of the data. That is, FA tends to give an
explanation in terms of what factors explain what processes.
I Factor rotation can be useful both in the case of PCA and FA.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
An Illustration of Factor Analysis

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis
Final Remarks

I The required textbook is ”Financial Econometrics: From

Basics to Advanced Modeling Techniques”.
I Please read Chapter 13 for today’s lecture.
I All concepts explained are listed in page 464 of the textbook.

Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical

Lecture
Economics
13 Principal
University
Components
of Karlsruhe
Analysis and Factor Analysis