Lecture Plan
• Matrix in Data science, Linear Maps, Matrix,
Matrix-Vector and Matrix-Matrix multiplications
• Types of matrix, Determinant and inverse of a
matrix, Norm of a matrix
• Tensor, addition and subtraction of tensors
Kronecker product, Hardamard product
Intended Learning Outcomes
• Understanding of Matrices
• Representing data using matrices
• Matrix Operations
• Representing high-dimensional data
Major challenge in Data science
• How to represent and perform operation on
(high-)dimensional data?
• How to extract key features/ information from
high-dimensional data "ef ciently"?
use matrix algebra
Éf
e ca G
Functions and compositeI functions
Let Lex be a function with one
independent variable
2x KEIR
Eg Lex 5
Let had g of x where
Ge 506
g
x
fix hea g Lees
35
1 7
2 9 45
3 11 55
H 13 65
Are f L linear maps
Ice D Jay LCxty Yes fly
linear
Let
f Go Dc 3 Is
f
Try L Canty I fastfly
WE Prove that
I
2x y Q LGD
fly LER
Actually
Jlaxty 2 2 0 3
I fed fly
Linear function in n-dimensional
Let see Rn then
if Cox e
fay
co
g o
Gets
f L GO
LG
l d fees moe MEIR
Scaling
2 d Let 2 13
I Define L as
x fees f is 3
M
Eg
het se
2 y 2 L 3 Y
Xx fi's
Ex
It Can also be derived as
I 2 L 2 819
1819
2
f b
2 2 it
4
It is enough to know
if
to find
I 9
f 9 a be R
Is there any Compact
form for
this mapping
can we write
1121 3 9 12
Is stretching a linear transformation
How about rotation 2
Matrix
Matrix is a linear transformation of (basis) vector.
É
Rotation
In general 3 313 a
Shear:
Dy
How about
De nition
Let m n EN A real valued
matrix A is an mn tuple of elements
ai ER I s it m is Jen which
is ordered as m rows d n columns
row
a a 9Heist
Am n nth
amn row
p p
nth Col
gsteal
men Square matrix
Matrix in Data science
Network
Adjacency matrix
2 3 4 5 6 7
I T I 0 0 0 1 T
2 I 1 O O O O O
3 0 0 1 0 1 I 0
A s y
5
I
i o o n o f
6 1 0 1 0 1 0
Most of the entries are zero
A is a sparse mat
can we store A efficiently
Sparse matrix storage
Ex Compressed
Sparse Row CSR
Store only non zero entries
need two
arrays of
size N where n is
non zeros
not of
Data Matrix
Suppose we collect n number
of
features of a person we have n persons
istfeature
921
A
Amn
mthfeature
am Ama
i n person
1stperson nth
Matrix Addition
Amant Bmxn
Let A B E RMN
tb aint bin
At B
icinntion
an tbn
Can we add or subrart
matrices AERMM BEA
where men
C ATB
4th F
Matrix-vector multiplication
D
X
I
Ry 3 An 922 923
Rz 7 931 931 9333 3
3 zx
111 E D'D
that that multi
to L
1
11 11
M M
Shear rotation
f
Ka
and
I T Ts
Ac Ae
where
a a
Gi D Ea
a an
scaleD
Note that
EYE
WA Y Ca
use Column
form find C
AM
Ie II
11 I
I at al's 3
D
Identity matrix Cat He B
j t
The square matrix I is said to be an identity matrix
of
Suppose A AT
A is symmetric
Multiplication of a matrix by a scalar.
Then where
Transpose of a matrix
ME
- writes Rows of A into Columns of A
Determinant of a matrix:
Al
ÉREY
3 2
Quanti es the change in the measure (area in 2d,
volume in 3d) due to linear transformation.
How about
How about
Same as above but the orientation got ipped
A EI
I Al y
LI
s
5 2 51
Suppose
FIRE It
Suppose 1 A1 0
Singual matrix
Inverse of a matrix
Let A be a square matrix of order n. Suppose a
matrix B is said to be an inverse of A then
Further, B is denoted as A is unique
Suppose A is a ai clockwise
rotation then A is a go
Counter clockwise rotation
IL A is a leftward shear
then it is a highward shear
Suppose we have
TT
known
Unknown
I known
We know the linear transformation (A), then question
is where should we start (x) to land at an expected
position (b)?
inverse transformation
Recall Norms of a vector:
Let X E IR ie x x sea
port
then I norm
of x is defined as
110011 is
71 boil
where 1 I is the absolute value
Hy la norm of se is defined as
1101122
Eh boil
Octoc
11041
Fix
How about the norms for matrix?
Column sum norm
Spectral norm
Where Imax is the largest eigenvalue
L_p norm
Spectral radius
Where a A denote the eigenvalues
A
of
Tensor
• How to represent three or more dimensional data?
xd
j Sy
yes
• How to represent in computer?
A JE JE
Tensor Addition / subtraction
Addition subsactin two tensors Lame
of of
size add subtract element wise
xd ax x dm
Let A B E Rd
die wthorder tensor then i ninth
element C At B will be
of
Ci ia in Ai ia in t bi ia in
NY
for D A B
di is in Ai iz in bi iz in
Tensor Product: Kronecker Product
Let A B E Rd edna
tensors then the right Kronecker
d x xant
product C AGB E IR
Example
A
I BE
I B 2B
C
B u B
I
D
E's an
Tensor Product: Hardamard Product
Let A B E [Link]
die withorder tensor then i ninth
element C A OB will be
of
Ci ia in Ai ia in bi ia in
Example
Remark:
Hadamard & Kronecker products are used in back
propagation implementation in Neural network
Recap Poll:
Q1: A real m-by-n matrix is a linear transformation from
R^m to R^n. (True/False)
Q2: Identity matrix will always be a square matrix.
(True/False)
Q3: A data matrix representing n-unique features with
m(>n) data points will always be a full-rank matrix. (True/
False)
Q4: Suppose A and B are m-by-n matrices. The Hadamard
products AoB and BoA are equal. (True/False)
Q5: Let u be a n-by-1 vector, then uu^T will be an n-by-n
matrix with ‘n’ linearly independent rows. (True/False)
