Scribd
Upload
159 views3 pages

2 Dierentiation With Respect To A Vector: Form Derivative WRT

This document discusses differentiation rules for matrices and vectors. Some key points: - It provides differentiation rules for scalar functions of vectors, including the derivative of kg(x)k^2 with respect to x being 2rg(x)g(x). - Derivatives of scalar functions of matrices are provided, such as the derivative of kAk^2 with respect to A being 2A. - Differentiating Ax with respect to the matrix A is explained using summation notation, resulting in a tensor containing the derivatives of each element of x with respect to each element of A. - As an example, the derivative of b^T Ax with respect to A is derived to be bx^T

Uploaded by

amera_na
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
159 views3 pages

2 Dierentiation With Respect To A Vector: Form Derivative WRT

This document discusses differentiation rules for matrices and vectors. Some key points: - It provides differentiation rules for scalar functions of vectors, including the derivative of kg(x)k^2 with respect to x being 2rg(x)g(x). - Derivatives of scalar functions of matrices are provided, such as the derivative of kAk^2 with respect to A being 2A. - Differentiating Ax with respect to the matrix A is explained using summation notation, resulting in a tensor containing the derivatives of each element of x with respect to each element of A. - As an example, the derivative of b^T Ax with respect to A is derived to be bx^T

Uploaded by

amera_na
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Matrix Di erentiation

Andrew Fitzgibbon, Naoufel Werghi


August 28, 1997

1 Introduction
This is a collection of di erentiation rules for matrices and vectors. Uppercase letters are matrices,
lowercase are vectors.

2 Dierentiation with respect to a vector


The rst table is for derivatives with respect to a column vector x. Note that x xT is then a 3 3
matrix.
Form Derivative wrt x.
kxk2 2x
kg(x)k2 2rxg(x)g(x)
kx + ak2 2(x + a)
kAxk2 2AT Ax
kAx ; bk2 2AT (Ax ; b)
Ax AT
x Ax (AT + A)x
T
a  x a0 1
;a3 a2 0
a x @ a3 0 ;a1 A
;a2 a1 0
ka xk2 2(kak2I ; aaT )x
3 Scalar function of vector
Let's consider g(x) a scalar function of x and v(x) vectorial function of x
then the derivative of g(x)v(x) is v(g )T + gv
0 0

(') means the derivative with respect to x

4 Scalar functions of matrices


This next table is derivatives of some common scalar functions of a matrix A. The results are
matrices the same size as A, with the derivative dened as:
0 @f @f  @f 1
BB @a11 @a12 @a1n CC
df BB @f
@a21
@f
@a22  @f
@a2n CC
dA = B@ ..
.
..
.
... ..
.
CA
@f @f  @f
@an1 @an2 @ann
Form Derivative wrt A.
bT Ax bxT
kAxk2 2AxxT
kAx + bk2 2(Ax + b)xT
2A(xxT ) + 2bxT
kA ; AT k22 4(A ; AT )
kAk22 2A
det(A) A = det(A)A 1 = adj (A)
y ;

tr(A) I
xT AT MAx 2MAxxT
kAAT ; I k22 4(AAT ; I )A
Ax Some horrible tensor . . .
5 Those horrible tensors in full
OK, we want to di erentiate Ax. What do we mean? Well each of the 3 components of x has a
matrix of derivatives:
@xi
@apq
And that's our answer. The question is how to do anything with it. Well, rst let's write out Ax
in longhand: X
Ax]i = aij xj
j
And using the summation convention1 we can drop the sigma. Now we have a table translating
common matrix operations to the summation form.
Operation Summation Form
a = x  y ai = xk yk
y = Ax yi = Aik xk
C = AB Cij = Aik Bkj
tr(A) Akk
AT AT ]ij = Aji
Note that we often need to bring in a new index to \do" the summation for us. Also, if we want
to have say uT v + aT b, we must use a di erent index for each half: up vp + aq bq for example.
@ bT Ax, which is given above as bxT . First we write
Now let's look at a specic problem: Find @A
the expression in the summation convention:
bT Ax = b  (Ax)
= b  Aij xj ]3i=1
= bi Aij xj

Now we've got two summation signs on the outside, we can just bring in the derivative operator
and apply normal rules to
@ @Aij
@A (bi Aij xj ) = bi @A xj
pq pq
@A is zero unless i = p and j = q . This is written using the Kronecker delta
Finally we note that @A ij

as:
pq

@Aij
@A = ip jqpq
1 Any repeated subscript is summed over.
Which goes back into the original problem to give
 @bT Ax  @
@A pq = @Apq (bi Aij xj )
= bi ip jq xj
= bp xq
= bxT ]pq

You might also like