Math for Data Science: Chapter 4 Linear Algebra

Yes, it’s true that I spend my days as an AI researcher at my current full-time job, but I’ve found myself behind in terms of hard AI skills. These skills I used to have from my college days, but since then I’ve grown rusty on a lot of the math. Lots of workers out there are looking to improve their AI skills in this AI-driven age, and you count me among them. This post is all about the Linear Algebra chapter from Math for Data Science. Essentially, these are my notes. I’ve tracked through the book using Julia rather than Python, and will provide code samples as I go.

Linear Transformations

Take the basis vectors i_hat and j_hat, transform them to form a basis matrix, and then multiply that basis matrix with an input vector to get a result.

function example410()
    # Stretch and shear.
    î = [2,3]
    ĵ = [2,-1]

    # Julia has matrices if you just leave a space between the rows.
    basis = [î ĵ]
    v = [2, 1]
    basis * v
end

The regular * operator when given vectors and matrices peforms a dot product. Useful in linear algebra

Matrix Multiplication

Each matrix is like a function. Multiplying matrices together is like multiplying many functions together to get a new function.

function multiply_mats()
    # Rotation Matrix
    # Each sub-array is a vector, so it's a column in the final matrix
    rot = [[1, 0] [1, 1]]
    # Shear matrix
    shear = [[0, 1] [-1, 0]]

    # multiply
    # [[ae + bg, ce + dg], [af + bh, cf + dh]]
    rot * shear
end

The result of this calculation

2×2 Matrix{Int64}:
 1  -1
 1   0

Now to apply it to a vector

chapter4.multiply_mats() * [1,2] # Yields [-1, 1]

NOTE: Matrix dot products are not commutative. Order matters!

Determinants

The degree to which we expand or squish space. Example is transforming i hat by 3 and j hat by 2, this expands by a factor of 6 from the unit vectors. This can be done with det in julia

using LinearAlgebra
det([[3, 0] [0, 2]]) # ihat 3, jhat 2, returns 6.0

This is 6 because we have 6 units covered by the matrix. The area of the square covered by the vectors is 6. Shears do not affect the area, will remain the same. Scaling will increase it, and rotation make it negative.

If a determinant is 0, that means the transformation is linearly dependent. Meaning all space is squished into a smaller dimension.

Linear dependence means that the problem is difficult or unsolvable given the current conditions.

Types of Matrices

Square matrix has equal columns and rows, defines a linear tranformation.

[[1,2,3] # 3 vectors, 3 rows
 [4,5,6]
 [7,9,0]]

Important for operations like eigendecomposition.

Identity matrix is square with 1s across the diagonal, 0s otherwise.

[[1,0,0] [0,1,0] [0,0,1]]

Means that we’ve undone a linear transformation, have starting vectors.

Inverse matrix, often annotated with a superscript of -1, will “undo” another matrix of linear transformations. Will get you back to the identity matrix.

Diagonal matrix is like the identity matrix, but the values on the diagonal are all nonzero. Represent applying scalars in a linear space.

[[1, 0, 0] [0, 3, 0] [0, 0, 9]]

Triangular matrix, the diagonal and everything above it are nonzero, while the rest are 0

[[4, 0, 0], [2, 1, 0] [9, 6, 5]]

Easier to solve when presented as a system of equations.

Working through Example 4-17

function example417()
    # 4x + 2y + 4z = 44
    # 5x + 3y + 7z = 56
    # 9x + 3y + 6z = 72

    # Represents the above system of equations
    # Can represent a matrix with spaces, and newlines for each row
    # https://docs.julialang.org/en/v1/manual/arrays/
    matrix = [4 2 4
              5 3 7
              9 3 6]

    # Get the identity matrix via dot product with inverse
    matrix * matrix ^ -1 # Note that floating point precision will bother us here.
end

Julia can take the inverse of a matrix with the -1 exponent. Floating point precision appears to bother us with the answer though, with two vanishingly small, close to 0 values present in the identity matrix.

example417 (generic function with 1 method)

3×3 Matrix{Float64}:
 1.0  0.0  2.22045e-16
 0.0  1.0  4.44089e-16
 0.0  0.0  1.0

Example 4-18

Takes the same matrix / system of equations as 4-17, and solves the system of equations.

function example418()
    matrix = [4 2 4
              5 3 7
              9 3 6]

    B = [44, 56, 72]

    inv(matrix) * B # Yields [[2, 34, -8]], but not with floating point weirdness.
end

To solve a system of equations, can do dot product of the RHS vector with the inverse of the LHS matrix. This is because the LHS inverse will decompose the system into the identity vectors, which is the solution. Each 1 in the identity matrix represents one of the variables in the system of equations. Also this is why it’s important that a system of equations be a square matrix.

Conclusion

Today I learned about

  1. Linear Transformations.
  2. Mulitplying Matrices
  3. Determinants - the degree to which we have “transformed” the space, the degree of change.
  4. Types of matrices - triangle is my favorite
  5. Solving systems of linear equations using the inverse Matrix for a linear transformation.

The final part of this chapter covers Eigenvectors and Eigenvalues. I will dive into this tomorrow morning. Upskilling can be fun! One day I may even attempt a graduate degree in mathematics.

 Share!

 
I run WindleWare! Feel free to reach out!

Subscribe for Exclusive Updates

* indicates required