1.

4
Our First Machine Learning Classi er
Part 3: The Training Process

Sebastian Raschka and the Lightning AI Team

1
fi
 Perceptron Summary for Making Predictions

Inputs Computation Outputs

b
x1
w1
Weighted
w2
Threshold
sum

x2

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

2
 How does the perceptron learn
the model parameters?

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

3
 Start with a Training Dataset

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

4
 x1 x2 y
0.77 1.14 0

-0.33 1.44 0

0.91 -3.07 0

-0.37 -1.91 0

-0.63 -1.53 0

0.39 -1.99 0

… … …

1.33 2.03 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

5
 y
Training
x1 x2 example
[1]
x
0.77 1.14 0

-0.33 1.44 0

0.91 -3.07 0

-0.37 -1.91 0

-0.63 -1.53 0

0.39 -1.99 0

… … …

1.33 2.03 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

6
 [1 2 ]
[1] [1] [1]
x = x , x
y
Training
x1 x2 example
[1]
x
0.77 1.14 0

-0.33 1.44 0

0.91 -3.07 0

-0.37 -1.91 0

-0.63 -1.53 0

0.39 -1.99 0

… … …

1.33 2.03 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

7
 x1 x2 y Training
example
0.77 1.14 0 [2]
x
-0.33 1.44 0

0.91 -3.07 0

-0.37 -1.91 0

-0.63 -1.53 0

0.39 -1.99 0

… … …

1.33 2.03 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

8
 Training
example
x1 x2 y x [20]

0.77 1.14 0

-0.33 1.44 0

0.91 -3.07 0

-0.37 -1.91 0

-0.63 -1.53 0

0.39 -1.99 0

… … …

1.33 2.03 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

9
 [1] [1] [2] [2] [n] [n] m
D = (hx , y i, hx , y i, ..., hx , y
<latexit sha1_base64="Wal6cJLWU5bzcyF5u646zqqA+4A=">AAADVXiclZLfbtMwFMZP0jFGB6ywS24sKiSQpiqpkOAGaYJd7HIguk1qs8lxndaa7UT2CaKK+lo8yG4m3oQbJE7dSvvTIYGjyJ+/851fEsd5pZXHJPkZxa2NB5sPtx61tx8/ebrTefb82Je1E3IgSl2605x7qZWVA1So5WnlJDe5lif5xadF/eSbdF6V9ivOKpkZPrGqUIIjWWXnB4zAAAeEKQiaNTRwAHNg8IHu11TV5FqY0CzJuU7nUFD2O2XPaB5CChnpPcrMbjkjcDcIe4Hxr8z+GrP/F2YvXP9Ht2t0ew/9TWAoWi/345qWU8+XQDMhg5Qy1OHDqoGE2CmpOTHOwLLt80436SVhsHWRrkQXVuPovHM5GpeiNtKi0Nz7YZpUmDXcoRJaztuj2suKiws+kUOSlhvpsyacijl7Rc6YFaWj2yIL7s2OhhvvZyanpOE49XdrC/O+2rDG4n3WKFvVKK1YPqioNcOSLY4YGysnBeoZCS6condlYsodF0gHsU2bkN795HVx3O+lpD+/7e5/XG3HFryAl/QLUngH+3AIRzAAEV1Gv+IojuOr+Hdro7W5jMbRqmcXbo3Wzh+cKM3Z</latexit>
i) 2 (R ⇥ {0, 1}
Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI
10
How does the perceptron learn
the model parameters?

1. De ne training set

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

11
fi
 1. De ne training set

2. Initialize model weights and bias to zero

b := 0
x1
w1 := 0
Weighted
Threshold
w2 := 0
sum

x2
Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI
12
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

13
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[1] [1] [2] [2] [n] [n] m n

D = (hx , y i, hx , y i, ..., hx , y
<latexit sha1_base64="Wal6cJLWU5bzcyF5u646zqqA+4A=">AAADVXiclZLfbtMwFMZP0jFGB6ywS24sKiSQpiqpkOAGaYJd7HIguk1qs8lxndaa7UT2CaKK+lo8yG4m3oQbJE7dSvvTIYGjyJ+/851fEsd5pZXHJPkZxa2NB5sPtx61tx8/ebrTefb82Je1E3IgSl2605x7qZWVA1So5WnlJDe5lif5xadF/eSbdF6V9ivOKpkZPrGqUIIjWWXnB4zAAAeEKQiaNTRwAHNg8IHu11TV5FqY0CzJuU7nUFD2O2XPaB5CChnpPcrMbjkjcDcIe4Hxr8z+GrP/F2YvXP9Ht2t0ew/9TWAoWi/345qWU8+XQDMhg5Qy1OHDqoGE2CmpOTHOwLLt80436SVhsHWRrkQXVuPovHM5GpeiNtKi0Nz7YZpUmDXcoRJaztuj2suKiws+kUOSlhvpsyacijl7Rc6YFaWj2yIL7s2OhhvvZyanpOE49XdrC/O+2rDG4n3WKFvVKK1YPqioNcOSLY4YGysnBeoZCS6condlYsodF0gHsU2bkN795HVx3O+lpD+/7e5/XG3HFryAl/QLUngH+3AIRzAAEV1Gv+IojuOr+Hdro7W5jMbRqmcXbo3Wzh+cKM3Z</latexit>
i) 2 (R ⇥ {0, 1})

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

14
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

15
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

16
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
(iii)

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

17
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
(iii) Update the weights based on the error

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

18
fi
 Let’s walk through an example!

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

19