1.

4
Our First Machine Learning Classi er
Part 4: Perceptron Training By Example

Sebastian Raschka and the Lightning AI Team

1
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
(iii) Update the weights based on the error

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

2
fi
 Let’s walk through an example!

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

3
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
(iii) Update the weights based on the error

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

4
fi
 b := 0
x1 Prediction
w1 := 0
Weighted
Threshold
w2 := 0
sum

x2

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

5
 b := 0
x1 Prediction
w1 := 0
Weighted
Threshold
w2 := 0
sum

x2

x1 × w1 + x2 × w2 + b =z
Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI
6
 x1 × w1 + x2 × w2 + b =z

b := 0
x1
w1 := 0 Compute
If z > 0
value

x2 w2 := 0 z else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

7
 x1 × 0 + 02 + 0
x2 × w =0

b := 0
x1
w1 := 0 Compute
If z > 0
value

x2 w2 := 0 z else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

8
 Epoch 1, Iteration 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

9
 Epoch 1, Iteration 1

If z > 0
else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

10
 Epoch 1, Iteration 1

z=0

If z > 0
else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

11
 Epoch 1, Iteration 1
1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

[i] [i]
a) For every training example hx , y i 2 D :
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
Sebastian(iii) Update the weights
Raschka basedFundamentals,
Deep Learning on the error Unit 1 Lightning AI
12
fi
 Epoch 1, Iteration 2

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

13
 Epoch 1, Iteration 2

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

14
 Epoch 1, Iteration 3

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

15
 Epoch 1, Iteration 4

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

16
 Epoch 1, Iteration 5

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

17
 Epoch 1, Iteration 6

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

18
 Epoch 1, Iteration 7

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

19
 Epoch 1, Iteration 8

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

20
 Epoch 1, Iteration 9

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

21
 Epoch 1, Iteration 10

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

22
 Pro tip: shuffle the dataset

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

23
 Epoch 1, Iteration 11

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

24
 Epoch 1, Iteration 11

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

25
 Epoch 1, Iteration 11

z=0

If z >0
else
if z≤0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

26
 1.0

b := 0
3.88
x1
w1 := 0 Compute If z ≥ 0

z
value
w2 := 0
else

x2 if z < 0

0.65

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

27
 x1 × 0 + 02 + 0
x2 × w =0
1.0

b := 0
3.88
x1
w1 := 0 Compute If z ≥ 0

z
value
w2 := 0
else

x2 if z < 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

0.65 28
 w2 + 1.0
x1 × 3.88 + x2 × 0.65 =z
1.0

b := 0
3.88
x1
w1 := 0 Compute If z ≥ 0

z
value
w2 := 0
else

x2 if z < 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

0.65 29
 w2 + 1.0
x1 × 3.88 + x2 × 0.65 =z

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

30
 0.83 × 3.88 + 3.94 × 0.652 + 1.0
w =z

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

31
 0.83 × 3.88 + 3.94 × 0.652 + 1.0
w = 6.78

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

32
 0.83 × 3.88 + 3.94 × 0.652 + 1.0
w = 6.78

If z>0
else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

33
 0.83 × 3.88 + 3.94 × 0.652 + 1.0
w = 6.78

If z > 0
else
if z ≤ 0

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

34
 After the Update in Epoch 1, Iteration 11
Decision boundary

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

35
 Epoch 1, Iteration 12

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 13

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 14

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 15

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 16

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 17

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 18

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 19

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 1, Iteration 20

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 2

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 3

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 4

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 5

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 6

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 7

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 8

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 9

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 10

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 11

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 12

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 13

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 14

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 15

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 16

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 17

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 18

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 19

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 2, Iteration 20

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 3, Iteration 1

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 3, Iteration 2

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 Epoch 3, Iteration 3

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

a) For every training example hx , y i 2 D :

<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>
[i] [i]

(i) Make a prediction

(ii) Compute the error
(iii) Update the weights based on the error

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

68
fi
 1. De ne training set

2. Initialize model weights and bias to zero

3. For every training epoch:

Iterate until 100% of the examples are correctly classi ed:
a) For every training example hx , y i 2 D :
[i] [i]
<latexit sha1_base64="KiRtzBWzB6OcgzuVeAuk4oyYAnU=">AAACJHicbZDLSsNAFIYn9VbrLerSzWARXEhJRFBwU9SFywr2Akksk+mkHTqZhJmJGEIexo2v4saFF1y48VmctFlo64GBj//8hznn92NGpbKsL6OysLi0vFJdra2tb2xumds7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648ui370nQtKI36o0Jl6IhpwGFCOlpb557jLEh4xAN0Rq5AfZQ36XOdTLj2CWlghdUXoon/owYtlV3jfrVsOaFJwHu4Q6KKvVN9/dQYSTkHCFGZLSsa1YeRkSimJG8pqbSBIjPEZD4mjkKCTSyyZH5vBAKwMYREI/ruBE/T2RoVDKNPS1s1hRzvYK8b+ek6jgzMsojxNFOJ5+FCQMqggWicEBFQQrlmpAWFC9K8QjJBBWOteaDsGePXkeOscNW/PNSb15UcZRBXtgHxwCG5yCJrgGLdAGGDyCZ/AK3own48X4MD6n1opRzuyCP2V8/wAUwKWr</latexit>

(i) Make a prediction

(ii) Compute the error
(iii) Update the weights based on the error

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

69
fi
fi
 Limitation: Classes Must Be Linearly Separable

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI

70
Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI
71
Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI
72
 Zooming In On The Learning Algorithm

Sebastian Raschka Deep Learning Fundamentals, Unit 1 Lightning AI