a

ba= 0 6
4ivea thet,

b4

idden laye=

O Sx| +

1 + e |4e-o3

0.2 X | +

D75
1te- b (te-lT

Xs :| Xo69 +
0. 069 + o. 2 23' + o.

0749

- Ys
Yragt O. 743
 c.795(ob.251)(- o.749)

O.'X .749x o.14

o176X - 0. |4)
Q.639/ o. 31)(o.

4i-y4) (wyrX )

b75 (o.25) (o-2475 X - D-4)

C-SX 0.69

Y3352

- 0. 00243

0.5 - 0. 00 2 43

o.497 S6
6.)3 7s
 Date

OX o-y99 Jx -o.3 00 13) +-0-6

ox (o.43 54) +
+ Ix (a. 19 757) + .4

C" 66 2

R.a) Jhe

Loss

Number. hidden rate

Valid etitn y HeuitiaA Export Kuskdge.

wadel conmplexity
Requlazaticn reduution .
ratc
 Py
3 ¢(Rg
P
, (2)
Pz
othenisc
M2
Py Pattern
Gracists Vouisking Rik o Recucecd
Flow Gradnt Better
’
eHfiiCompuitinaal
en
’ )
4. ( Fe cture Sele on Featwe Exractiou

sle cts sbs et of elwant

on origial deianct. eetes.
featnes
mathem t cal
Reeney Yyelevnt
trans formato reate

eots.
PcA LDA
Meth do

Feture ¬Xra cti e eature Lerma

Aut matcaley disco ven

’ Uses predefined matheaaticad

vetok)

diuioeli'!
to exhracthiuarchia
pattesns.

ML
A Subset AI tht en alt es ML thet
sA uhal metorles.
w th Smal datayet.’ data
Les Com patatinally Tuteni cmputational.
Ex svM,
svM tza liyim trees, CNN RNN
 Not ")
hidden(entered z-
)
(tank)- Tangent rahbo
uscd Rasely -)
the inwed hiplirability
i)tan
se
1+
e
unctien 1
Activatiin Siqneid
molls mplex Co
simple
n
gradinte,
bayd rotes
adaptive c
Adan
 Linear uit)
ReLU
(Rectifid
ctiiecd Línear
maxlo,).

wsed in hicden layels

due to etfiuut gradicnt profagin.
") Not wyed becausc it
cutput laye
pIGcluceA

0.05 o.01
10.

0,20
twNg. 026

byo60

te = 6,5

Activ" Fynth =

O, 377S

D59

Q39 2S
O, 25 x 0.oS +

. o-39
1,1

1+eI
 Dat:

|223

1iet,22 ) o.772G

E= E(Tk-O) !(o.ol - o:7)'4 4 (o99 -o. 7724)

o.294 4.

d
(o) o75(1 - o75) .(375

C1337

0-2726(I-o-7 726)= . 756

6.t. f(z)= n't
nitial þeint i(xo,y) =(32) , Leanuig 9ate' =0.2

tep 1: compute oiradients

Jhe patial derivatives q fay
2-4

2y-6

Jteratien1 (aa3, yes 2)

Cqradient at (3, 2) :
JA = 2 (3) - 4 = 6-4 = 2

2(2) - 6 = 4-6- -2

qraicnt descet:
updete win at
= o- a . f = 3 - 0.2(2)= 3- o.4=
2,6

2- o,2(-a) 2to.4 = 2.4

Gradiut at (aa, 2.4)

2(2-6) - 4 e lo 2

- |.2
2(o.4)-6=
Lodate
26- 0. 2 (. 2) = a.36

= 2.4 - o. 2(-1-2) 2. 64
 stenatiou 3 (Xa 2. 36, y2- 2.64)
Gradicut at C236, 2.64)
2(2.36 - 4=
- 0.72

yaate
*-K= 236 - o.2 (o.41) =2.
f - 2.64 - o"2 (-ó. 72) 2.734.

Final Res ulb ata 3 Jtey ations

= (a 6, 2.4)
(2,4)> (2:36, 2.6 4)
A31 93) (2.216, 2.784)

We e derivetive
a l1-a)
uehee Hhe

To eeweient
ind the dei vate
ayerbeie (tanh)

d tanh (n)= 1- tauh(n)

et tanh (n)
2
d tanh (n) 1 -

a (i-)
2
divative = 1- a
 f (x)= n- 4 + 4
Jnitial perut , Me 5

Conyate22-4 Cpatict

to ydate cacl

tep 23 Ponfovm Jterations

Hicatien1(0 5)
af= 2(5)- 4 = 6

Tiuatien2(x=4.4)

4.4- o.| (4.8) = 3.92

3 2 : 3.92)
2(3.92) - 4 = 3.34

3.g2 -o.1(3.84) - 3.536

Finl Aeyults
4.4
3.92
3.536
3
9. Role of LoSs Fuueticys

(a) Measuns
Model Peufersmauce
the mocdel's di ctions matel the atuad values,

ojtiizeh (eg s4o, daw)

the

Adagtabily

classiticetien Los Lou

Measwcs how weell

freditd pedickd
elcses math actal lases . Values re cloe to atuel values.

Common fh: Cross -enfrory ComMon " : MSE MAE

los
Hinge
Disrete ontnuows

clays ates cat oY dog ej prei cting houwe pries

) Penalizes incorY ect casifcotnu ltnali zes Aarge deatons

Ex Ex'- sto ck