a) hoadag 7 edied wes giadicd doemd Sypitle ond ternal ? << oe rina, in maine Wasniy & 00 lading kh tesla AE ly ta, Ye the ae Gut Met 8 hg dat oh th lin ve pl yun Rent « a x Me Goagt f Oaatad ua ahd ndbabad darian iv alec TE eb ak en, a’) ae Sen aLFOYS G09) _ alk) 4 (q4O) apo . al yee) 89), 669 (Sa) ceo) 2409 = FO sae) ee esr Se = ONC ae ate) ae Bou Petes eb. ha ds CLE a) ot gl | . 5 0 function Fem) » FR Ctongli FLy)> ¥q ieMeniy wr oe Son . StS oe SY ae gtary(es) ee) = then wy yy am th wut bw te bier af 1, om th ing - kk Or ee me wt do om juan Crapgalion «Tr flail Cepryatn we tae ore nike fat be th bbs ie Xl ts Hen seg Be OE uy, ct Won tee Sen Wd ts diver Junction ad 9 hes Goth tats alee» Suiholy ve dh Cebcles wat ww BIN TL Ts chee th Ln prc tod BN ak ey updab the weit sunbeds Guan > wo, - lanadg ak y alu) dle) Wy 2 wy learn nef» dL Un) See cit) Mus) ©) soFo YU any only, by Uaagiy ww auf b =, af be oF Wt we gt a Gn Lan 2) Siihl, me gt UA elo seo by, OS baby ettbn “tue! gt 2sily,---tn Mo We TW el Poph badisen Weigh US blow us ls sine Ur her & dia 40 BaP om ae We We ony tut of Uhlan so,cw gt rogagh & eld as Sham inane > wild So, M4 mein ain of th giadid Mtart hb gt Uh wdttoad bie b we Bath gt ma UA. Th & Hedy hit Us gradntdoas ait bd Loy Sumy te qagh os fon tary HL kar Ue bn i bo Bt Gwpute ohl is a tiene alich Gnd Alewiy tal os we do -Apodint ue got th need paidt Wa = Oa lamig gk x SLb) don) Re ts ht Ga, 1) Wh, feb met pot + Ts Rome ati wt we gt Yh qckt = 0, becaun Ue th odlange OU pS ns, oun oe GK BS ot arn ghbIhminin, . So ue Slt PS wa ade Goat unt » ©) Gredint duet? Te ereding fered sue the all th doe abatine, Compute equalled, updat Th cid, raped JP nr runbus of pohly « WO Sted gradi dicot) cue Rardin, SBulile Sapl OL Carin St Cte oMIC Cotloche pitts Hodet , pereepion model, matt ayer Perception moder ft. ER Planation, diagram, odvantagts § duadventegt 4 2 tee ate Cuttoch Pit nennat mnocit, ubich was UD earliest Ani De ay Wesonty two types ef lapats— Exetiectery aind Inbibitery- aon Wh us ‘ Helene) We Exeter Inpumr have wmerghts of pesttive wages i : eoectela in We WMbitey weigh hawe weight of magative ™eg NONE input ot HC Cullach — Pitts nennon conld be efthir Oot |, db hes a Mrcibeld Qunctin as activatern —furcten> We perceptron receives a cot of Inpul Kiy Rao ~ Be Bere had Uli cheese wheal the vesmtliry Cm’ Peritive 02 negatne. y pecttire outpur a te and If magative outro ol ri btas asa Hewever pereepten Irctrda on acdjmetolle vale ht AAditiomal wetghr We. Thi aolditrenat merghe 4 attad adem wepur mo which U asevgned Value Of), A Ve 2. veeng Leo Ongue Me = OD. i +, Voo pa .CI20) > Me objective 04 Ue percapton is clareetied ate cet op input tte two clasees C and co Pt com be dene with a Simple decicron wale 4 IApurc: hes ety percephon be Sens +] and 2 bese val w Z wa 20 to + Two inpur signals denoted by Vomiale %) ard as Wo he +1, +W,7,2 0 Wo Tia, tWi1, 50 Pengo rn always be emplyed uous decisions Juice 04 dacicia, Reape rp lanes wi docitien plans ee oeeer Ee Ora lees penseetwn nguaoe ancy Cent et dined te used & Pl WE aral mctooneiinnal ions) Wen Ceplttly Real Cfatictionl models urbich meant lca model Con eC ace Gastron Meaty function. > We Cp ertpate a function unbich O giqnerid earity Chat is Unter Disadvantages. Dolind egGieek Magia nosis OL PINE wewral netweovid con epi e > the dericien b net TELE es Meseesl atid bar only allewed te be hyperplanes. Tus aundet onty worl ctor UL Untorly cepronbte ete ata a fay connected dente laren, urcbich teareferm eg inpur dimention ts Ube Artive Aimmcation > IE bos wwlti ayes, B reac nerral network we tembing Wiwrons bog So Wat Ure coutpuG of Cowl newren are IApes” eh CUA venous, ‘ > We © 4 fan tee hee that Zirnpug Ue 2 IAput wodos gud biel, has Pe lager 2 nody (to sutpit la A ewe & outputs ae hodw FOutre Lye Layer Ley cy tate apne ant peruserd he Aer fell, pres W Ure dtagaem Pie) Wes os tad lager formate Chery owetnt creas of 3 nedte hedelen leg Aud jn Ue game wey Le ce etter ceca Uae Iveta. and panu tt & output Loupe te Witt Cametd at action furction. es bekwoen © andl. 5 WE nodes in We Input tay [tfoker real values Os Inpet and converse Lem te Aucmto Y= aah ul + cep) ron) ple eat tne gl ole Cam be Applied © combux nor— Load probter, ee a > worlr with welt arqe in put data = Prvids VAC prediction ae. Be MS come acenran ig ra pigeicay, eda clu ved Se TREE —neTn Eon even ull omateravant; ‘ pee ie pe se egleut cack Indeperdent warialte oO tHected by Ure dependant yavnble: Compuctassions ADfHeate ond tone Crnsuemting , > ke PIOT furctiovirg eq the model dapends Om Guallly oy training ‘:% Dedoribe Vvarou — ackivation functions , use and alo : plabions. ' PR to ates jlin bw wy Soylent pan an astitided = nerd network , trey barically decide whelhn the neuron shod be arhiated ot not: Sr ANN, activation function defines Hu output Es that eet- noolu given an inpuk ot seb of tmpult: Thon au dita byes of achvalion funcktons, such as & Cinta, — Activation furction: Linear Activation feretion alo known as "no aclivalion’ bk whe te = ackivalien fopertoral tp te énpuky Jhis funcion dount dp anything to te weighted wun, it uni oud the value Ut was given Mathunatically , an Le ducibed ax, TIN) =n Pised vantages > dtu not peulble bo we back propagahion os the Anivebive of te pundcion 4 9 Constant and hos np station to input o, BREEN Lofn ba neal’ semi ol collapse Brineeropy” | me Unio activation function “uy Wt besenbiolty & — Unan achvadion function teuna the newrad rdwesk — urtp jut one Gayer& Neon Linas Activation Funchons, : ee Beare of Umibed pours of Ura furnclions, ‘ins allow Ye mots fo wk —womplec. mappings the —ndbwotk’s dnpuch ard oulpuds Bn Pdiay attoo backrcpagelion because now te furetion would be adlaled to inpul, and ‘abs PMible to go back and understand uhih welebds in He tnpud ramroru aN pick a beleer predictor ad ray obte? the tacking g multiple lays of rewsons Be uu woull ty rG bes nen tino. tombiral'on of input. > Shmeld /logistic Activation purction: This function taku any sual value a inpul and culput = values in the sane of 0 bo J. The ong te pul, the closer the oulpud value to Le J, Be UL Wey eat, tte chosen pa ape cae hem emenh Function: Tork function & vey winter tp te ogmotd/togie adivabion funchion, and wen has tte fame Shape with Ue diffuse othe ouehprad sarge — foen me fod tn Tar te tong the inp » tre he He oudyp ut value wil Le 4 whueas the Srraliy the inp the cog tte oulpd will be boo.au Tank: Fin) O'- 8 on ene 4 os t= 7 ys aon oe NO entON TANH FUNCTION > Reo Function: ReLv vtarch — for Rechified! inten Ove: Rev has a deivalie function and CUES Myre Ce eee oudy paling tue aly ped tas pacts dou ee all te neuters ab He same She newsons erty activated if the etlped of fra Waban. trapuforrna- -bln ob kes Han oO Reo: Hn) © maxclo,w ad Leaky RAO purction: Leaky Rel0 & an impuo veel weldton of ReLU punchlen fo selie the Dying Relv problem a it pm a Sra pte slope tn ntgatlve oem, Leaky Reo + fn) 2 maxc(oln , a) > Setar Function: Soffmox Function k dumbed au a eprnbinalbn of rubbiple atymetdls H- caleba, Ue Aclalie probabilities,Similar fo the Gigmold/togistic ackvahin punchion Gym “functos shun te probability of tach Aas. SE matt Er ae oe Tecliesomypa sa? fo etait lay gf rural netuok 2 te ont ae ee Softmax: —scftmar(xi) ~ _eplei) ZB, ples) rot a ee ‘ Oe " 6 Ob) a : pA L tr wo sweeo Shoe OBS S00 ge ON a aay er “10 -¢ se 0 RLU FUNcTion LEAKY Rev SOFTMAX Function) We nee fo mat ou acHaHon ferdlion por ouk oulput lap AF poedlichen predicted vaviatte . a PRL funchon could be Adder SE ee perce should .. be Used in baigelen (ayes at they male He ae imei Hespceptibee Lp robles diwsing Fairing due bo Yariuting — padiiends , Re gee fontken could oe ay tn gu eee oy Ye but pelle Mebabitity bard, annemDerivation qradlienk durant of Sgmord and tanh activation punchors, whith of tun ‘belle, fo we bared on application fprachcal scnaxio A) Lojeble Gymeld Adlivatien Furetion: Jhis function Jaku ony ol tolut 08 input value and —oulpuk value in tu harge of 0 lo 4. Shs often used hg binary elossitjcobiorn problarns as tre output function of tw model. Gigrmeid : Fl). | ee n legistic Wgmet on dito be deed 0s the madmum UkeVhood solution to for legistfe Aegausion tn halite, Coleulaling te deivolive gf bw logistic sigmreld purclon makes use of the qpotionk su and a woay tab adds — botte ond arbbackh 9 ome fom rumnaler ten. (ree) Jn 9 Sy eete (tery (by dain rule) aoa =] eerie s ; a! ag ee ef 0-8) as siqmorn (pewalve) As ue ue an st fiom ta paph above , he guodint volun aa only ianificant fm range 2 0 3, ard he gioph gee much flab in oth Aigions dt implitn thal fo volute prota thon 3 das fran -2, fre functon WIM ha vem Small yadlerds. eee ego scree nie, be cos to an ond afd fiom Vanishing guadfent problem ae The. oul of logiste porcton Nyt ; rramehAs Ss a To cite i. corey Wit, : We oki, a eign ane ndtwok — rere tps cull ore ay oa: ‘ba F ee ny ee et Bs. Be cn ea ee ; Shoup th leva gmail woke wad!» oral may cause 09. neurad oy iz a a ive input qh “ we tke aid -exttvolbny to cadeulate pruamneler guostioy ts can Ault ph modal ee ee oe updaled Aagulaaly Hon we would Hike fo, so er alraln actiralin forction, fark; Fade e%_e" ee) Gradient fr larr funcHen Faz 4 tar) mm =) Giakta)/cosh(W)) an 9 Mahl) x wohl — ) wahin) x Sint) an on z eaihilex) — Yoh'tn) cog h'(a) e 1 — Lan h*(m) The dlawabve. dG tanh function Uke tha samord $ Spunction’s — dlawalie 8 funchion % feedTonk (Devvotive) Jhus abso face Be pooblum of vanisling gadiods Geir bo te nymneld achvoten purcten + She gadient of anh furdton n abo much cepa 4 compared 9 agmoid function > Mithough beth Glgroid and fonh pace varishing guaclik dosnt mobi, lank Bb saxo erdeud ard the gradients ow nob auabricled fo mow alain dirscten. Thasofiowe , teak neonUnsanty 4 abuso petfovsd to gigrneid) nenUnansttySone \ € eNO} a. Concept of loss functons , derivation yf MSE e. Cross et | loss functions | nrg Loss Function: Te is O punction that computes the gicrance between the Curvene overs’ OF He) | aigoninm and the expected output. Ts 0 meinod bo evaluate how the algorithm modeu ine data s Types Of Loss functions encea between the | @) Mean Squaved ery: Mean of saqyraved aiefe* actual and predicted value Gi) mean Absoute Ero: Caicutate, the avevoge of ene AbSOWE difference petoeeM actval and predicted values Gi} Binary Cross Eneopy: Te gives the probability vawe bere O and \ for a classification taxk- Cv0ss - entropy catcuiares the average difference berween ke predicted And acksal Probabitite. tin) Hinge Loss Used when the target vaniabte ha» | oy -1 a Clas abe. Te penatixes the Model when there is a difference in tne Sign between Actual and predicted powd Lines ave the ewors bebweem actwat point and predicted > The General equation @ tine Having store mH ana intercept iS: Yemasc: we have to find m and ¢ svch that tt minimize te envy 2 To catcutate the predicted wawe y we Need an equabon : | mse = (Cu -emaaG)) + (Ys (ont +Oy a (go - tenta+Oy can be simpliefied ar mse = ye - ays (orate) * (ort Abd at YR AYA GNTN*E) + Conta +6 Groveing eimnilay items —-togethey = mse = Cyrene) CUS > = sagya) - de Cyr +4) * om (aes sao) 9 ammo (ay y-= 10) 498 EEE3019 A NEw characky for al mean Squared VaWuer Wawaeaud 4 n ad yeaah aygars syd gro The equation is tyanksmed a4: 5 ; gs nauapagre Mees nyt — amnay - abNY 4 NT + armbax +7 5) Since we ave wwoKing for a minimum point, Wwe will take pata derivatives & Compare to 0 Ouse = QMSE . 9 am Oo e AA “ . | Sse -anzy 4 an%m +2bNx =O om omse x -hj 4 2 m4 ter ao ab ase = mt ache ow 2 6MSE = oy om | s © lez | Suptiating fast from Second 2 a |* Mse = a penvation of Binary roa - ENOOPY a} ier wi amsume two fonctions Ye Fac) L Cangd = ~y 03a = (1 -y) log Ci -) 4q ig comatant while a fs oD = Vre* (2,6) = w1tb | Ldwia, boy) = -y 4 i | find portiat derivative of Lb with 2 Lecged] = Cg) 10d 2 ab a, 3z aod ea a% Aw o ou. ee = & (-ytog @ -Cr-y) 10g C1-2)) Ls composite fonction function dependen = WHTD WA a Peandnng omiecad tb Oneels Q4e*) Cae) gag \ (eS a) | = ACi-ay. —— = Ob do Ot A BQ at Hw : (223 (20-09) yo By SYauroy a ecieuyan =p eh (1-0) S Ck Qa) Cupar Sepa oe ; ee |} 1S @ common We fonction used in binary elaxsificaton with a WAXSke yeqvesion model.