Trtsoduckton fo Owp (gcentng Ly grapora £0 tsccatntg Naural puefwore, Nouset wekoot z Houring poet Pxachickrorn ot aS on pave a Site . oa -» O arte es cw cy) cf oo paymok wakwortl. S RELL aon Buncktov geach pdnoout unth. Woure Pita pgfan Cr a a vasownne 2 6 _2mity size od a) C3dnple woicalat! oo, ZAP LO d2 —>OD m4 £2 weneoost) cpostot coe me eo 3 ©) eno! es Cie) ~~ WwW aaa . ple prc ouk OY ge v eel.Neuse Netwonk Hrg in aa ON awarey inputs Sunucovy Pret - con raakot si Oe Anker. caconmabely mncrP AHO Se CO Korot ty Bie Coa wealn Juperwuirect Lapoenting, ota Weuxct Natur omnak makwoHll UM> one type &{ ML ceetack cel wonntng. Supa) eurccett'on Ni ! freet ae <> wrnpubit visto a we we _> pontine aduststny Cproto 19ging) W Js Root estate ge SP Spann rRcoqntHon - _5 Pukonomus a Qo macond ssaang (ation odttutag. Sh o4tfm | nybsicl NN.forr proto tagging Cima axppuceton) we ure Ae Ceonvounrional nounot naokwor'e) wd WR RVD CRaessa nt Stowewad hate a E roones 04 At Caagonad moan ) Cao's) anitoubwatl py Puckro immant Ree debe (pinot yatwn) lurwonk, Deoplicerning Compubis (20 Pug wo newsal cuckwecl data mug baleen. undoatandh Fe un pewicd nintwonke fsreanspormecl Saperuried Seen why ts Daeploasning fataing off ? ee { 1s_0 aa cal Ss er ae Sad dred ap teooniny Prog ee comme SY & aa i Fy srnall nn NS Eradivonal Lung cdg ent, Lise 4 Amount 6 data (lobsteot clater')Pufonmanst a Bigger ~ {2 data 2, vomputen CCPO / orP) 3- Algor nm > / @ J ( Joye algerattnm - i poe y aa (2 Eaposiore mS Cok peur rakenesets yh eee Nekwork. Suatteliing 42407 Siqmotd fundton Fe Rew function Provuare 9) Simon geopa 9{ kre Suction, Ree seco a cae So ces nuny pecormes S100 Slsw panaenebin nani oe ee apaevant =| Na Yatwws Fost om pukion 1S important uy ts spoodling up for gptteny caporimentol seul factNowra nnkwortls leas logish't Roqpursion on a_ muna ai Forward Propagation is the way to move from the Input layer (left) to the Output layer (right) in the neural network. The process of moving from the right to left i.e backward from the Output to the Input layer is called the Backward Propagation Forward Propagation: Receive input data, process the _ information, and generate output. Backward Propagation: Calculate error and update the parameters of the network. Binasey cdorsiRlorteon logistic re ton yan angersinm Cherargreatton. Cy = (10) 1 Brno The activation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. The purpose of the activation function is to introduce non-linearity into the output of a neuron: yoithouk actuation Runcitor, Neunad Nekuworrt us aba Unger saqparcinin ect The activation function does the non-linear transformation to HETApUETIEkingi f complex tasks _vce Shere imag Uh Cornpubtrn / Clow inoue Sopasake cratsas (ROB) N= hy SD Brandon \npuk foakuse0s % C&eobusre vecter') Notattoy) Ora ra CU Single Socedning RU os youn Cay) = re RS, yesad ae Ae ie . m tote nea oxen PQ Wea yo) 38 C307 oft) IW) 2 cen We i * 2 i fen] Vax Cotes \ ) — F gor: aay conuantton Sf a Me XM paanad rrocste) ww E1R > LNx 0) aan | Sgeal ee Ny. grog 2c4 9 y ein’Logistic Seqersion (eon tiny Gly ore ten in Sep tre output (abst 18 £017 suite Wewrarny Whee Ovuen %/ wank ae ay etl) %ER™ 5 paramkin wer, per Dutpuk 5 $ =otwe ab) C&rqmord Punck'on] A a Be L> Unser groopers tor Cz) Rar binwry cAevargicateon Clo) wn O25 20 ie es Os Co ae (acm a JZ Longer, Co) ae = \+o HNQ (Oye, \ 2 Po a la Yoga, akg Poor mba I ( Wd or Sopa Ron porns) NiksoHle } oS weer OPE + C ecesrartLogestee geacpurdtoy Lott {untb’on —— —— roth Yuncktor bo Loreen tho (ogiShc Seepamron mockel, ES ire) cee a lao Logeshe Sxaqyurston CAP (oe uiausscls Gs UE Crraut Nowseot Usoxt. puckton, computes Fre assem for a Seng 0 1 Ig Myo GUeaeye She loss jmautning acampe ne vot funcko Bf Hine (oss Sunckton oy the eniigce sorcuuncng cel. ee paver Seat) «CAMA Fowont SEM 4 PG Oa 2 a pontt cLo- (ogg Runukon (OUR) Lote chy? Gor routas 0 optimise Cesnvests Lede Cgtos } « Cty) loglt-99) wank Lots {uncon bo be Smal 2nough- TCE LLG,y) = — og fh > want (04 F Laxat, want o losegg; a Rysos (4 Y) = -(i-y) loglt-F') 9 want (ogCt-7) lax, want G sma. Wht funckion m Jone Lg 20 7s) yr t a Cy } g) m - <1 eens i - 2 oto as ‘oq ee Se loyl te" J (rowebtont Dawn pus att funckon Reynosa cst] Ceca le Lords Cantotnize s(44P) a arom) Rout £ (0 ncn ake 2 wie We LATA of: BOM er Caae “ 4 ator dw Lp wr? oa x LI) 7d or? _ el qo) co) p(w) +ve eC tohctt Dw ovtatmum Cuuvats .—> Slope & the Quncdton asy) (wx) Swprak s Gono) : oc a (Paral olouvake ) Ore be Coie) cho x Oyrivalve: —D Slope ay line Derivative is the slope and slope is different in different points in the function thats why the derivative is a function. eas Geivg O{ ABUVALWe cone aifpount ak Lenk Pn ; 1s on tre eau2s bor nakto of hot vs warght Rirgenont- ir gore? glow © > a Computation Gixaph Its a graph that organizes the computation from left to right. Tle) = 3Ca* et) uw Be > ae) aSLompubing Cs yoltuos* 2 Connpues om Right Jo Lule Kren He? up cOMLS (ot mor Dov CIF 0 CSaruwerl propagation) Qw Qasxivat Dorut votive sity Os Lomnputatton Gmagh alo dartorttlouNaaten 2-% LAS Yo wre Huss VETO Of Has roc - on & Daw sop A Backwond propaga ben rou output Computation qrapn ypilola. dou vettue Oy 401 Yosutalplz « Qr2vot cts ay C.Chein cule) av aa fo clanivaktue Socom sutqnt (© Ve duiseectvon Lom pubakion XPM (Claes gone pycto seyht bo Lomput? cost Santonlogéstc Regguraton (tore deont Descont 2a eee) i Se Ce euce) Maa Ne Ccica a Crayproge -29 onty 2 BrokswecA’ a) Cet aac ate f 2aeolz) to penebiien LOK - Lp Alay) forusesrcl Propagation —> fo compute fire Loss. Ginactlent aunt form! fgrountng oncomple cn by Gil ices A ceatg”) Pe cea)vorkoutzo \fon Deep learning shines when the dataset are big. However for loops will make you wait a lot for a result. Thats why we need vectorization to get rid of some of our for loops. Upckorude cata Con signtfecontty gpaed up youre verte Sotoabolitty of eronpeasinuny (S Rone oN 2 &Pu © Gago, prcoewthyy unit) In Jupykan Nokelocolc —> CPV Born CPO BS EAPO haus Posuatbizektory Wnttorudton- Mn coud SIND (ngkscuchtion Lb staglg Wnttoarckdon MULe'Ple pate. (apo te oot Avotd fon Loops, We vackosured fos yautontaing (ogistee saqpatOor fo Qind loop wp Ze np dot (WwtAy* » Vocksutzation. P= | Lana enet-?? Broadcasting works when you do a matrix operation with matrices that doesn't match for the operation, in this case _ operation by broadcasting the values.In general principle of broadcasting. If you have an (m,n). matrix and you add(+) or subtract(-) or multiply(*) or divide(/) with a (1,n) matrix, then this will copy itm times into an (m,n) matrix. The same with if you use those operations with a (m, 1) matrix, then this will copy itn times into (m,n) matrix. A wo Cap Wye Vedsizatton He Compute Daciewarret Propagation , to vompuke ayencleont . ish Ponts Grsach’gnt vackoaizing legishe Ro, fan's Crack? psa vornputa ken m =o bo) 2! an tunlee) ONT te rn az! Aw oe satel) . { az cy zh eg oel ae a oe ] m Zz weil =n dot(w-T*) sual) Gedeamd/e | ho np Oz aah en ae f aot yn ot —?P | awe Ym% Ab = ton MP Summ ae)ont Desert update Croatian Dent upde C5 Seale jotixetion . We w- 6 dw rssae Ore) = w- “db ee a Bre fon (oop 1S tag in pylhon secyrirud for ‘an! Broadttaaling 4 oon pytron cocte stun foarte. (few Une) autscO 3, Sunn intrclly, HL AHO > onpond bo Size | Me Pyro wil reco Coca ee vauran’y v. : w aku - jt ma loutA. voy stnangy and and fo in If you didn't specify the shape of a vector, it will take a shape of (m,) and the transpose operation won't work. You NEES Rone © Counce Cate ca Vlukan NOV column vedo) Pon urg Tank | arncuy (0 pebsrct Newonl< O57 Sernope Khe array, tig Core) fo choell tna cocke f and Quan rusheps the ary’)Explanaton Of logistic meqrealon lost fanction what dork A neuston compute ? Cp pusput Of & Newson az gw) whee Gg tc Ho oeltvarton goadtor Nounak rabusorle —> torn putatconcul moctel Ooep fonsntng —> Approach to AT [+e (nese sue Se plustlo) Con) mH Yet pial=s 4 4 y=o cy\x) =} y | ply (x) y? co aD oe onto (oq PIO) a 4 eat x cry) og lt) Dy Ase O{ WY Guncron . tye Feta ng goonntng obgoruthm + (agp. V2, if you You want to raatee probeutor lity(ry Woy rskXC magneton, Wk wen ko mintmite Mt Woes fundc’on , masiming kre Lor OF Hae groloclatlr ty, on MAG sao pl: wk funcktor oe on antrsre Levl fore nung Ske Censure, ™ pc Lolo in Arwintng, Carel ec a oye? \xed) i= pot MLE Cakab's we) ——— cy » U oly (xe?) ea (og pClarers or fraintng’) = 14 m~ Uae PC a) ot 7 tp ‘ ani Gon C Yeo) c sack to prinimizh tre use 8%, te with (oat hc Cost 2 Jl») " ay Wa ow we a stqn. carwwnrg OWS msShe He Sign T he v2 Sign fn non 6 the loss funtion tc Ud tO ens wae wie thet he optimizafio algasuthm minim he (Ze (055 Stattun than maniming iE ygumo Nakeorle Bacreusosr A \ cotouserbton Fs ss 2 yf \ gen = “> ¥ in or) geet msn ee . Sergi) Ns congeet af a os Pole lee oe > a) a oe") . pant pg buson He Strmtlast — psrocar® anoHn layin (en pracrect’) Supers 2d asin ¥>7tedotan onan Kay ln bovcutetng gk tra cover values ip Hr’ noc tH 9 mriddtg COV not obssweel me inpub Lay —> 08°? Coaktvation’) / " ne ool Nikos? Yolues Prot cbyysant Lays o Kne reused sipsengsants (Ore axe pwning on ko He § nt Aw Ingut iw Neal nokwonk, we gor lo loys Choy www) nicdan looper apneraks Some cet tompubing, & jausad NaKOn bate Mee oi _ oe 2 aye Ce A af atkivation . (i Sy W wows nh (agen Output layer ae cual) ad a : fe Hidden Layers Zewi a aw, A= eee) > 22wyab : : ' 4 ozclt) a ‘ Corktvatton ) Von Oren nk Laerall) = o(2it) al! = oz! 1 peace {1 8 ae a chante ounyoune nediss 1h Cackuatton lou varntdcoll yous) cg Zama wo i a au. ¢ (283) Gace cinped ayer) Vertortat wikis eoca.rn PQ ° peta) tees Ee cAI Ded ay S 28 wigan bo yt > Peel Gre ACL G2 ee of em ym APG) Aor? «a L Co gaamp le 1°Ctnain| penpagation ° “foul Indu corsearponcls fo atggsunt Hortzon » focataing acorn PL (eet fe sulght ) > vetial (nclag comeeaponcls fo oleffoont nvoolls 17 fro nauacd ne tenos le \ ben | | | \ ge ws i cy Vege a ' ae Le ae Zoo | Rc coer v9cto3n \ \ It Ackva tron Jusseon Seqenord Junuecor Ye a autuateen Ancor oe “2 it @ ows. eee ) C word be non-lUinean Sunckion) Pe a0 a function x (outer ggmotd Yundtor tan funtion on jypenloolc tron Stqmoiceae tonnt2) tanh > © VW Se, fonh = neo c gigas von o sigmoid J (Baek bother la ned dan unt, QC2) = sonnle) worth wun &} than Sigmow bez vor Peas; Tusk, Ouetuatton Of idelan Let ose THO. Peay ono) Coe myages woeyek 22H MIEN eat ageing tA G ws waseru'ny for ery nook (Myst mo WH pit goats Drea ton prBeween Ca Brooy chong Pico tt n P 5 Segmot i+ - oly a\ Ne ie Output layer D iN > ~<5 Input Layer -\& Hidden LayersCoktvation funtion can toe CAR ant on ges") > + OS Ryoea MAE - aikgorw ne layer - Deconsraa ©} sdyrrotel g tone Wey Smal , bron Qpcctaten t ot Knots Runt! ovr cutent cont Qe ig very laxqe om Ayivel a Of Mre Soper oeorna, vary SAU C Slow down 9 Rocrcgind Lingo wart cau) Oo as fey Orch. COCO ace @ portkue \ lot ay OMY gattve. Aystva us VS dard erttue glope Ze NM ZxO Cok choytnach pQsvedtve ) Rods actuation functvon Ye Wed tA \uldoben Loupe Some! re fanh Crommonly wyck )Anottw: uoador eo RELO MS Lent RELU Locday Roba Roa (erage Qebuatton fanckon. Advantages dy Rr 2 Casey Retr clyuluaktup ackuakion Lor of gpawe Of Z 7 ine QW AHOV) Cu nekey QL’) ontior Qanckto , slop2 Quniow 1S puryount AXOTO UNO ¢ Wr eo cee ola muon 4 Dugusced Ngtvd07" ror ria) oak wo vrel Cos €O ehgeck Of Sloped a Ly woley Balu ur Stqgmord Geuktuatton omy i TH binvosey Cawatpicatton PXHOCUN- Ol C tary affect of 80078 Arg ¢ a (Gs) can 02 Zunuton qth ae p) ponamekor tort which Slows aowe gow adgeratne. (eosriny~UWnea why Aor a neat nakwonk nod c Non ukivakion Junckion 2 If we removed the activation function from our algorithm that can be called linear activation function. Unwwn on ‘County. ackuakion funckon a Cos) a a y oS ace See we J nee funcr’or Oc cee ey Ua ‘ Rc = : a inpuk al a} - wr? (we a ve) ab aca wd 27 3 ( Accs weit)x a (ass) Lae) ) Se ee a oy Vannes Logi'rtrc pracpurston vsithoul chan (Oye. any Wet Hidden Layers <7 CUtreast fanck'on ) Un aun, tn Unxan ackuah'on fancton wugjut wnin you're Loin joe gaqpenaton groom. C YER) Qrz) =e Uintan Gakizebion In pubput (ayo > Rasyomston luk woun Lyn Ue Rete | tanh [sgmoret You might use linear activation function in one place - in the output layer if the output is real numbers (regression. problem). But even in this case if the output value is non- negative you could use RELU instead. Dostuebtue 64 Aicktuctton fusion gakion, you nasd to compusl _ UNcHO™s - Bactt prte Po a ie cakivatton f step2 ( clxdvakue io Sigmord : (ayy | 220,Glera ! eye See) Co Ly ie cove! de tops 4 aay atZ ' _. cate) = Slop Qa = Oe (1~ ne ) . noe Coe Ole) Ci- 42) =alt-a)ZeWd 2d =O ae . 2 a cg@\) =© a is oO pace = le. ae oe ae a Tantn asktvarion fandren gtz) = banh(z) = ole One ican eg ( = glezy) = & Cqtz)) = Stop2 % 4° abe aw 2 mo Ve Cranhizy) a oF pero adivakion furekion gir) = MO Coe) a Qtr) = ‘ o, 4 ae \ oe ces wer orn L) Be, Nunagoral 2 = Aer on oe $aornweatly but works fine “ fa coceeUsiay Rae oe 3 max (O-01z, 2) puch paton Hr2 Rev, apreesank = Oo oka ree ES oy Row Quucdon fe an inne. Role Ayartor gunudon tn zo wwe sow cuotkty ayaa, fulan grodl tn tnok SRYLOM mnoy core CEM Rol prove atizabion - (Ur worry cloy ut ince nk, UM prontom purn’ou7 fon symm oy pls cod ouyownt gu wound nokwe pnteratiaattor fo race Yon mn tO com pure olds zi] Vo lLULed, witl Ine pore NY opeesuiant cpsedny upBonk pro popatiovy wyske gnome vor oS _ > ewe 4 Ps ol oc) a Gare) eA -— gprwarrch PROP darivahive for Reeepesererh Pos Leaiyy < yh ed Pe EA So, yor end UP in ra Bak port sania [ sigmmet (roving wid oe Slow, so, soon | opradent : a) advcont Sty ops un aC Wary Sma Ye Slow - waters whet (S$ a Arp Noured waren? Trerrndseotty , VoeeNe geanyreraco VY SC ta paso puwow pps Nak WOH IC Gea a pool wprwornk Lp now pu ewortle tyith moe hon pager ( JOU (ouyorasou Cu deen. unt) aM%=nx= 2) chy UO op a quitvaktony tn (ouyor Ls Lae 33 al. gt ezt ) vas bore rar Of ws waqnt for # valu qetivalt : 23 (cup Jonwond propa anit in o_pesep petro preuralretwor > Ll jnchecokes Ihe clasp -- Way" No-Of pececton OP" ® ol PS il ae a, oo tv ( sung fseung eaampu, I< aca (za)c23) 2 st - Loz 4 fon trculnlng oxarnh* , ; as \ co Seeing Hot ZS 2 Q me) en, \ | } oneal vpor Of gon wore propagation ) pckton ON" {pwd > cuppereare | achvolo na fe aus fon-(oor fo compurt tho (cay U1 & dhe Nunrod (oy anol network vith & Slagle hidlolen quae seapnat' hg that moyen FM. pumentatior) Maksux Nimeng(or Custosdzcl 1M cy a ww", of pd HN 4 5 (AT pH CMY - ot 0 2d, wilx od Cusekoset ee \ \ si cs) . oie ae) ae 200 | Sai cat \ \ U : q Shake 19 column wos an aD ! % > Cased, me)Cop touning coampo stack horizon rally. pos a Cactd 9) qpakry © og iran Poe pode (nem) i bpm, Ce, cack het adacrunt OW cod form ) ey ae ae ! Za ce is Cusuor gor) 2a, act. com) CO ye, Cne07,> Qs O2 aS a) gar? (0, m) Denp Rugpaatetiord] Circuit theory and deep learning Informally: There are functions you can compute with a , “small” L-layer deep neural network that shallower networks asuien Lay require exponentially more hidden units to compute. =X, KOR wy KOR aa KOR KOK oO nousiat oli) | » Jol meet 2 a | 8) irs as dabei “> \e \¥’ oS clmper {20 PD \ * \o aly : Le. ure 2d Fogel in (ote @ wrnposdng Mam boy! wan more! of a neurad nubwor So Moy COM eee fanckor Chita CNN)we Fre Simple HhcngAs (olstached) 40 in.orsen Co cuteck ¢ things. mor compel losin CL mop paral pdbhsott pna.tegy dw > human Circuit theory and deep learning Informally: ote hat shallower networks rho require exponentially more hidden = to compute. ar oot K, KOR alee KOR. KORE 4 eas Yoh “A. ae «7 ASO % R oy fergesone hw Wa J <= ve. | oh 5 ak ose PUL! De >6% 4 \ ~ (4 oe vomnpeks ae me cose woth AP nyptusertlt Hou Frewtow pptworkr. ction 2 Boskwerror — funckt Foruworta ey cA) yet) prtT coun > © . = . : CL) yout Cbasusna, 9% pa te es Wer 497 . z poppogation. at ghz cet)Forward and backward functions » 3 a we od UO Su |S 5 ) wd ESEe +) @Be ot ( ¢ | f dc pms? = Bo Se | aa aS ee : LS we adit? — ph LB EM ag pox Andrew Ng QR wrt) one teeeer( Stl = uw bg & a os) wt) _ pte -ackot Grace os Cons * 9 ko pars informe. OVS Poem ono be moore ( fond. > veeele'\ Postamakars ancl hay pi pooro.k.tiennwyel paramos > wt, bo . thenrcebrion one) Cocesunding secete Col\ /ne-§ huge parent c pukormint Hrs other posccurnthes, Hraoton Layen L, Heclelen wns, activation fanclcorhype forwereken 5 vontxol He CW,b) and cetouming M0 fined vet Bf w, lo. tg cr vary gen pisu'cad ae Appitvert arp Uden g \ A Cp tq out lof F e p> uw on (aa \F e ‘nant v 2 a cou le cD poke ny pooomebe pig ht chang $0 Tho vouee Pr hyper puter inprosire ee whats Fodoy nel wpa from now ous 19 Onn po, OPO and SS no-of lagers 'S (np (yor €¢ Not counted. lata pele 078 changing. rhe noo, racioten (angers +1 8