01924, [Link] Lung_Cancer_Detecton ipynb - Colab TO THE CORRECT LOCATION (/eagale/‘nput) IN YOUR NOTEBOOK, 4 NOTE: THIS NOTEBOOK EAVIRONMENT DIFFERS FROM KAGSLE'S BYTHON | ENVTRONMENT SO THERE MAY BE MISSING LIBRARIES USED BY YOUR ingore 0s nport sys ‘ron tonpftle import WanedtenpararyFtte fron urtisb. request Seport urlopen from [Link] import uncucte, uriparse ‘from [Link] iaport HITPECrer ‘from ripfile import Ziprile Ingort tortie ‘nport shuttl| ‘can size = 40360 fava SOURCE_M>PING = “[Link]-date-setsk2F174849942"28827862Fbundle XxGGL_INPUT_PATic"/kagate/ input ace NOREING_ PATS" hae working” ‘6 €_S¥ML INK eagele” mount eaggle/input/ 2> /éevinut [Link]("/Raggle/ingut', gnore_ervorseTrue) ‘ssnakedins (KAGCLE_INPUT_PATM, 00777, ex4st_ok=True) fs. makedirs (KAGGLE_MORKING PATH, 00777, extst_oxetrue) ‘5. symTirk{KAGGLE_INPUT_PATH, [Link](*..", "input"), target_is_directorysTrae) cexcapt Filebaistservor: ty: (8. SyRLEPK(KAGGLE WORKING PATH, [Link](™..", “worklng'), target_Ss_dtrectory= ‘for data_source_sapping in OATA.SOURCE_ MAPPING. spis(",") directory, dowload url_encoded = data_source_napping.splie(*:") dontond uel = unquate download url_encoded) ‘lenane = urlparse(soanlead_sr)-paeh destination path = os. [Link](KAGGLE_INPUT_PATA, directory) ‘with urlopen(counlond_url) as fileres, NanesTenporaryrale() as tale: ‘otal, length = #[Link]{‘content-Length | print("Gounloading {directory}, {total_Length) bytes conoressed') data = [Link] CHUNK SIZE) while Ten(data) > @ ah Len(eata) ‘fileswrise(aata) done = int(se * di / tne(total_tengeh)) sys. stdout wette(#*\n[('=" * done){" + (S8-done)?] {61} bytes downloaded) sys-staout flush) data = [Link](CWNK_s1Ze) 46 [Link]( 2p") witn Ziprile(efile) as 2/ile 2file-ertractall (estination_path) tse eh tare. open([Link]) a6 tanftte: ‘anfieextractall (destination path) print(#"\rbountoaded and uncongressea: (atrectory)") except HITPEeror 38 © Drint(#"Failed to load (ikely expired) (dowrload_uel) to path {destination_path)') continue except Ostrron as e: print(#"Fatled to lead {dounioad_url} to path {destination path}') continue print("ate source inport complete.) Fe oowloading tgorhnccé-lung-cancer-éataset, 208562228 bytes conpressed tom nvo=) 208562220 Dyte5 downloaded Danleadea and uncompressed: igothreed-lung-eancer-dataset Data Source Inport complete. |ntips:[Link] google. comisrvel bOp8q87OW-FL 0OKOP7VOCaBqSoxgMUEHscrol we,01924, [Link] Lung_Cancer_Detection ipynb - Colab 4 nip snstall tensonflow-2.12.0 # Install a compatible TensorFlow version 4 psp snetall tensorflow-addoneeeB.19.9 # Install the specifses version of TensorFlow Addons 4 pip snetaLl kerase-2,12.@ # install a compatible Keras version ‘contig, [Link] Jeet neat nunpy 3 m9 Inport pandas as pd inport [Link] 2 elt ingore watplotlio-inage 25 noing ‘from POL sport nage lngoet seabor as sms ingot 012 moar random Inport 05 snoort images ‘ngort plotly.gragh_cbjects 35 go lnoort [Link] as px ‘noort plotly-tigure factory as +f ‘from plotly. subplots tnport make_subplots from cottections inport Counter from [Link] import stancardscaler ‘from sklearenedel selection inport train_test_split ‘fron [Link] inport LocalOutlierFactor ‘from skiearninetries import accuracy-score, recabl_score, precision score, classification report, confusion matnix, Confustonkatesxots ‘ron skieorn model_seleetion inport RandorizedSearchCV, eross_val_acore, Repeateustratifscckfold from Snbiearn.ver_sonpling ingore SHOTE noort tensorflow a5 tf ‘noort tensorflow addons as ta # After re-installation of tensorflow_adéons, iaport should work ingort keras ‘from [Link] Seport Sequential ‘ron koras layers inport Dense, Uropout, Activation, Flatten from Karas. layers inport Comv2D, ¥axPooling20, GlobslAveragePooling20, SatcHorsalization from [Link] inport resnet ‘from [Link]. applications inport EFticientNetae, EfficiantNetel, EfFSclenthetO2, EFFLcLentNAthD, EFFictentNetD, EFFLcientNets! ‘from keras. applications resnet Inport Reshets® ‘from [Link] ingort load ing, sng to_array, array tong & Inport load_ing, Sng te_array, and array_to_ing from tensortle ‘rom fentortlon. eras. preprocessing. nage import InageDataGenerstor # Inport InagebataGenerator fron ventorflow kerss. preprocessing. in lptp install. tensonflow addons Fe collecting tensonflow addons DownLoacing tensoriow_addons-0.23.8-cpS10-cpB1@-nanylinux_2_17_x86_64.nanylinuxZ014_x86_64.[Link] (1.8 48) equirenent already satisfied: packaging in /usr/locai/iib/pythons-187ast-paccages (From Fensorfioe-addons) (24.1) Collecting typeguarde’.2.0,>92.) (fram tencortlot-addens) Downtoading typegusré-2-13.3:pySerone-any whl metadata (2.6 8) Downloading tensorFlow_sedons-0.23,0-cp318-cpsie-nanylinwx_ 2 17_x86_64,nanylinos2014 386.6 —evrertrs‘s':-'- “vO eID. A/ ETH. A KB SL? Ho/e 9 0700:00 Devntoading typegusrd-2.13.3-py3-none-anyuht (27 KB) Installing collected packages: typeguard, tensorflow-adéons ‘nctenpting wninstall: typeguare Uninstalling’ typeguard-a.5.0: ‘uecesstully untnstalled typequard-€.3.8 tnaoa: pip's dependency resolver oes not currently take into account al the packages ‘hat are installed. This behaviour is the sot inflect’ 7.4.0 requires typequard>e4.0.1, but you have typeguard 2.13.3 whieh is incompatible Successfully installed tensorflow-sdgons-0.28.8 typequard-2.1.3 unt (611 ks) rectory = "Jkaggle/input/igothnccd-Lung-cancer-dataset/The TQ-O7INECD lung cancer dataset/The T0-OTINEED lung cancer dataset categories = ("engin cases", “Waigrant cases’, “‘Wormal cases") size data = 0) for Bin categories: path = [Link](atrectory, £) ‘lass_num = [Link](3) tom. dict = 0 tor fie in os. 1istatr(path): ‘ilepatn = [Link]-joingpath, file) height, width, chansels = [Link]( filepath) shape SF str(neight) « "x" + str(aith) dn tenp_diet [Link][stedhetgnt) + "3" 4 staGwiaihy) t= 1 ose ‘enp_dict{str¢helgnt) + ° x 4 ste(aiathy) = 2 size_data(i) = tenp_ctct ntips:[Link] google. comidrvelbOp8q8N7OW-FL 0OKOP7V0CdBqSoxgMUEHscrolITo=xEBCSRrSOF¢-SprinMode=true aire01924, [Link] Lung_Cancer_Detection ipynb - Colab channels = Snageso, inread(FSlepath) -shape (rsengin cases"? ('5u2 x 512"? 120), Walignant cases": ("S12 x 512": Sot, ‘490 x 511": 1), Normal case's {0592 x 512": 435, "331 x 506° 1D) path = os,[Link](sirectory, 1) lass_run = categories. index(3) for file tn os Listair(path): #iLepatn = [Link], Fie) prints) ng = e2-Snread(Filepath, 8) it. inshowCine) pis. sow() break ana |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol01924, [Link] Lung_Cancer_Detection ipynb - Colab ° 100 ° ° ing_size = 258 ‘for i in categories: Fig, se = plt-subplots(sanples, 3, flgsizen(25, 25)) [Link](i) path = [Link](etrectory, 4) ‘lass_oum = categories. index(i) for cirr-cnt, File in eouerate([Link](psth)): ‘8lepatn’= [Link]-Join(path, file) fing = ev2-inreaa(Filepath, 0) ana |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol01924, [Link] Lung_Cancer_Detection ipynb - Colab ange = [Link], (ing size, ing_size)) Gausstanalue(inge, (5, 5), @) alent, @)-snshou( ng) Beene, 1]-3nshom( nae) axfert, 2]-inshowtingy) samples [Link]() |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol site01924, [Link] Lung_Cancer_Detectionipynb - Colab Bengin cases Malignant cases nitpsileolab research [Link]!1hOp8qw70W-FLOOKDP7vOCdEqSoxgMuF#scrolTo=xEOCSRFSOFd-kprintMade=true01924, [Link] Lung_Cancer_Detectionipynb - Colab01924, [Link] Lung_Cancer_Detection ipynb - Colab ana |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol01924, [Link] Lung_Cancer_Detection ipynb - Colab aaa = 1) path = os,[Link](sirectory, i) ‘lass_pim = categories. index(3) tor file in os.1istair(gat®) ‘ilepatn = [Link]-join(path, file) ng = ev2-inveas(filepath, 8) ng = ev2-resizeling, [Link], ing_size)) [Link]¢ Ling, elass_nin]) [Link](data) x yet 0 Xcappena( feature} y-appere(iacel) print (x Lergens", Jena) Printy counes:", counter(y)) X= mpwarray(X).reshape(-1, ing_size, ing_size, 1) aK) 255.0 y= mpearray(y) (a: 61, 2: 426, 0: 1209) ain, KLVBLid, y_train, [Link] = train_test_split(k, y, random stratigney) prine(len(x srain), x train-shape) print(Len(Xvalid), [Link]) 22 (622, 256, 25 25 275, 256, 255, prine(counter(y_train), Counter(y_valie)) FE courter({2: 429, 2: 352, @: 96)) counter((a: 242, 2: 104, 8: 39)) print(len([Link]), [Link]) train = [Link]([Link]], ing_sizering_sizera) print (eng als), XLtratn. shape) 22 (622, 256, 25 22 (622, 65836) print( "Before sHoTE:", [Link])) note = SHOTE() train sanpled, y_tea print (‘After SMOTE:', Counter(y_tratn_sa »_saupled = snote.tit_resanple(X train, [Link]) 1ea)) Fe setore SAOTE: counter( {ts 420, 2: 312, @: 99)) ‘eter SWOTE: counter({2: 429, 2: 428," «20)) X train = [Link](X [Link](], Sng_size, sng_size, 2) Xteain_sanpled - X train sanpled,reshage(X train_senpled-ehape[@], ing size, ing size, 2) print(Len(X train), [Link]) print Len(x_train,sanplee), X train. sanpled. shape) 22 (422, 286, 256, 3) 1260 (4260, 256, 256, 3) eauential() rode -adl(Com20(64, (3, 3), Input shepe-k tratn-shape(:])) rode adl(Aerivation( rela") Inodel1adl(maxPooling20(pool_size-(2, 2))) rodel2-adl(coov20(64, (3, 3), activatione'relu")) model all(MoxPooling20(pool_s1ze=(2, 2))) [Link](Flattent)) |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol oneoraz, 105A rode -nal(2ens0(36)) Iodel1add(oense(3, activetic rode. sumary() model -conptte(los history © model1 f2t(x_tratn_sanpleé, y_erain_sanpl print(classification_report(y_valid, y_pred_seol)) print confusion natrin(y_true-y valid, y_pred-y pres bool)) pie pie pie |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol activation (Activation) ax_pooling2 (MaxPookt (con20) 2% pooling?_1 (HaxPoo! %) ‘atten (Flatten) dense (Dense) ense_a (Dense) ‘rainable parans: 3,973, Non-trainable parans: @ Epoch 1/7 158/158 [ Epoch 2/7 158/158 [ Eooch 3/7 158/158 [ fpoch 4/7 159/358 [ Epoch 5/7 158/358 [ Epoch 6/7 158/258 [ Epoch 7/7 158/158 [ oe soft ” ‘wore, Bea, 4, ea) (Wore, 254, 254, 64) rg20 (None, 127, 127, 64) (Wore, 125, 125, 68) 149g (Wore, 62, 62, 68) (one, 245616) (Wore, 16) (Wore, 3) spool = [Link](y_pred, axis2) red = [Link](XLvalid, verbose=t) batch, 2705 2/9809 = 265 ss/step Support 919 | nacre ove 9.97 0.08 weighted ave 8.99.09, [29 8 4) [eae [ 2 0:07 plot([Link][" accuracy", Isbel=""ratn’) plot(his fite( Model Accuracy") label ("accuracy") label "Fpocn") Tegend) shout) plot(his ry history| loss", labelsTrain") plot (history nistory[val_loss"], Labels valsdation‘) 2s/step asistep 2s/step asistep ae/step as/step Lung_Cancer_Detection ipynb - Colab toss ry-history|'val_accuracy', 1abel= validation’) securacy'}) epochse7, validation catae(K valid, y_valid)) accuracy += volLloss val_accuracy val accuracy sore019128, + 05 AM [Link]( Model Lose") ple [Link](epoen") pits legenat) Lung_Cancer_Detection ipynb - Colab = Model Accuracy & oss ae ile tach Mode Loss ia Gaon tach roses « sautint not. acon(t, (3), Ip shige a sae) meiner nt etd. aon, (2,9), meetin’ rl’) Teri tatnelinasgeat atest) ott. aerinean) rates sete sms thors a ao % |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol wwe01924, [Link] Lung_Cancer_Detection ipynb - Colab "ax pooling2d_3 (MaxPooling (Hone, €2, 62, 64) 8 2) ‘latten.s (Fatten) (one, 246016) ° dense_2 (Dense) (Wore, 16) sereama ense_3 (Dense) (one, 3) sa otal params: 2,973,692 Trainable porans: 3,973,891 Non-trainable parans: 8 [Link](losse’sparse_categorical_crossentrony', optinizers'adan’, natricso{"accuracy'}) rew_wesghts = { 8: X train. shapela]/(3*counter(y_trein}(e]), A: [Link][9]/(3*counter(y_train)I]), 2: XLtrain.snapef9]/(3*counten(y_train)(2]), > 9 now vesghts(e) = 0.5 4 noccnesghts(t] = 20 ew weiehts| @: [Link], 1: 0,6523809523809524, 25 @.8782051282051282) Matory = model? fsttaLtenin, y_train, baten_sizest, epocnse7, velidation_esta-(4 valid, y_velia), class wetentonewvetenss) pooh 3/7 183/103 [ = ths 2e/step - Loss: 0.6302 = accuracy: 0.7616 = Epoch 2/7 163/203 [ = 284s 2/step - loss: 0.0730 - accuracy: 0.9781 = Epoch 3/7 103/103 [onesee = 2824 24/step ~ Loss: @.0640 ~ accuracy: 0.9927 Epoch 4/7 163/103 [ = 176s 2s/step ~ loss + accuracy: 0.9964 - fooch 5/7 1037103 [ = 274s De/step Loss: 0.0386 = accuracy: 0.9999 = + ab3s 2e/step + lose: 0.0075 = accuracy: 0.9988 = = 2875 24/step ~ Loss: 0.4296 - accuracy: 0.9606 - Yared = [Link](X valid, verbose-1) Yipred bool = npcargnax(yapred, a¥ise3) print(classification_report(y_valid, y_pred_seo1)) print(confusion natrin(y_true-y_valid, y_predey_preé_bool)) 9/3 = 355 As/step ‘ftiscore support nacro avg 0.97 ea 8 weignted 2B e995 29 8 a) [ese 3) Tz 3y [Link] (history nistory[accuracy"], Iabel= Train’) pltiplot([Link]['val_accuracy'], label='Validation’) ple-tithet Mosel accuracy") [Link]( accuracy") ple adaben( Epoch") it tegend() me |[Link] [Link]/drivelbOpsq870W-ELoOKDP7VOCdBqSoxgMuEWscrol voile val_loss vol_loss val_toss valLloss 0.2296 006 val accuracy val_accuracy val_ accuracy val accuracy val_sccuraey 0.9055 0.9782 rane