MODEL REGULARIZATION Optimization is basically divided into two. Removing the variables that cause multiple linear regression or are unnecessary in estimating the target variable. The estimation parameters of the feature variables are weighted according to their importance in estimating the target variable, and in this way, the complexity of the model is balanced between error due to the bias of the model and error due to its variance. The technique is called regularization. The first of the mentioned regularization techniques is L1 and the second is L2. 1-) Lasso Regression The process of regularization the weights of the features towards zero using Lasso L1 regularization is called penalization or shrinkage. Cost Function -_yr __ f.)2 RSS = jel (yj — 95) n: number of observations Penalty Term D1 Yh -1 185 p: number of features Lasso L1 regulation cost function RSSzasso = Dj (¥j — 93)? + Lh |B import mathimport numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns from sklearn.fodel_selection import train_test_split,GridSearchCv, cross_val_score from sklearn.linear_model import Lasso, Ridge, ElasticNet,, LinearRegression from sklearn.metrics import r2_score,mean_squared_error,mean_absolute_error df = pd-read_csv(r'D: \githubProjects\Machine-Learning\Supervised Learning\Model Regu df.head() Unnamed:0 price rooms m2 floor age ° o 5 14 0 6 1 1475-1 58S 2 2 4c 91 50 0 7 3 2450158 é 4 44 1 4 207 df.drop(‘unnaned: 0° ,axi df.shape (a1, 5) HOLD-OUT df.drop( ‘price’ ,axis=1) df['price’] y X train, X test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42) Lasso Model lasso = Lasso(randon_state=42) lasso. fit(X_train,y train) Lasso(random_state=42) In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org. train_score = lasso.score(X_train,y_train) test_score = lasso.score(X_test,y_test) coef¥_used = np.sum(1asso.coef_!print("Train R2 :',train_score) print(‘Test R2 :',test_score) print("Number of features with a weight greater than zero :',coeff_used) Train R2 : @.7324650909211012 Test R2 : @.6862906745619393 Nunber of features with a weight greater than zero : 4 The alpha parameter was used as 1. Now let's look at the model performance for different alpha values. alpha = 0.01 1ass0001 = Lasso(alpha=0.61,randon_state=42,max_iter=1000000) Lasso0et.Fit(x_train,y_train) train_score9o1 = lassode1.score(Xx_train,y_train) test_score@@1 = lassoae1.score(x_test,y test) coe##_used@@1 = np.sun(1asso9e1.coe#_ !=@) print("Train R2 (alpha = @.01) :',train_score9e1) print("Test R2 (alpha = @.01) :',test_score@e1) print("Nunber of features with a weight greater than zero (alpha = @.01) :',coeff_us Train R2 (alpha = 8.01) : 0.7337398316689663 Test R2 (alpha = 0.01) : 0.6986142688656938 Nunber of features with a weight greater than zero (alpha = 0.01) : 4 alpha = 0.001 1ass00eee1 = Lasso(alpha=0.0001,randon_state=42,max_iter=1000000) ‘asso9eee1. Fit (x_train,y_train) train_score90ee1 = 1ass00001.score(x_train,y train) test_score9ee1 = [email protected](x_test,y test) coet#_used@0001 = np.sun(1ass0ee0e1.coef_ !=@) print('Train R2 (alpha = 9.001) :",train_scoreae@e1) print("Test R2 (alpha = @.001) :",test_scoreeeeet) print("Nunber of features with a weight greater than zero (alpha = @.001) :*,coeff_. Train R2 (alpha = 0.001) : @.7337399616438325 Test R2 (alpha = 0.001) : 0.6987270960928149 Nunber of features with a weight greater than zero (alpha = 0.001) : 4 Comparing the results with multiple linear regression In = LinearRegression() -Fit(X_train,y_train) Lp_train_score = Ir.score(X_train,y_train) Ip_test_score = In.score(X_test,y test)print('Train R2 :',1r_train_score) print("Test R2 :',1r_test_score) Train R2 : @.733739961656849 Test pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. pit. plt. pit. plt. R2 : 0.6987282352837991 Figure (Figsize=(10,5)) suoplot (1,2, 1) plot((1,2,3,4),1r.coef_,alpha=0.7, linestyle='none’ ,marker='o" ,markersize=15,colc plot((1,2,3,4),1asso.coef_,alpha=1, linestyle='none’ ,marker="x' ,markersize=10, col plot(,1r.intercept_,alpha=0.7,1inestyle='none’ ,marker='0" ,markersize=15,color=| plot(0,1asso.intercept_,alpha=1,1inestyle='none’ ,marker='x' ,markersize=10, color= xticks([@,1,2,3,4], (‘constant term’, ‘rooms’, 'm2", 'floor', 'age"), rotation='70') xlabel('Features ', fontsize=13) ylabel('Features coef’ ,fontsize=13) Legend (fontsize=13, loc=" upper right") subplot (1,2, 2) plot((1,2,3,4),1r.coef_,[email protected],1inestyle='none’ ,marker='0" ,markersize=15,colc plot((1,2,3,4),[email protected]_,alpha~1, linestyl none’ ,marker="+' markersize=1 plot(0,1r.intercept_,alpha=0.7,1inestyle='none’ ,marker='0' ,markersize=15, color= plot (0, 12ss000001. intercept_,alpha=1, linestyle="none’ ,marker="x' ,markersiz xticks([0,1,2,3,4], (‘constant tern’, ‘roons' ,'n2' ,"Floor',, 'age"), rotatio xlabel('Features ',fontsize=13) ylabel( ‘Features coef" , fontsize=13) Legend (fontsize=13, loc=" upper right") show()ao Q @ LnearReg = a ® © LinearReg X Lassoa=1 + Lassoa=0.001 == == Ey x Ben . ° e ® g e © @ fp ot # 8 F fF Pee Features Features ‘As the alpha value approaches zero, the estimation parameters of the model created with Lasso approach the estimation parameters of the multiple linear regression. As the jicient increases, the narrowing in some parameter coefficients becomes L1 penalty coef visible. lasso. coef_ array([103.4072838 , 1.61924751, 7.08642243, | -4.05517555]) ass00eee1.coef_ array([118.34915944, 1.13314119, 7,09688539, -4.66564465]) An.coef_ array([118.35066956, 1.13309207, 7.09688622, | -4.66570638]) 2-)Ridge Regression The Ridge algorithm, on the other hand, aims to reduce the cost of the model by applying the L2 regularization technique. While estimating the feature coefficients, it weights the features according to their importance in estimating the target variable, as in Lasso/L1.It does not exclude any feature.RSSpipce Linear Model I= LinearRegression().fit(X train,y_train) Ridge Model pr = Ridge([email protected],randon_state=42).fit(X_train,y train) # L2 = alpha = 0.1 pri@9 =Ridge(alpha=100, randon_state=42).fit(X_train,y train) # 12 = alpha = 168 train_score = Ir.score(X_train,y_train) test_score = Ir. score(x_test,y test) alpha = 0.01 Ridge R2 ridge_train_score = rr.score(X_train,y train) pidge test_score = rr.score(x_test,y test) alpha = 100 Ridge R2 ridge_train_scorei@@ = rri100.score(x_train,y_train) ridge_test_score1@0= rr1@@.score(X_test,y_test) print("Train R2 Lin.Reg : ',train_score) print('Test R2 Lin.Reg : ',test_score) print('Train R2 Ridge alpha = 0.01 :',ridge_train_score) print('Test R2 Ridge alpha = 0.@1 :',ridge test_score) print("Train R2 Ridge alpha = 108 :',ridge train_scoreiea) print("Test R2 Ridge alpha = 102 :',ridge test_score1ee) Train R2 Lin.Reg : 0.733739961656849 Test R2 Lin.Reg : 0.6987282352837991 Train R2 Ridge alpha = 0.01 : 0.7337398851682211 Test R2 Ridge alpha = 0.01 : 0.698643301640363, Train R2 Ridge alpha = 100 : 0.6682250934479661 Test R2 Ridge alpha = 100 : 0.558431182301198¢ plt. Figure(Figsize=(20,5)) plt.subplot(1,2,1) Plt. plot ((1,2,3,4),1r.coef_,alpha=t .7,linestyli fone” , marker markersize=15,colc Plt. plot ((1,2,3,4),rr.coef_,alpha= Linestyle="none' ,marker="x' ,markersize=1@, color= none" , marker plt.plot (@, Ir. intercept_,alpha=0.7, Linesty: 0" markersize=15, color:plt. plot (@,rr.intercept_,alpha=1, Linestyl none" marke x! ymarkersize=10, color="b] plt.xticks([0,2,2,3,4], (‘constant tern’, rooms’, 'm2", "floor', ‘age"),rotation='70") plt.xlabel('Features ',fontsize=13) plt.ylabel('Features coef, fontsize=13) Plt. legend fontsize=13, 10 plt.suoplot(1,2,2) plt.plot ((1,2,3,4),1r.coef_,alpha= upper right") .7,Linestyli markersize=15,colc pt. plot ((1,2,3,4),[email protected]_,alpha=1, inestyle="none’ ,marker='+' ,markersize=10, col plt. plot (@, Ir. intercept_,alpha=0.7, Linesty! plt. plot (@,rr190.intercept_,alpha=1, Linestyle="none’ ,marker="x" ,markersiz plt.xticks([0,1,2,3,4], (‘constant tern’, “rooms’,'m2", "floor', ‘age"),rotatio plt.xlabel('Features ',fontsize=13) plt.ylabel('Features coef", fontsize=13) Plt. legend fontsize=13, 10 plt.show() x0 © Linear Reg ee _ 3 ® ° ® & é 0, f Features “upper right") none" , marker ‘o" markersize=15,color=" 10, color= 70") @ Linear Reg + Ridge a= 100 Features coef a e 0 o + @ + f fg & § 8 gf of Features {As in Lasso, the Ridge prediction parameters narrow as the alpha increases from 0 to 100. There is a significant narro\ 19 in both the Lasso and Ridge model parameters constant term and number of rooms coefficients. This makes us think that the number of rooms feature may be the source of overimitation or ovet ing3-)Elastic-Net Regression Elastic-Net is an algorithm that combines the features of Lasso and Ridge regressions and aims to minimize the cost function. It uses both L1 and L2 regularization techniques to penalize the model so that it does not make mistakes. RSS EuasticNet = +. 1(Y5 ~ 9i)° io 1 — 7: L2 ratio L: alpha: L146 enet = ElasticNet (alpha=1,random_state=42)..fit(X_train,y_train) train_score = enet.score(x_train,y_train) test_score = enet.score(X_test,y_test) coeff_used = np.sum(enet.coef_ !=0) print("Train R2 alpha strain_score) print("Test 82 alpha = 1:',test_score) print("Nunber of features with 2 weight greater than zero (alpha 1) :",coeff_used) Train R2 alpha = 1 : 0.670871407649752 Test R2 alpha = 1: 0.5627894869336407 Nunber of features with a weight greater than zero (alpha = 1) : 4 alpha = 0.01 enetoo1 = ElasticNet(alpha=0.01,randon_state=42).fit(x_train,y train) train_scoredo1 = enetde1.score(x_train,y_train) test_scorede = eneteei.score(X_test,y test) coef¥_useda@1 = np.sum(enetoa.coef_ print(*Train R2 aplha = @.01:",train_score0@2) print('Test R2 aplha = @.01:',test_score@e1) print("Nunber of features with a weight greater than zero (alpha = @.01) :',coeff_us Train R2 aplha = 0.01: 0.7333501064823666 Test R2 aplha = 0.01: 0.6923107858960038 Nunber of features with a weight greater than zero (alpha = 0.01) : 4 alpha = 100 enetiee = ElasticNet (alpha= 100, randon_state=42).fit(x_train,y train) train_score1@e = enetiae.score(x_train,y_train)test_score100 coeff_used100 eneti99.score(x_test, np. sum(enet108.coef_ test) print('Train R2 aplha = 100:',train_score1@a) print('Test R2 aplha = 1@0:',test_score1ea) print("Nunber of features with a weight greater than zero (alpha = 100) :",coeff_use Train R2 aplha = 100: 0.5844501052872215, Test R2 aplha = 100: 0.47748967369285467 Nunber of features with a weight greater than zero (alpha = 100) : 3 For alpha = 100, the coefficient of one of the Elastic Net model features has become zero and the number of features has decreased to 3, but all features are still used. Linear Model In = LinearRegression(). Fit(X_train,y_train) Ir_train_score Ir_test_score In.score(X_train,y_train) Ir. score(x_test,y test) print("Train R2 :',1ntrain_score) print(*Test R2 :',1n_test_score) Train R2 : @.733739961656849 Test R2 : 0,6987282352837991 plt. Figure(Figsize=(10,5)) plt.subplot(1,2,1) plt.plot((1,2,3,4),1r-coef_,alpha=0.7,1inestyle=' none’ ynarker='o' ,markersize= plt.plot ((1,2,3,4) enetee1.coef_,alph: »Linestyle="none’ ,marker="x" ,markersizé -7,Linestyle='none’ marker 0" markersize=15, colo plt.plot(@,1r-intercept_,alph plt.plot (2, enetoe1. intercept_, alph Linestyle="none’ marke + jnarkersize=10, cole pit xticks([0,1,2,3,4], (‘constant term’, ‘rooms’, 'm2", "floor, ‘age’),rotation="70') plt.xlabel('Features ',fontsize=13) plt.ylabel('Features coef" , fontsize=13) plt.legend(fontsize=13,loc="upper right") plt.suplot(1,2,2) plt.plot ((1,2,3,4),1r-coef_,[email protected],1inestyle='none’ ,marker="o' ,markersize= plt.plot((1,2,3,4) ,enet100.coef_,alpha=1, Linestyle="none' ,marker="+' ,markersiz plt.plot (@, Ir.intercept_,[email protected], Linestyle='none’ marke 0" markersize=15, colo plt.plot (@, enet1e0. intercept_, alpph Linestyle="none’ ,marke x! ymarkersiz 2, coleplt.xticks([0,1,2,3,4], (‘constant tern’, rooms’, 'm2", "floor', ‘age"),rotation='70") plt.xlabel('Features ',fontsize=13) plt.ylabel('Features coef", fontsize=13) plt.legend(fontsize=13,1oc="upper right") plt.show() ao ® os = @ © rear Ro X _ ElasticNet ¢=0.01 < + ElasticNet a= 100 20 x = = 3 3 8 0 80 2 ® Bw = 2 » ° @® ®@ @ . + @&€ @ & fof 8 8 fo % ¢ 8 Features Features enetiee.intercept_ 281, 3000462397627 enet10e_coef = pd.DataFrane(enet1@@.coef_,X.colunns,colunns=["ElasticNet (alpha=100) enet10e_coef ElasticNet (alpha=100) Features coef rooms 0.000000 m2 4.233598 floor 927408 age 0.114527 A practical method that gives the best results by trying alternative values for hyperparameters is the Grid Searchtechnique. So let's do it practically instead of trying it one by one by hand. 1-)Hyper Parameter Optimization with GridSearch for Lasso lasso Lasso(randon_state=42).fit(X_train,y_train) params = ‘alpha’ : [0.@20000000000001, @.0000000001, 0.00000001, 2.0001, 0.001, 0.¢ 1asso_6S = GridSearchCV(estimator=1asso, paran_grid=parans, cv=5,scoring="r2").Fit(Xt C:\Programbata\Anaconda3\envs\batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: ConvergenceWiarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.909e+05, tolerance: 1.557e+02 model = cd_fast.enet_coordinate_descent( C:\ProgranData\Anaconda3\envs\batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: Convergenceliarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 2.139e+05, tolerance: 1.481e+02 model = cd_fast.enet_coordinate_descent( C:\ProgranData\Anaconda3\envs \batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: Convergenceliarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.829e+05, tolerance: 1.425e+02 model = cd_fast.enet_coordinate_descent( C:\ProgramData\Anaconda3\envs \batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: ConvergenceWiarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.380e+05, tolerance: 1.295e+02 model = cd_fast.enet_coordinate_descent( C:\ProgranData\Anaconda3\envs\batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: Convergenceliarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.579e+05, tolerance: 1.410e+02 model = cd_fast.enet_coordinate_descent( Lasso_GS.best_params_ ('alpha': 1} lasso_final = lasso.set_parans(**1asso_6S.best_params_).fit(x_train,y train) lasso_final.score(x_test,y_test) 2. 6862906745619393CROS-VALIDATION all_accuracies = cross_val_score(1asso_final,X=X_train,y=y_train, cv=5) all_accuracies.mean() 2. 7070875030803956 lasso_final. intercept_ 290, 7383188226408 pd.DataFrame(1asso_final.coef_, X.columns, colunns=['Lasso (alphae1) features coef" ] Lasso (alpha=1) features coef rooms 103407284 m2 1.619248 floor 7.086422 age 405517 print(*Train R2 :',1asso_final.score(X_train,y_teain)) print(*Test R2 :',lasso_final.score(x_test,y_test)) Train R2 : @.7324650909211012 Test R2 : 0,6862906745619393 df['price_pred’] = lasso_final.predict(x) Gf[['price’, ‘price_pred’ J] head() price price pred 0 475 434584450 1475 462928338 2 450 446721749 3 45 465959585 4475 452.798357 Let's look at the real prices and estimated prices of the houses with a graph plt. Figure( figsize=(10,6)) pltxticks(df[ ‘price'],df.index.values) # Location, Labels plt plot (df['price'],‘g*', 1abel='real') plt .xticks (dfl 'price_pred' ],df. index. values) plt.plot(df[ 'price_pred’], ro’ ,label="prediction’ )plt.xlabel(‘houses for sale’, fontsize=15) plt.ylabel('pred/real' ,fontsize=15) Plt. legend fontsize=13,1oc="lower right") plt.show() ‘09 predireal g oye - SoMa IE aa Tale o a aA Ea enor Mita a0 es ea we prediction houses for sale example roons = 3 ua 7 tes floor = 4 ee lasso_final.predict([{rooms,m2, floor, age]]) C:\ProgranData\Anaconda3\envs\batuhan\1ib\site-packages\sklearn\base.py:45@: UserWar ning: X does not have valid feature names, but Lasso was fitted with feature names warnings.warn( array ([766.88544391]) a Hyper Parameter Optimization with GridSearch for Ridge ridge = Ridge(randon_state=42) params = {‘alpha’ : [0.0a8eeeaee0e0001, 0.0000000001, 0.00000001, 0.0001, 8.001, 0.¢ ridge_GS = GridsearchCV(ridge, parans, scoring='r2",cv=5) fit(X_train,y train) print (ridge_GS-best_parans_) print (ridge GS. best_estimator_) print (ridge GS-best_score_){'alpha': 1) Ridge(alpha=1, randon_state=42) 9.7071772649347372 all_accuracies = cross_val_score(ridge_6S.best_estimator_,X=x_train, /_train, ev=5) print(all_accuracies) print(all_accuracies.mean()) print(all_accuracies.std()) [0.59028756 086202159 @.75828323 0.58997173 @.73532222] 8.7071772649347372 2, 10467192818729874 ridge_final = ridge.set_parans(**ridge_GS.best_parans_).fit(x_teain,y train) pd. DataFrame(ridge_final.coef_,X.colunns, columns=[ ‘Ridge (alpha~1) features coef" ]) Ridge (alpha=1) features coef rooms 107776167 m2 1478146 floor 7.135185 age 4.234666 ridge_train_score = ridge_final.score(x_train,y_train) ridge_test_score = ridge final.score(x_test,y test) print("Test R2 :',ridge_test_score) print(‘Train R2 :',ridge_train_score) Test R2 : 0,6904116932614515 Train R2 : @.7331046122177594 d€['price_pred’] = ridge final.predict(x) df['price_pred’ ].head() 2 436.204755 1 462.611617 2 446.751552 3 465.512136 4 453.631191 Name: price_pred, dtype: floatea pit. Figure( Figsize=(10,6)) plt.xticks(df[ 'price'],df.index.values) # Location, Labels Plt. plot (4f["price'],‘g*',label="real") pt. xticks (dl ‘price_pred'], df. index. values) pit. plot (4f['price_pred’], ‘ro’ , label=" prediction’) plt.xlabel("houses for sale’ ,fontsize=15) plt.ylabel(' pred/real' ,fontsize=15)Plt. legend fontsize=13, 10 lower right") plt.show() 5 ‘ Z ad g v 3” Perec oe g een ashes 1 Senta Pte hearer a ia houses for sale & ridge_final.predict({([roons 2, floor ,age]]) C:\ProgranData\Anaconda3 \envs\batuhan\ 1ib\site-packages\sklearn\base. py:450: UserWar ning: X does not have valid feature names, but Ridge was fitted with feature names warnings.warn( array ([767.90801287]) 2 Hyper Parameter Optimization with GridSearch for ElasticNet enet = ElasticNet (random_state=42) params = ‘alpha’ ; [0.eeee09000ee@001, @.0000000001, 0.00000001, 0.0001, 0.001, 0.¢ enet_GS = GridSearchcV(enet, parans, scorin r2",evs5). Fit(X_train,y_train) print (enet_GS.best_parans_) print(enet_GS.best_estinator_) print(enet_GS.best_score_) C:\ProgramData\ Anaconda3 \envs \batuhan\1ib\site-packages\sklearn\linear_nodel\_coordi nate_descent.py:648: ConvergenceWarning: Objective did not converge. You might wantto increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.947e+05, tolerance: 1.557e+82 model = cd_fast.enet_coordinate_descent( C:\ProgramData\anaconda3 \envs \batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 2.119e+@5, tolerance: 1.481¢+82 model = cd_fast.enet_coordinate_descent( C:\ProgramData\anaconda3 \envs \batuhan\ 1ib\site-packages\sklearn\linear_nodel\_coordi nate_descent.py:648: ConvergenceWiarning: Objective did not converge. You might want to increase the nunber of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.948e+05, tolerance: 1.425e+82 model = cd_fast.enet_coordinate_descent( C:\ProgranData\ Anaconda3 \envs \batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.398e+05, tolerance: 1.295e+82 model = cd_fast.enet_coordinate_descent( C:\ProgramData\ Anaconda3 \envs \batuhan\1ib\site-packages\sklearn\linear_model\_coordi nate_descent.py:648: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider in creasing regularisation. Duality gap: 1.878e+05, tolerance: 1.410e+82 model = cd_fast.enet_coordinate_descent( 0.01) asticNet (alpha=t 9.7873610834707613 .01, random_state=42) enet_GS-best_params_ {(‘alpha": 0.01) enet_GS-best_estimator_ ElasticNet(alpha=0.01, random_state=42) Ina Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook, On GitHub, the HTML representation is unable to render, please try loading this page with nbviewerorg. enet_GS-best_score_ 2.7073610834707613 all_accuracies = cross_val_score(enet_GS.best_estimator_,X=X_train,y=y_train,cv=5) print(all_accuracies) print(all_accuracies.mean()) print(all_accuracies.std()) [@.58966797 0.86205089 0.75871268 @.59061914 0.73575473] 9.7873610834707613 9.10474012760014134enet_GS.best_estimator_. intercept_ 296,9768160015012 enet_final = enet.set_parans(**enet_GS.best_parans_).fit(X_train,y_train) pd. bataFrane(enet_final.coef_,x.columns, colunns=['ElasticNet ([email protected]) features ¢ HlasticNet (alpha=0.01) features coef rooms 110087357 m2 1.403373 floor 7.126647 age -4328035 enet_train_score enet_test_score enet_final.score(x_train,y train) enet_final.score(x_test,y test) print("Test R2 :',enet_test_score) print('Train R2 :',enet_train_score) Test R2 : ,6923107858960038 Train R2 : @.7333501064823666 df[‘price_pred’] = enet_final.predict(X) df['price_pred’ ].head() 2 437.210859 1 462.589482 2 4a6.916549 3 465. 388095, 4 4sa.152981 Ne jane: price_pred, dtype: floated plt. Figure(figsize=(10,6)) plt.xticks(df[ ‘price’ ],df-index.values) # Location, Labels plt.plot (df[ ‘price'], 'g*',label='real') plt.xticks(dF{ ‘price pred’ ] df. index. values) plt.plot (df[ ‘price_pred' ], ‘ro’ ,label=' prediction’ ) plt-xlabel( ‘houses for sale’ ,fontsize=15) plt.ylabel( 'pred/real' ,fontsize=15) plt.legend(fontsize=13,1oc=' lower right’) pit. show()‘09 predireal g wa © rodiction houses for sale example rooms = 3 m2 = 105 Floor = 4 age = enet_final.predict ([[rooms,n2, floor, age] }) C:\ProgranData\anaconda3 \envs\batuhan\ 1ib\site-packages\sklearn\base. py:450: UserWar ning: X does not have valid feature names, but ElasticNet was fitted with feature na nes warnings. warn( array [768.41529786])