07/10/13
libsvm for MATLAB - Kittipat's Homepage
Kittipat'sHomepage
Pesquisarosite
Home
Relevantprojectsshowroom
Academics
Announcements Home Botinanutshell Resume(ShortCV) CV MVPAandMachineLearningfor BrainfMRI fMRIpreprocessMATLAB toolboxversion1.0 Convertdatabetween MATLABandNiftiformat MutualInformationbased FeatureRankingforMVPAof fMRIbraindata BrainFunctional DistinguishabilityMeasure simpleclassificationonfMRI data SimulatedAnnealing(SA)for VoxelsSelection Classspecificmeasures:CMI andCD functionalbrainclustering biclusteringalgorithmon categorysensitivedescriptor 3Dsupervoxelforbrainimage sparseregularizationon lexicaldata ClassificationwithSparse Regularization Relevantprojectsshowroom EMalgorithmforGaussian mixturemodelongridded space logisticregressionforfeature selection statichandposture recognition ImageSegmentationusing GMMBIC 3DLiDARpointcloud segmentation IrregularTreeStructured BayesianNetworks Classes Usefulstuff Toolsfordatascience libsvmforMATLAB Hierarchicalbiclustering toolbox SpectralCoClustering (Biclustering)algorithm informationtheoreticlearning (ITL)MATLABtoolbox OftenusedLaTeXtechniques OftenusedMATLAB Techniques OftenusedRtechniques ProgrammingwithPython Downloadavailablecodesand toolboxes Sitemap Recentsiteactivity
libsvmforMATLAB
7
libsvmisagreattoolforSVMasitisveryeasytouseandisdocumentedwell.Thelibsvmpackagewebpageismaintained byChihChungChangandChihJenLinofNTU.Thewebpagecanbefoundhere.Imadethistutorialasareminderfor myselfwhenIneedtouseitagain.Allthecreditsgoforthelibsvmdevelopers.Hereishowyoucancitethelibsvm.
Content
Inthisshorttutorial,thefollowingtopicswillbediscussed: HowtoinstallthelibsvmforMATLABonUnixmachine LinearkernelSVMforbinaryclassification kernelSVMforbinaryclassification crossvalidationforCandGamma multiclassSVM:onevsrest(OVR) Morereadytousematlabexample Availablematlabcodestodownload
Hereishowtoinstallthetoolbox
Justreadthereadmefileinthepackage.It'sveryeasy.YoucandoitinbothterminalandinMATLABworkspace.On Ubuntumachine,justtomakesureyouhavegccinyourmachine.Ifnot,youneedtoinstallitusingthecommandbelow: s u d oa p t g e ti n s t a l lb u i l d e s s e n t i a lg + +
BasicSVM:LinearkernelSVMforbinaryclassification
Belowisthefirstcodetorun.Thecodeisforbinaryclassificationandusethevariablec=1,gamma(g)=0.07and'b1' denotestheprobabilityoutput. %T h i sc o d ej u s ts i m p l yr u nt h eS V Mo nt h ee x a m p l ed a t as e t" h e a r t _ s c a l e " , %w h i c hi ss c a l e dp r o p e r l y .T h ec o d ed i v i d e st h ed a t ai n t o2p a r t s %t r a i n :1t o2 0 0 %t e s t :2 0 1 : 2 7 0 %T h e np l o tt h er e s u l t sv st h e i rt r u ec l a s s .I no r d e rt ov i s u a l i z et h eh i g h %d i m e n s i o n a ld a t a ,w ea p p l yM D St ot h e1 3 Dd a t aa n dr e d u c et h ed i m e n s i o n %t o2 D c l e a r c l c c l o s ea l l %a d d p a t ht ot h el i b s v mt o o l b o x a d d p a t h ( ' . . / l i b s v m 3 . 1 2 / m a t l a b ' ) ; %a d d p a t ht ot h ed a t a d i r D a t a=' . . / l i b s v m 3 . 1 2 ' ; a d d p a t h ( d i r D a t a ) ; %r e a dt h ed a t as e t [ h e a r t _ s c a l e _ l a b e l ,h e a r t _ s c a l e _ i n s t ]=l i b s v m r e a d ( f u l l f i l e ( d i r D a t a , ' h e a r t _ s c a l e ' ) ) ; [ ND ]=s i z e ( h e a r t _ s c a l e _ i n s t ) ; %D e t e r m i n et h et r a i na n dt e s ti n d e x t r a i n I n d e x=z e r o s ( N , 1 ) ;t r a i n I n d e x ( 1 : 2 0 0 )=1 ; t e s t I n d e x=z e r o s ( N , 1 ) ;t e s t I n d e x ( 2 0 1 : N )=1 ; t r a i n D a t a=h e a r t _ s c a l e _ i n s t ( t r a i n I n d e x = = 1 , : ) ; t r a i n L a b e l=h e a r t _ s c a l e _ l a b e l ( t r a i n I n d e x = = 1 , : ) ; t e s t D a t a=h e a r t _ s c a l e _ i n s t ( t e s t I n d e x = = 1 , : ) ; t e s t L a b e l=h e a r t _ s c a l e _ l a b e l ( t e s t I n d e x = = 1 , : ) ; %T r a i nt h eS V M m o d e l=s v m t r a i n ( t r a i n L a b e l ,t r a i n D a t a ,' c1g0 . 0 7b1 ' ) ; %U s et h eS V Mm o d e lt oc l a s s i f yt h ed a t a [ p r e d i c t _ l a b e l ,a c c u r a c y ,p r o b _ v a l u e s ]=s v m p r e d i c t ( t e s t L a b e l ,t e s t D a t a ,m o d e l ,' b1 ' ) ; %r u nt h eS V Mm o d e lo nt h et e s td a t a
Boringitems
Bot'sBibTex Readinglist ResearchWriting Bot'sSillyTools WhiteBorderRemoval Toolbox
%= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = %= = = = =S h o w i n gt h er e s u l t s= = = = = = %= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = %A s s i g nc o l o rf o re a c hc l a s s %c o l o r L i s t=g e n e r a t e C o l o r L i s t ( 2 ) ; % T h i si sm yo w nw a yt oa s s i g nt h ec o l o r . . . d o n ' t w o r r ya b o u ti t c o l o r L i s t=p r i s m ( 1 0 0 ) ; %t r u e( g r o u n dt r u t h )c l a s s t r u e C l a s s I n d e x=z e r o s ( N , 1 ) ; t r u e C l a s s I n d e x ( h e a r t _ s c a l e _ l a b e l = = 1 )=1 ; t r u e C l a s s I n d e x ( h e a r t _ s c a l e _ l a b e l = = 1 )=2 ; c o l o r T r u e C l a s s=c o l o r L i s t ( t r u e C l a s s I n d e x , : ) ; %r e s u l tC l a s s r e s u l t C l a s s I n d e x=z e r o s ( l e n g t h ( p r e d i c t _ l a b e l ) , 1 ) ; r e s u l t C l a s s I n d e x ( p r e d i c t _ l a b e l = = 1 )=1 ; r e s u l t C l a s s I n d e x ( p r e d i c t _ l a b e l = = 1 )=2 ; c o l o r R e s u l t C l a s s=c o l o r L i s t ( r e s u l t C l a s s I n d e x , : ) ; %R e d u c et h ed i m e n s i o nf r o m1 3 Dt o2 D
Favoritewebsites
Machinelearningwebsites Aiqus AISocial(BrenOCon) MLTheory/News(Langford) NLPers
https://sites.google.com/site/kittipat/libsvm_matlab
1/8
07/10/13
ClassCentral Udacity Coursera ProjectEuler Kaggle ChaLearnAcademy Kittipat'sBlog GoogleResearchBlog
libsvm for MATLAB - Kittipat's Homepage
d i s t a n c e M a t r i x=p d i s t ( h e a r t _ s c a l e _ i n s t , ' e u c l i d e a n ' ) ; n e w C o o r=m d s c a l e ( d i s t a n c e M a t r i x , 2 ) ; %P l o tt h ew h o l ed a t as e t x=n e w C o o r ( : , 1 ) ; y=n e w C o o r ( : , 2 ) ; p a t c h S i z e=3 0 ;% m a x ( p r o b _ v a l u e s , [ ] , 2 ) ; c o l o r T r u e C l a s s P l o t=c o l o r T r u e C l a s s ; f i g u r e ;s c a t t e r ( x , y , p a t c h S i z e , c o l o r T r u e C l a s s P l o t , ' f i l l e d ' ) ; t i t l e ( ' w h o l ed a t as e t ' ) ; %P l o tt h et e s td a t a x=n e w C o o r ( t e s t I n d e x = = 1 , 1 ) ; y=n e w C o o r ( t e s t I n d e x = = 1 , 2 ) ; p a t c h S i z e=8 0 * m a x ( p r o b _ v a l u e s , [ ] , 2 ) ; c o l o r T r u e C l a s s P l o t=c o l o r T r u e C l a s s ( t e s t I n d e x = = 1 , : ) ; f i g u r e ;h o l do n ; s c a t t e r ( x , y , 2 * p a t c h S i z e , c o l o r T r u e C l a s s P l o t , ' o ' , ' f i l l e d ' ) ; s c a t t e r ( x , y , p a t c h S i z e , c o l o r R e s u l t C l a s s , ' o ' , ' f i l l e d ' ) ; %P l o tt h et r a i n i n gs e t x=n e w C o o r ( t r a i n I n d e x = = 1 , 1 ) ; y=n e w C o o r ( t r a i n I n d e x = = 1 , 2 ) ; p a t c h S i z e=3 0 ; c o l o r T r u e C l a s s P l o t=c o l o r T r u e C l a s s ( t r a i n I n d e x = = 1 , : ) ; s c a t t e r ( x , y , p a t c h S i z e , c o l o r T r u e C l a s s P l o t , ' o ' ) ; t i t l e ( ' c l a s s i f i c a t i o nr e s u l t s ' ) ; Theresultshows: o p t i m i z a t i o nf i n i s h e d ,# i t e r=1 3 7 n u=0 . 4 5 7 4 2 2 o b j=7 6 . 7 3 0 8 6 7 ,r h o=0 . 4 3 5 2 3 3 n S V=1 0 4 ,n B S V=8 1 T o t a ln S V=1 0 4 A c c u r a c y=8 1 . 4 2 8 6 %( 5 7 / 7 0 )( c l a s s i f i c a t i o n ) Thewholedatasetisplotted:
Theclusteringresultsmightlooklikethis:
Theunfilledmarkersrepresentdatainstancefromthetrainset.Thefilledmarkersrepresentdatainstancefromthetestset, andfilledcolorrepresentstheclasslabelassignedbySVMwhereastheedgecolorrepresentsthetrue(groundtruth)label. Themarkersizeofthetestsetrepresentstheprobabilitythatthesampleinstanceisassignedwithitscorrespondingclass labelthebigger,themoreconfidence.
KernelSVMforbinaryclassification
Nowlet'sapplysomekerneltotheSVM.Weusealmostthesamecodeasbefore,theonlyexceptionisthetraindataset,
https://sites.google.com/site/kittipat/libsvm_matlab
2/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
trainData,isreplacedbythekernelizedversion[ ( 1 : 2 0 0 ) 't r a i n D a t a * t r a i n D a t a ' ] andthetestdata,testData,is replacedbyitskernelizedversion[ ( 1 : 7 0 ) 't e s t D a t a * t r a i n D a t a ' ] asappearedbelow. %T r a i nt h eS V M m o d e l=s v m t r a i n ( t r a i n L a b e l ,[ ( 1 : 2 0 0 ) 't r a i n D a t a * t r a i n D a t a ' ] ,' c1g0 . 0 7b1t4 ' ) ; %U s et h eS V Mm o d e lt oc l a s s i f yt h ed a t a [ p r e d i c t _ l a b e l ,a c c u r a c y ,p r o b _ v a l u e s ]=s v m p r e d i c t ( t e s t L a b e l ,[ ( 1 : 7 0 ) ' t e s t D a t a * t r a i n D a t a ' ] ,m o d e l ,' b1 ' ) ;%r u nt h eS V Mm o d e lo nt h et e s td a t a Thecompletecodecanbefoundhere.Theresultingclustersareshowninthefigurebelow. ' L i n e a r 'k e r n e l o p t i m i z a t i o nf i n i s h e d ,# i t e r=4 0 3 7 9 6 n u=0 . 3 3 5 7 2 0 o b j=6 7 . 0 4 2 7 8 1 ,r h o=1 . 2 5 2 6 0 4 n S V=7 4 ,n B S V=6 0 T o t a ln S V=7 4 A c c u r a c y=8 5 . 7 1 4 3 %( 6 0 / 7 0 )( c l a s s i f i c a t i o n ) ' p o l y n o m i a l 'k e r n e l o p t i m i z a t i o nf i n i s h e d ,# i t e r=1 0 2 3 8 5 n u=0 . 0 0 0 0 0 1 o b j=0 . 0 0 0 0 8 6 ,r h o=0 . 4 6 5 3 4 2 n S V=6 9 ,n B S V=0 T o t a ln S V=6 9 A c c u r a c y=7 2 . 8 5 7 1 %( 5 1 / 7 0 )( c l a s s i f i c a t i o n ) ' R B F 'k e r n e l o p t i m i z a t i o nf i n i s h e d ,# i t e r=3 7 2 n u=0 . 8 9 0 0 0 0 o b j=9 7 . 5 9 4 7 3 0 ,r h o=0 . 1 9 4 4 1 4 n S V=2 0 0 ,n B S V=9 0 T o t a ln S V=2 0 0 A c c u r a c y=5 7 . 1 4 2 9 %( 4 0 / 7 0 )( c l a s s i f i c a t i o n ) ' S i g m o i d 'k e r n e l o p t i m i z a t i o nf i n i s h e d ,# i t e r=9 0 n u=0 . 8 7 0 0 0 0 o b j=1 9 5 . 4 1 7 1 6 9 ,r h o=0 . 9 9 9 9 9 3 n S V=1 7 4 ,n B S V=1 7 4 T o t a ln S V=1 7 4 A c c u r a c y=6 0 %( 4 2 / 7 0 )( c l a s s i f i c a t i o n ) ' M L P 'k e r n e l o p t i m i z a t i o nf i n i s h e d ,# i t e r=1 2 4 7 n u=0 . 3 5 2 6 1 6 o b j=6 8 . 8 4 2 4 2 1 ,r h o=0 . 5 5 2 6 9 3 n S V=7 7 ,n B S V=6 3 T o t a ln S V=7 7 A c c u r a c y=8 2 . 8 5 7 1 %( 5 8 / 7 0 )( c l a s s i f i c a t i o n )
LinearkernelSVM:85.7%accuracy
PolynomialkernelSVM:72.86%accuracy
RBFkernelSVM:57.14%accurac
SigmoidkernelSVM:60%accuracy
MLPkernelSVM:82.86%accuracy
CrossvalidationofCandGamma
Theoptionforsvmtrain n f o l dc r o s sv a l i d a t i o n :nm u s t> =2 U s a g e :m o d e l=s v m t r a i n ( t r a i n i n g _ l a b e l _ v e c t o r ,t r a i n i n g _ i n s t a n c e _ m a t r i x , ' l i b s v m _ o p t i o n s ' ) ; l i b s v m _ o p t i o n s : ss v m _ t y p e:s e tt y p eo fS V M( d e f a u l t0 ) 0-C S V C 1-n u S V C 2-o n e c l a s sS V M 3-e p s i l o n S V R 4-n u S V R tk e r n e l _ t y p e:s e tt y p eo fk e r n e lf u n c t i o n( d e f a u l t2 ) 0-l i n e a r :u ' * v 1-p o l y n o m i a l :( g a m m a * u ' * v+c o e f 0 ) ^ d e g r e e 2-r a d i a lb a s i sf u n c t i o n :e x p ( g a m m a * | u v | ^ 2 )
https://sites.google.com/site/kittipat/libsvm_matlab
3/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
3-s i g m o i d :t a n h ( g a m m a * u ' * v+c o e f 0 ) 4-p r e c o m p u t e dk e r n e l( k e r n e lv a l u e si nt r a i n i n g _ i n s t a n c e _ m a t r i x ) dd e g r e e:s e td e g r e ei nk e r n e lf u n c t i o n( d e f a u l t3 ) gg a m m a:s e tg a m m ai nk e r n e lf u n c t i o n( d e f a u l t1 / n u m _ f e a t u r e s ) rc o e f 0:s e tc o e f 0i nk e r n e lf u n c t i o n( d e f a u l t0 ) cc o s t:s e tt h ep a r a m e t e rCo fC S V C ,e p s i l o n S V R ,a n dn u S V R( d e f a u l t1 ) nn u:s e tt h ep a r a m e t e rn uo fn u S V C ,o n e c l a s sS V M ,a n dn u S V R( d e f a u l t0 . 5 ) pe p s i l o n:s e tt h ee p s i l o ni nl o s sf u n c t i o no fe p s i l o n S V R( d e f a u l t0 . 1 ) mc a c h e s i z e:s e tc a c h em e m o r ys i z ei nM B( d e f a u l t1 0 0 ) ee p s i l o n:s e tt o l e r a n c eo ft e r m i n a t i o nc r i t e r i o n( d e f a u l t0 . 0 0 1 ) hs h r i n k i n g:w h e t h e rt ou s et h es h r i n k i n gh e u r i s t i c s ,0o r1( d e f a u l t1 ) bp r o b a b i l i t y _ e s t i m a t e s:w h e t h e rt ot r a i naS V Co rS V Rm o d e lf o rp r o b a b i l i t y e s t i m a t e s ,0o r1( d e f a u l t0 ) w iw e i g h t:s e tt h ep a r a m e t e rCo fc l a s sit ow e i g h t * C ,f o rC S V C( d e f a u l t1 ) vn:n f o l dc r o s sv a l i d a t i o nm o d e q:q u i e tm o d e( n oo u t p u t s ) Inthisexample,wewillusetheoptionenforcingnfoldcrossvalidationinsvmtrain,whichissimplyputthe'vn'inthe parametersection,wherendenotenfoldcrossvalidation.Hereistheexampleofusing3foldcrossvalidation: p a r a m=[ ' qv3c' ,n u m 2 s t r ( c ) ,'g' ,n u m 2 s t r ( g ) ] ; c v=s v m t r a i n ( t r a i n L a b e l ,t r a i n D a t a ,p a r a m ) ; Intheexamplebelow,Iwillshowthenestedcrossvalidation.First,wesearchfortheoptimalparameters(candgamma)in thebigscale,thenthesearchingspaceisnarroweddownuntilsatisfied.Theresultsarecomparedwiththefirstexperiment whichdoesnotusetheoptimalparameters.Thefullcodecanbefoundhere. Bigscaleparameterssearching Mediumscaleparameterssearching
Smallscaleparameterssearching
Accuracy=84.29%whichisbetterthanusingthenonreallyoptimalparameterc=1and gamma=0.07inthepreviousexperimentwhichgives81.43%accuracy.
MulticlassSVM
Naturally,SVMisabinaryclassificationmodel,howcanweuseSVMinthemulticlassscenario?Inthisexample,wewill showyouhowtodomulticlassclassificationusinglibsvm.Asimplestrategyistodobinaryclassification1pairatatime. Herewewilluseoneversusrestapproach.Infact,wecanjustusetheoriginalcodes(svmtrainandsvmpredict)fromthe libsvmpackagetodothejobbymakinga"wrappercode"tocalltheoriginalcodeonepairatatime.Thegoodnewsisthat libsvmtutorialpageprovidesawrappercodetodosoalready.Yes,wewilljustuseitproperly. JustdownloadthedemocodefromtheendofthisURL,whichsays [ t r a i n Yt r a i n X ]=l i b s v m r e a d ( ' . / d n a . s c a l e ' ) ; [ t e s t Yt e s t X ]=l i b s v m r e a d ( ' . / d n a . s c a l e . t ' ) ; m o d e l=o v r t r a i n ( t r a i n Y ,t r a i n X ,' c8g4 ' ) ; [ p r e da cd e c v ]=o v r p r e d i c t ( t e s t Y ,t e s t X ,m o d e l ) ; f p r i n t f ( ' A c c u r a c y=% g % % \ n ' ,a c*1 0 0 ) ; Thecodesovrtrainandovrpredictarethewrapper.Youcanalsodothecrossvalidationfromthedemocodebelow,where get_cv_acisagainthewrappercode.
b e s t c v=0 ;
https://sites.google.com/site/kittipat/libsvm_matlab
4/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
f o rl o g 2 c=1 : 2 : 3 , f o rl o g 2 g=4 : 2 : 1 , c m d=[ ' qc' ,n u m 2 s t r ( 2 ^ l o g 2 c ) ,'g' ,n u m 2 s t r ( 2 ^ l o g 2 g ) ] ; c v=g e t _ c v _ a c ( t r a i n Y ,t r a i n X ,c m d ,3 ) ; i f( c v> =b e s t c v ) , b e s t c v=c v ;b e s t c=2 ^ l o g 2 c ;b e s t g=2 ^ l o g 2 g ; e n d f p r i n t f ( ' % g% g% g( b e s tc = % g ,g = % g ,r a t e = % g ) \ n ' ,l o g 2 c ,l o g 2 g ,c v ,b e s t c ,b e s t g ,b e s t c v ) ; e n d e n d
Thefullimplementedcodecanbefoundhere.Resultsshowthat
row12000:trainingset.
Theonevsrestmulticlass SVMresults.Herewedo parameterselectiononthe trainsetyieldingthe accuracyforeachclass: class1:Accuracy= 94.3508%(1119/1186) (classification) class2:Accuracy= 95.4469%(1132/1186) (classification) class3:Accuracy= 94.1821%(1117/1186) (classification) overallclass:Accuracy= 94.0135% Thebestparametersare c=8andgamma=0.0625. Notewhentheparameters arenotselectproperly,say c=8,gamma=4,the accuracyisaslowas 60%.So,parameter selectionisreally important!!!!
Moreexamples
Youmayfindthefollowingexamplesuseful.Eachcodeisbuiltforsomespecificapplication,whichmightbeusefultothe readertodownloadandtweakjusttosaveyourdevelopingtime.
Bigpicture:Inthisscenario,Icompiledaneasyexampletoillustratehowtousesvminfullprocess.Thecode contains: datageneration determiningtrainandtestdataset parameterselectionusingnfoldcrossvalidation,bothsemimanualandtheautomaticapproach trainthesvmmodelusingoneversusrest(OVR)approach usethesvmmodeltoclassifythetestsetinOVRmode makeconfusionmatrixtoevaluatetheresults showtheresultsinaninformativeway displaythedecisionboundaryonthefeaturespace Reportingaresultsusingnfoldcrossvalidation:Incaseyouhaveonly1dataset(i.e.,thereisnoexplicittrainor testset),nfoldcrossvalidationisaconventionalwaytoassessaclassifier.Theoverallaccuracyisobtainedby averagingtheaccuracypereachofthenfoldcrossvalidation.Theobservationsareseparatedintonfoldsequally,the codeusen1foldstotrainthesvmmodelwhichwillbeusedtoclassifytheremaining1foldaccordingtostandard OVR.Thecodecanbefoundhere. Usingmulticlassovrsvmwithkernel:SofarIhaven'tshowntheusageofovrsvmwithkernelspecific('tx').In fact,youcanaddthekerneltoanyovrcode,theywillwork.Thecompletecodecanbefoundhere. Forparameterselectionusingcrossvalidation,weusethecodebelowtocalculatetheaverageaccuracycv. Youcanjustadd' tx ' tothecode.
c m d=[ ' qc' ,n u m 2 s t r ( 2 ^ l o g 2 c ) ,'g' ,n u m 2 s t r ( 2 ^ l o g 2 g ) , 't0 ' ] ; c v=g e t _ c v _ a c ( t r a i n L a b e l ,[ ( 1 : N T r a i n ) 't r a i n D a t a * t r a i n D a t a ' ] ,c m d ,N c v ) ;
https://sites.google.com/site/kittipat/libsvm_matlab
5/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
Training:justadd' tx ' tothetrainingcode b e s t P a r a m=[ ' qc' ,n u m 2 s t r ( b e s t c ) ,' ,g' ,n u m 2 s t r ( b e s t g ) , 't0 ' ] ; m o d e l=o v r t r a i n B o t ( t r a i n L a b e l ,[ ( 1 : N T r a i n ) 't r a i n D a t a * t r a i n D a t a ' ] , b e s t P a r a m ) ; Classification:the' tx ' isincludedinthevariablem o d e l already,soyoudon'tneedtospecify' tx ' againwhenclassifying. [ p r e d i c t _ l a b e l ,a c c u r a c y ,d e c i s _ v a l u e s ]=o v r p r e d i c t B o t ( t e s t L a b e l ,[ ( 1 : N T e s t ) ' t e s t D a t a * t r a i n D a t a ' ] ,m o d e l ) ; [ d e c i s _ v a l u e _ w i n n e r ,l a b e l _ o u t ]=m a x ( d e c i s _ v a l u e s , [ ] , 2 ) ; However,Ifoundthatthecodecanbeveryslowinparameterselectionroutinewhenthenumberofclassand thenumberofcrossvalidationarebig(e.g.,Nclass=10,Ncv=3).Ithinktheslowpartmightbecausedby [ ( 1 : N T r a i n ) 't r a i n D a t a * t r a i n D a t a ' ] whichcanbehuge.PersonallyIliketousethedefaultkernel (RBF),whichwedon'tneedtomakethekernelmatrixX*X',whichmightcontributetoaprettyquickspeed. Completeexampleforclassificationusingnfoldcrossvalidation:Thiscodeworksonthesingledatawherethe trainandtestsetarecombinedwithinonesingleset.Moredetailscanbefoundhere. Completeexampleforclassificationusingtrainandtestdatasetseparately:Thiscodeworksonthedataset wherethetrainandtestsetareseparated,thatis,trainthemodelusingtrainsetandusethemodeltoclassifythe testset.Moredetailscanbefoundhere. HowtoobtaintheSVMweightvectorw:PleaseseetheexamplecodeanddiscussionfromStackOverflow.
Listofavailablematlabcodes
code demo_libsvm_test1.m binary/multiclass parameter classification kernel selection separated/n fold binary no, separated default manually (RBF) dataset description
heart_scale Thiscodeshowsthesimple (perhapssimplest)usageofthe svmlibpackagetotrainand classify.Veryeasyto understand. Thiscodejustsimplyrunthe SVMontheexampledataset "heart_scale",whichisscaled properly.Thecodedividesthe datainto2partstrain:1to200 andtest:201:270 Thenplottheresultsvstheirtrue class.Inordertovisualizethe highdimensionaldata,weapply MDStothe13Ddataandreduce thedimensionto2D
demo_libsvm_test2.m binary demo_libsvm_test3.m binary
no, separated manually
semi separated automatic, butthe codeis stillnot compact demo_libsvm_test4.m multiclass,OVR semi separated automatic
Specified heart_scale Identicalto_test1exceptthatit showshowtospecifythekernel (e.g.,'t4')inthecode. default heart_scale Identicalto_test1exceptthatit includearoutinesearchingfor goodparameterscandgamma
default
dna_scale Thiscodeshowshowtousethe libsvmforthemulticlass,more specificallyonevsrest(OVR), scenario.Forbothtrainingand classifying,weadopttheOVR wrappercodespostedinthe libsvmwebsite: 1. ovrtrain.mand 2. ovrpredict.m
demo_libsvm_test5.m multiclass,OVR multi separated scale automatic butnot perfect
default
10class spiral
respectively. Hereboththetrainandtestset aregeneratedfrom10class spiralmadeavailablehere.The datasetisveryintuitive. Inthiscode,wealsomakea routinetodeterminetheoptimal parametersautomatically.The usercanguessaninitial parameter,theroutinewillkeep improvingit. Herewealsomodifytheoriginal trainandclassifyfunctionabit: 1. ovrtrainBot.m< ovrtrain.m 2. ovrpredictBot.m< ovrpredict.m Furthermore,theconfusion matrixisshownintheresults. Wealsoplotthedecisionvalues inthefeaturespacejusttogive anideahowthedecision boundarylookslike.
demo_libsvm_test6.m multiclass,OVR no, leaveone manually outnfold cross validation
default
10class spiral
Inthiscodewewanttoillustrate howtoperformclassification usingnfoldcrossvalidation, whichisacommonmethodology
https://sites.google.com/site/kittipat/libsvm_matlab
6/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
tousewhenthedatasetdoes nothaveexplicittrainingandtest setseparately.Suchdatasets usuallycomeasasinglesetand wewillneedtoseparatethem intonequalparts/folds.The leaveoneoutnfoldcross validationistoclassify observationsinafoldkbyusing themodeltrainedfrom{all}{k} models,andrepeattheprocess forallk. Theuserisrequiredtoseparate thedataintonfoldsbyassigning "run"labelforeachobservation. Theobservationswithidentical runnumberwillbegrouped togetherintoafold.Itisa preferencetohaveobservations fromalltheclasseswithina certainfold.Infact,assigningthe runnumbertoeachobservation randomlyisfineaswell. Thiscodeisdevelopedbased on_test5.Whatweaddare: 1. betterautomaticcross validationroutinethan _test5.m 2. kernelspecificcode snippet Wefoundthathavingkernel specificismuchslowerthan usingthedefault(without'tx'). Atthispoint,Ipreferusingthe defaultkernel. demo_libsvm_test8.m multiclass,OVR multi separated scale automatic, quite perfect default and specific arefine here 10class spiral Thecodeisdevelopedbasedon _test7.Theimprovementisthat theautomaticcrossvalidationfor parameterselectionismadeinto afunction,whichismuchmore convenient.Thefunctionis automaticParameterSelection demo_libsvm_test9.m multiclass,OVR multi leaveone scale outnfold automatic, cross quite validation perfect default 10class spiral Thiscodeisanexcellent examplecompletecodefor classificationusingnfoldcross validationandautomatic parametersselection. Thecodeisdevelopedbasedon _test8.Thedifferenceisweput thenfoldclassification(from _test6)intoafunction: classifyUsingCrossValidation demo_libsvm_test10.m multiclass,OVR multi separated scale automatic, quite perfect default 10class spiral Thiscodeisanexcellent examplecompletecodefor classificationonstrain test_separateddatasetand automaticparametersselection.
demo_libsvm_test7.m multiclass,OVR multi separated scale automatic, quite perfect
default and specific arefine here
10class spiral
demo_libsvm_test11.m multiclass,OVR multi separated scale automatic, quite perfect
demo_libsvm_test12.m multiclass,pair wise(default methodfor multiclassinthe libsvmpackage)
multi separated scale automatic, quite perfect
Thecodeisdevelopedbasedon _test8and_test9. default 3classring Thiscodeisdevelopedbased and ontest10,exceptthatthecode specific ismadetoworkforanykernel. arefine However,theresultsarenot here goodatall.Moreover,therun timeisnotgoodeither.Wefound abetterwayusingmulticlass pairwiseSVM,whichisthe defaultmulticlassSVMapproach inthelibsvmpackage.Inthe nextversion(_test12),wewill testthepairwiseSVM. default 4class Thecodeisdevelopedbasedon and spiral _test11.Ifigurethatthefunction specific svmtrainandsvmpredict, kernelare originallyimplementedinlibsvm, finehere. supportmulticlasspairwise SVM.Wedon'tevenneedto makethekernelmatrixourself, weyouneedtodoisjustpick yourkernel'tx',parameters'c ygz',andyouwillgetthe results.Withthisregard,Imake anotherversionofparameter selectionroutineusingcross validation: automaticParameterSelection2.m <onlyslightlydifferentfrom
https://sites.google.com/site/kittipat/libsvm_matlab
7/8
07/10/13
libsvm for MATLAB - Kittipat's Homepage
automaticParameterSelection.m> whichcallthenfoldcross validationclassificationroutine: svmNFoldCrossValidation.m Iwouldsaythisisthebestsofar codetorunonseparateddata setasitprovidesparameter selectionroutineandthetrain andclassificationroutines.Very easytofollow. demo_libsvm_test13.m multiclass,pair multi leaveoneout default 4classring Thecodeisdevelopedbasedon wise scale nfoldcross and _test12.Theonlydifferenceis automatic, validation specific thatthiscodeusenfoldcross quite kernelare validationwhenclassifyingthe perfect finehere. "single"dataset,i.e.,thedata setwherebothtrainandtestset arecombinetogetheroften foundwhenthenumberof observationsislimited. Thisisthebestcodetouseto runonthesingledatasetusing nfoldcrossvalidation classification. Allthecodecanbefoundinthezipfilehere.
Subpginas(8): Classifyusingnfoldcrossvalidation Completeexample Completeexampleforclassificationusingnfoldcross validation Completeexampleonclassificationinmostgeneralscenario demo_libsvm_crossvalidation.m demo_libsvm_kernel.m demo_libsvm_ovr_multiclass.m UseMulticlassOVRSVMwithcrossvalidationandkernelspecification
libsvm3.12.zip(840k) use_libsvm.zip(1597k)
Comentrios
Vocnotempermissoparaadicionarcomentrios.
KittipatKampa,08/05/201309:58 KittipatKampa,11/01/201312:52 v.1 v.3
Fazerlogin | Atividaderecentenosite | Denunciarabuso | Imprimirpgina | Removeracesso | Tecnologia GoogleSites
https://sites.google.com/site/kittipat/libsvm_matlab
8/8
