Phương pháp Bayes và ứng dụng trong mạng Nơron

day: XI ~ U yXn U-7 I Hamphituye'nI Hinh 3.1:Ndronnhant(;lO D~uvaocuandronnhant(;lOg6mn tinhit%uXi(i =1,2,...,n).D~urala tinhit%uy. tqngthaibelltrongcuandrondu'Qcxacdinhquabe)t6ngcacd~uvaocotrQngso' Wi (i =1,2, ...,n).d~uray cuandron du'Qcxacdinhquahamphi tuye'nf naodo Nhu'v~ymahinhdinhlu'Qngcuandronnhant(;lOnhu'sau: n yet)=1(2:WiXi(t) - B) i=1 Trang63 n (j daynet=I WiXi(t) =I(t) la tinhit%ut6nghQpd~uvao,Wi- cactrQngs6,i =1,2, i=1 ...,n d~ctru'ngchotinhlienke'tcuacackhopsynep,e la ngu'ongkichho~tnaron, t - thai gian,n - s6tinhit%ud~uvao,f - hamkichho~t. Ngu'aitathu'angdungky hit%usau:D~ura out=yet)=tenet). TomI~icoth€ xemndronIahamphituye'nhi~ud~uvao,mQtd~uta. Cac ndronco th€ lien ke'tI~i voi nhaut~othanhm~ngndronnh~nt~o.Vi dl;l ndronlien ke'tvOindronj theohai chi~uthu~nnghich(co thongtinphanh6i) nhu' (j Hlnh 3.2: ~o ?t W.. "-JI ~ ---70 ? ~ 70 / WiL Hlnh 3.2: Lien ke'thai chi~ugiuandroni va ndronj 3.2.2.Suem~nheuamo hlnh m~ngndron Nhungmohlnhndrondfftrlnhbaycoti~mnangt~onenmQtcuQccachm~ng trongcongnght%maytinhvacacquatrlnhxii'Iy thongtin.Nhungmongmu6nva hyvQngdobtitngu6nchuye'utucacd~ctntngsau: a. Kha nangcuacacquatrlnhxii'Iy songsongva phantan: co th€ du'avao m~ngmQthiQngIOncacndronlienke'tvoinhautheonhungIu'Qcd6voicac kie'ntruckhacnhau. b. Khanangthichnghivatvt6chilc:v~d~ctru'ngnay,ngu'oitad~c~ptoikha nangxii'Iy thichnghiva di~uchinhb~nvungdvavaocacthu~toanhQc thichnghivacacquyttictvt6chilc. Trang64 e. Khi nangdungthil l6i : e6gangbatehu'oekhi nangdungthil l6i euanao rheanghIah~th6ngcoth€ liSpWelamvi~eva di6uehlnhkhinh~ntinhi~u vaome)tph~nthongtinbi sail~ehho~ebithiSu. d. Xli'ly caequatrlnhphituySn: d~etru'ngnayra'tquailtrQng,vi d1,ltrongxa'p Xlm~ng,mi~nnhi~u(eha'pnh~n hi~u)vacokhi nangphanlOp. 3.3.Ca'utruem~ngndron ca'utrueeuam~ngnc1ronehuySudu'Qed~etru'ngbdi lo~ieuacaenc1ronva m6ilienh~xli'lythongtingiuachung. V6 diu trueeuanaron:ehuySungu'oitaquailtamtoicae"t5ng"caetinhi~uvao, ngu'ongt~im6inaronvacaehamehuy€n- hamkiehho~t. 3.3.1.Hamkiehho~t Hamkichho~teuatungnarontrongm~ngnarondongvai tro quailtrQng trongsvlienkStgiuacaenaron.Hamnayd~etru'ngehomilede)lienkStgiuacae naron,phept5nghQpcaetinhi~ud~uvao vathu'ongdu'Qekyhi~udu'oid~ngsau: vi d1,ld6ivoinaronj comtinhi~ud~uvaoXi m netj=LXiWji;Wji =(Wjp...,Wjm) i;1 I d~uraeuanaronj thu'ongdu'Qekyhi~ula outjho~efj . Saudayla me)ts6hamkieh ho~t. a. { I (net. - B.) ~0, J .I Ii =out= -l,(netj-Bj)<O b.D~nghamGauss: Ij =outj=exp(-(netj_Bj)2 c.D~nghamsigmoid(hayhamlogistic): J; =outj =(1+exp(-netj - Bj)))-1 caelo~ihamkichho~tconconhi6ubiSnth€ khaenhau. Trang65 3.3.2.Lien ke'tm~ng Sv lienke'ttrongm(;lngmjrontuythuQcvaonguyenly tudnglacgiii'ad~uracua tungmiron.V~nguyenH1csecora'tnhi~uki€u lienke'tgiii'acacmiron.M6i ndron lamQtnutcuam(;lng.MQts6ca'utruchayg~ptrongnhii'nglingdl;lngcod(;lngsail: 3.3.2.1.Caem~ngtruy~nth~ng ~ Do la mQtd6thidinhhuonghii'llh(;lnkhongchutrmh(acycIid).M6i nutla mQtndroncophanbi<%tnutvaovanUtfa.CacndronchiarhealOp,trenm6iclingco trQngs6Wijn6indronj voindroni tronglOpsail.M6i nutk khongphainutvaoco gallnguongOk'M(;lngtrlnhbaylIen Hlnh3.3 seminhho(;lchomQtm(;lngndron truy~nth£ngco3lOp. Lopvao Lop fin Lopra 7 ~o °/0 "- 7 7 Hlnh 3.3:Ca'utructruy~nth£ngphanlOp 3.3.2.2.M~ngn6i ngliqe: Cacm(;lngcothongtinvaxli'ly rheahaichi~u(con6ingu<;1ccongQiIam(;lng h6iquyrecurrentnetworks) ~ 0 4 """"""""'}1°"""""""""""""""""""""""""""""""""""""""'o---r ;;.O ~ Hlnh3.4:M~ngHopfield,dunglienke'tphanh6i Trang66 A B H K Hlnh3.5:M6 hinhm~ngIantruy~ndalOp 3.3.2.3.M~ngIantruy~ndalop [3] Trongca'utrucnay,lopthunha'tI gQila lOpnh~p,cacndronnh~pnh~ndfi' lil$uvaotITA vachuy~nchocacndrond lopke-tie-poLop thun,K gQila lOpxua'tva caclOpHi , i=I,2,...,n,la caclOpgn. MC;lngIantruy~ncohaitr~ngthaithihanh: tr~ngthaianhx~vatrC;lngthaihQC. * Trongtqng thaianhXc;l,dfi'lil$uduQcnh~pvaoI va xua'tra d o. giatri cuacac nutOJduQcxacdinh: OJ =F(L wijOj- Bj) ;F thuongla hamtruy~n(hamsigmoid)gall chondronj ? * d trC;lngthai hQC,m~ngIan truy~nslYd1:mgnhi~ul~ntr~ngthai anh Xc;ld~hil$u chlnhIC;licacgia tri trQngs6wijcuano. Va'nd~chinhd~trakhixftydt!ngmQtmC;lngndronlavi~cxacdinhcacwijthichhQp chom6ibai loan.B~giaiquye-tva'nd~ nayconhi~uthu~tloanduQCnghienCUll va cai tie-noMQttrongnhfi'ngthu~tloanchugnva n6i tie-figdo la Thu~UtminIan truy~nngtigc[6][12],thu~tloannhusail: Cho Q ={Vp=(Xp,Tp)IXp =(X1P,...,Xmp),Tp=(~p,...,Tmp)) bi~udi€n dfi'lil$unh~ptrongm§:uco duQcdunglamdfi'lil$uhQcchom~ngndron, trongdo: Xpla vectorbi~udi€n dfi'lil$unh~ptrongm§:u. Trang67 Xip,i=1,2,...,mla cacbie'nthanhphfincuadfi'li~unh~p. Tpla vectorbi<€udi~ndfi'li~uxua"ttrongm~u. Tjp,j =1,2,...,nla cacbie'nthanhphfincuadfi'li~uxua"t £)~tE =~I~(Tjp -Ojp)2; E du'QcgQi1aham16i.p J Ojpla bie'nthanhphfincuake'txua"tm~ngva dfi'li~unh~pXp Thui;itoanhQeIantruy~nnguqeg6mcaehuoe: Buoe1:Khdit~otrQngs6 CactrQngs6lienke'tgifi'acaendroncaclOpvangu'ongeuam6indrondu'Qcgallcac giatring~unhienxE[- r,r], thu'ongr du'Qc1a"y giatri 1. Buoe2 :Tinhcacke'txua"t 1. Cacgiatrike'txua"tcuandrond lOpnh~pdu'Qcquydinhbdi cacbie'nthanh phfincuadfi'li~unh~p. 2. Giatrixua"tracuacacndronlOpgnvalOpxua"txacdinhbdi. O. =F ("w..O. -8. )J L".yz J thu'ongF 1ahamsigmoidco d~ngF(x) =1 l_Dx;trongdo D 1ah~s6bi<€udi~n+e tinhphituye'neuaF. Buoe3 :SU'atrQngs6 QuatrinhsU'atrQngs6du'Qetie'nhanhl~pnhi~u1finquanhi~ulOp,b~tdfiutitcac ndronlOpxua"t,saudode'ncaendronthuQclOpgn. TrQngs6lien ke'thi~uchinhtheoeongthuc: w..(t+l) =w..(t)+~w..y y y Trang68 Wij(I) la gia tr~trQngs61ienke'ttu ll(jroni toi ndronj t~ibuocl~pthut l1ijla gia tr~bie'nthientrQngs6t~ibuocl~pthu(t+l) 1. Bie'nthientrQngs6duQcxacd~nhnhusau: 8E l1w..=8-=88.0. 1J bW.. ] ] 1J 8 la thams6t6cdQhQc(0<8<1,8j la gradient16it~indronj 2. Gradient16iduQcxacdinhtheocangthuc: * D6ivoicacndronthuQclOpxua't: 8. =O.(l-O. )(T. -0. )] ] ] ] ] Tj lagiatqbie'nthanhphffncuaduli~uxua'trongm§:utudnglingvoindronxua'tj OJ lagiatr~ke'txua'tcuandronxua'tj * D6i voi cacndronthuQclop ffnHi bj =OJ(1-Oj)L8kwkj k bkIa gia tr~gradient16it'.lindronk thuQclOpke'tie'pHi co lien ke'tvoi ndronffnj L~pl~iquatrinhtubuoc1chode'nkhim~ngndronhQitl,1.Vi trihQitl,1lamQtmuc chuffn16iduQcchQntnl'oc.MQtbuocl~pg6m:bi€u di~nbie'n,tinhcacke'txua'tva hi~uchlnhcactrQngs6trongm~ng. M(~lllgndronnhi~ulOpIantruy~n guQcla mQtghHphaphUllhi~uchocangvi~c mahinhboa,d~cbi~tvoi quatrinhphuct~pho~ccdche'chuaro rang.No khang doihoiphiiibie'truocd'.lnghams6ho~c acthams6. 3.4. Di~uki~ndunghQc[3] Trang69 Cac di~uki<%ndungla ye'uto'khongth€ thie'ukhi xay dl,tngthu~ttoanhQc ehom(;lngndron.Thu~ttoanhQcse dungkhi di~uki<%ndungduQcthoaman.f)i~u ki~ndungxacdinhtrenmQtso'tieuchuftnsau: * Dl,tatrendQ16igiUake'txufitcua m(;lngva ke'txufitm~utren cungmQtdii'li~u m~uduavao. Thu~ttoandunghQekhi 16iE<Go,Gola nguong16iho~ekhi khong con16i.Tuy nhientieuehuftnnaykhongdambaatinht6ngquatd6ivoi dii'li~u maio * Dl,tatrengradient16i:thu~toandungkhi gradient16idu nho(gradientb~ng khongt(;livi triclfeti€u) 3.5.Kiim tra tinhdungdiinvatinht6ngquatcuam~ngndron[3] MQth~dl,tdoanla mQth~th6ngbaag6mmQtm(;lngnaronva caetri thue xacdinhmQtanhX(;ltrendii'li~u.Cach~dl,tdoand~ucomQtdQtine~y,tinhdung d~nnhfitdinh.Voi m(;lngnarontinhdungd~ncuanoth€ hi~nquahaiye'uto': - KhananghQCvahinhthanhtrithuctucacdii'li~um~u - Kha nangt6ngquatboachocacdii'li~umoikhongduQchQc. QuatrinhhQchinhthanhtri thucxacdinhnhii'ngm6ilienh~rangbuQc,gQila cac anhX(;l,giii'acaeye'uto'diiuvaovadiiuratrongt~pdiYli~um~u. AnhX(;lthuduQcco th€ rfitdungtrencacdii'li~uhQc.Tuynhienne'uchidungcac dii'li~uduQchQcd€ danhgiatrithuchinhthanhthichiphananhduQcmucdQphu hQpgiii'atri thucva dii'li~uhQcmakhongdunoilenduQcdQtinc~yciinthie'tcua trithucnayd6ivoicacdiYli~ukhongduQchQc. Dodo,bellc(;lnhcacphepki€m chungv~khananghQccuam(;lngnaron,taciinco cacki€m chungv~khanangt6ngquatboacuam(;lngnarond6i voi cacdii'li~u khongduQchQc. Trang70 M,;mgncJronchidu'QcxemlamQtht$hoanchinhne'ucotinht6ngquaboa. 3.5.1.Xac dfnhtinht6ngquatboa. Vit$cxac dinhtinht6ngquatboa du'Qctie'nhanhtrenmQtt~pdli lit$uki@m ITaphanbit$tvoi t~pdli lit$uhQc.Phu'dngphapnayd1!atrenmQtphu'dngphapth6ng kegQilaphu'dngphapthamchie'ucheo. 3.5.2.Phlidngphapthamchie'ucheo Trongphu'dngphapnay,t~pm~udu'Qcphanthanh2 t~priengbit$t: t~p luyt$nvat~pki@mtra. Saudaylaytu'dngcuaphu'dngphapthamchie'ucheo- k fold crossvalidatation: Chiang~unhient~pdli lit$uthanhk t~pconphanbit$t - L~pl~ik l~n,m6i l~nchuara mQtt~pcond@ki€m ITa,ph~nconl~id@ luy~n. - Tinhgiatfi crossvalidatationQ2chom6il~nl~pl~id trenk=1,2,...,n n L(Y; _yJ2 Q2 =1- ;=1n L(Y; - yJ2;=1 Yi la ho~ttinhdoannh~nduQcungvdi ho~ttinhYi - Y lagiatriho~tinhtrungblnh GiatriQ2cangg~n1thlkhanangdoannh~nho~tinhdli li~ucangchinhxachay n6icachkhacm~ngcokhanangt6ngquatboa. 3.6.Hi~n tlitjng overfitting [3],[4] ; MQttrongnhlingnhu'Qcdi@mIoncuaphu'dngphapm~ngndronla hi~ntu'QngI l overfitting. Trang71 Hi~ntuQngoverfittingla truanghQpmam~ngsaukhihQcKongchok€t quahQitv trent~pm§:unhungkhidungm~ngvoi trithliccoduQcdod€ dVdoank€t quacua dli li~umoi thll~i hoantoankh6ngchinhxac. Nhuda:noi, m~ngnc5ronduQcKaydvngnh~mxac dinhmQtanhx~gilia cacy€u to' d~uvaovad~uratuanglingcuadli li~u.Anhx~xacdinhc~ncodQchinhxaccao. ~ Thlfct€ voit~pdli li~um§:uchotruoc,chIcoth€ dambaam~ng iaiquy€tbaitoan cimQtgioih~nnao\1o.Cacthaotaclamgiaml6i trongquatrlnhhQc'chlnh~mml;lc dichxacdinhmQtanhx~phuhQpvoim~uduavaomakh6nglamgiamcacl6i sinh rakhis1l'dl;lnganhx~trencacdli li~umoic~ndvdoancuabaitoan. Vi~cchok€t quahQitv trent~pm~uda:lambi€n d~ngcacanhx~c~nKaydvng d~nd€n hi~ntuQngoverfitting. Hi~ntuQngoverfittingcothSh~nch€ trongcaccachsau: tangkichthuoct~p m§:u, thayd6iso'nUtffn,ki€m soatthaidi€m dunghQc,... 3.7.Anhhtidngcuas6ndrontrentinhi~ucuam~ng[3],[4] M~ngIantruy~nguQcoth€ xftpXlmN anhx~codQphlict~ptuyyn€uno cocftut~othichhQp.Cftuhlnhcuam~ngnaronIantruy~nguQcduQcquydinhbCii 86naronlOpnh~p,so'naronlOpxuftt,so'lOpffnva s6narontrongtunglOpffn. S6luQngnarontunglOpnh~p,lOpxufttduQcquydinhtheoyeti c~uva bai toansa dl;lngm~ng.Vi~c xac dinhcftuhlnhm~ngthichhQpse ti€n hanhquavi~cthayd6i trenso'lOpffnva so'naroncualOpffn. M(~mgnaronco th€ co nhi~ulOpffn,tuynhientrongdas6 cactruanghQpm~ng ndronmQtlOpffncoth€ xftpXlmQtanhX~bfttky.Vi~cb6sungthems6lOpffnlam phlict~pquatrlnhtinhtoanvachuangtrlnhch~ych~mbon. Trang72 Ne'us6 ndronfin nho, mC;!ngkhongth€ xac dinhduQcanh xc;!phu hQpvdi dfi'li<$u hQc.Tangs6naronfincoth€ lamanhxc;!x~pXlcacdfi'li<$uhQct6thall.d mQtmilc naodo, anh xc;!hinhthanhr~tphuhQpvdi dfi'li<$uhQcnhungkhii nangoverfitting xayra clingr~tIOnvi the'clingcfinxac dinhmQttieu chufindungd6i vdi vi<$ctang s6luQngnaron. ._.