NËoi AlgÏrijmoi Ekpa–deushc Teqnht¸n Neurwnik¸n Dikt ... - Nemertes

Nèoi Algìrijmoi EkpaÐdeushcTeqnht¸n Neurwnik¸n DiktÔwn kaiEfarmogècAristotèlhc E. KwstìpoulocDidaktorik DiatribPanepist mio Patr¸nSqol Jetik¸n Episthm¸nTm ma Majhmatik¸nPˆtra 2011

Stouc goneÐc mou kai thsÔzugì mou pou mepisteÔoun kai sthrÐzoun

PerÐlhyhH paroÔsa didaktorik diatrib pragmateÔetai to jèma thc ekpaÐdeushc emprìsjiwntrofodotoÔmenwn teqnht¸n neurwnik¸n diktÔwn kai tic efarmogèc touc. H parousÐashtwn jemˆtwn kai twn apotelesmˆtwn thc diatrib c organ¸netai wc ex c:Sto Kefˆlaio 1 parousiˆzontai ta teqnhtˆ neurwnikˆ dÐktua, ta ofèlh thc qr -shc touc, h dom kai h leitourgÐa touc. Pio sugkekrimèna, parousiˆzetai pwc apìtouc biologikoÔc neur¸nec montelopoioÔntai oi teqnhtoÐ neur¸nec, pou apoteloÔn tojemeli¸dec stoiqeÐo twn teqnht¸n neurwnik¸n diktÔwn. Sth sunèqeia anafèrontai oibasikèc arqitektonikèc twn emprìsjiwn trofodotoÔmenwn teqnht¸n neurwnik¸n diktÔwn.To kefˆlaio oloklhr¸netai me mia istorik anadrom gia ta teqnhtˆ neurwnikˆdÐktua kai me thn parousÐash kˆpoiwn efarmog¸n touc.Sto Kefˆlaio 2 parousiˆzontai merikoÐ apì touc upˆrqontec algìrijmouc ekpaÐdeushcteqnht¸n neurwnik¸n diktÔwn. GÐnetai mia perilhptik anaforˆ tou probl -matoc thc ekpaÐdeushc twn teqnht¸n neurwnik¸n diktÔwn me epÐbleyh kai dÐnetai hmajhmatik tou montelopoÐhsh pou antistoiqeÐ sthn elaqistopoÐhsh miac sunˆrthshckìstouc. Sthn sunèqeia gÐnetai mia perilhptik anaforˆ stic mejìdouc pou basÐzontaisthn kateÔjunsh thc pio apìtomhc kajìdou, stic mejìdouc deutèrac tˆxewc ìpouapaiteÐtai o upologismìc tou EssianoÔ pÐnaka thc sunˆrthshc kìstouc, stic mejìdoucmetablht c metrik c, kai stic mejìdouc suzug¸n klÐsewn. Katìpin, parousiˆzetai oq¸roc twn bar¸n, h epifˆneia sfˆlmatoc kai oi diˆforec teqnikèc arqikopoÐhshc twnbar¸n twn teqnht¸n neurwnik¸n diktÔwn kai perigrˆfontai oi epipt¸seic pou èqounsthn ekpaÐdeush touc.Sto kefˆlaio 3 parousiˆzetai ènac nèoc algìrijmoc ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn basismènoc ston algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatockai sthn autìmath prosarmog tou rujmoÔ ekpaÐdeushc qrhsimopoi¸ntac plhroforÐadÔo shmeÐwn [81]. H kateÔjunsh anaz thshc tou nèou algorÐjmou eÐnai h kateÔjunshthc pio apìtomhc kajìdou, allˆ gia ton prosdiorismì tou rujmoÔ ekpaÐdeushc qrhsimopoioÔntaiproseggÐseic dÔo shmeÐwn thc exÐswshc qord c twn mejìdwn yeudì Newton[5]. Epiplèon, parˆgetai ènac nèoc rujmìc ekpaÐdeushc proseggÐzontac thn nèa exÐswshqord c, pou protˆjhke apì ton Zhang [90], h opoÐa qrhsimopoieÐ plhroforÐaparag¸gwn kai sunarthsiak¸n tim¸n. Sthn sunèqeia, ènac katˆllhloc mhqanismìc3

4epilog c tou rujmoÔ ekpaÐdeushc enswmat¸netai ston algìrijmo ekpaÐdeushc ¸stena epilègetai kˆje forˆ o katˆllhloc rujmìc ekpaÐdeushc. Tèloc, gÐnetai melèth thcsÔgklishc tou algorÐjmou ekpaÐdeushc kai parousiˆzontai ta peiramatikˆ apotelèsmatagia diˆfora probl mata ekpaÐdeushc.Sto Kefˆlaio 4 parousiˆzontai merikoÐ apotelesmatikoÐ algìrijmoi ekpaÐdeushc oiopoÐoi basÐzontai stic mejìdouc beltistopoÐhshc suzug¸n klÐsewn [43]. Stouc upˆrqontecalgìrijmouc ekpaÐdeushc suzug¸n klÐsewn prostÐjetai ènac nèoc algìrijmocekpaÐdeushc pou basÐzetai sthn mèjodo suzug¸n klÐsewn tou Perry [60]. Epiprìsjeta,proteÐnontai nèoi algìrijmoi suzug¸n klÐsewn pou prokÔptoun apì tic Ðdiec arqèc pouproèrqontai oi gnwstoÐ algìrijmoi suzug¸n klÐsewn twn Hestenes-Stiefel, Fletcher-Reeves, Polak-Ribière kai Perry, kai onomˆzontai klimakwtoÐ algìrijmoi suzug¸n klÐsewn.Aut h kathgorÐa algorÐjmwn basÐzetai sth fasmatik parˆmetro klimˆkwshcpou protˆjhke apì touc Barzilai kai Borwein [5]. Epiplèon, enswmat¸netai stouc algìrijmoucekpaÐdeushc suzug¸n klÐsewn mia apodotik teqnik grammik c anaz thshc,pou basÐzetai stic sunj kec tou Wolfe kai sthn diasfalismènh kubik parembol [77].Akìmh, h parˆmetroc tou arqikoÔ rujmoÔ ekpaÐdeushc prosarmìzetai autìmata se kˆjeepanˆlhyh sÔmfwna me èna kleistì tÔpo tou protˆjhke stic ergasÐec [77] kai [80].Sth sunèqeia, efarmìzetai mia apotelesmatik diadikasÐa epanekkÐnhshc, ètsi ¸ste nabeltiwjoÔn peraitèrw oi algìrijmoi ekpaÐdeushc suzug¸n klÐsewn kai na apodeiqjeÐ holik touc sÔgklish. Tèloc, parousiˆzontai ta peiramatikˆ apotelèsmata gia diˆforaprobl mata ekpaÐdeushc.Sto teleutaÐo Kefˆlaio thc paroÔsac didaktorik c diatrib c, apomon¸netai kaitropopoieÐtai o klimakwtìc algìrijmoc tou Perry, pou parousiˆsthke sto prohgoÔmenokefˆlaio. Pio sugkekrimèna, en¸ diathroÔntai ta kÔria qarakthristikˆ tou algorÐjmouekpaÐdeushc, efarmìzetai mia diaforetik teqnik grammik c anaz thshc hopoÐa basÐzetai stic mh monìtonec sunj kec tou Wolfe. EpÐshc, proteÐnetai ènac nèocarqikìc rujmìc ekpaÐdeushc ([41]) gia qr sh me ton klimakwtì algìrijmo ekpaÐdeushcsuzug¸n klÐsewn, o opoÐoc faÐnetai na eÐnai apodotikìteroc apì ton arqikì rujmìekpaÐdeushc pou protˆjhke apì ton Shanno [77] ìtan qrhsimopoieÐtai se sunduasmìme thn mh monìtonh teqnik grammik c anaz thshc. Sth sunèqeia parousiˆzontai tapeiramatikˆ apotelèsmata gia diˆfora probl mata ekpaÐdeushc. Tèloc, wc efarmogekpaideÔetai èna poluepÐpedo emprìsjia trofodotoÔmeno teqnhtì neurwnikì dÐktuo meton proteinìmeno algìrijmo gia to prìblhma thc taxinìmhshc karkinik¸n kuttˆrwn touegkefˆlou kai sugkrÐnetai h apìdosh tou me thn apìdosh enìc pijanotikoÔ teqnhtoÔneurwnikoÔ diktÔou [42].H diatrib oloklhr¸netai me to Parˆrthma A', ìpou parousiˆzontai ta probl mataekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn pou qrhsimopoi jhkan gia thn axiolìghshtwn proteinìmenwn algorÐjmwn ekpaÐdeushc.

SynopsisIn this dissertation the problem of the training of feedforward artificial neural networksand its applications are considered. The presentation of the topics and theresults are organized as follows:In the first chapter, the artificial neural networks are introduced. Initially, thebenefits of the use of artificial neural networks are presented. In the sequence, thestructure and their functionality are presented. More specifically, the derivation ofthe artificial neurons from the biological ones is presented followed by the presentationof the architecture of the feedforward neural networks. The historical notes and theuse of neural networks in real world problems are concluding the first chapter.In Chapter 2, the existing training algorithms for the feedforward neural networksare considered. First, a summary of the training problem and its mathematicalformulation, that corresponds to the uncostrained minimization of a cost function, aregiven. In the sequence, training algorithms based on the steepest descent, Newton,variable metric and conjugate gradient methods are presented. Furthermore, theweight space, the error surface and the techniques of the initialization of the weightsare described. Their influence in the training procedure is discussed.In Chapter 3, a new training algorithm for feedforward neural networks basedon the backpropagation algorithm and the automatic two-point step size (learningrate) is presented [81]. The algorithm uses the steepest descent search directionwhile the learning rate parameter is calculated by minimizing the standard secantequation [5]. Furthermore, a new learning rate parameter is derived by minimizingthe modified secant equation introduced in [90], that uses both gradient and functionvalue information. In the sequence a switching mechanism is incorporated into thealgorithm so that the appropriate stepsize to be chosen according to the status of thecurrent iterative point. Finaly, the global convergence of the proposed algorithm isstudied and the results of some numerical experiments are presented.In Chapter 4, some efficient training algorithms, based on conjugate gradient optimizationmethods, are presented [43]. In addition to the existing conjugate gradienttraining algorithms, we introduce Perry’s conjugate gradient method as a trainingalgorithm [60]. Furthermore, a new class of conjugate gradient methods is proposed,5

6called self-scaled conjugate gradient methods, which are derived from the principles ofHestenes-Stiefel, Fletcher-Reeves, Polak-Ribière and Perry’s method. This class is basedon the spectral scaling parameter introduced in [5]. Furthermore, we incorporateto the conjugate gradient training algorithms an efficient line search technique basedon the Wolfe conditions and on safeguarded cubic interpolation [77]. In addition, theinitial learning rate parameter, fed to the line search technique, was automaticallyadapted at each iteration by a closed formula proposed in [77] and [80]. Finally, anefficient restarting procedure was employed in order to further improve the effectivenessof the conjugate gradient training algorithms and prove their global convergence.Experimental results show that, in general, the new class of methods can performbetter with a much lower computational cost and better success performance.In the last chapter of this dissertation, the Perry’s self-scaled conjugate gradienttraining algorithm that was presented in the previous chapter was isolated and modified.More specifically, the main characteristics of the training algorithm weremaintained but in this case a different line search strategy based on the nonmonotoneWolfe conditions was utilized. Furthermore, a new initial learning rate parameterwas introduced ([41]) for use in conjunction with the self-scaled conjugate gradienttraining algorithm that seems to be more effective from the initial learning rate parameter,proposed by Shanno in [77], when used with the nonmonotone line searchtechnique. In the sequence the experimental results for differrent training problemsare presented. Finally, a feedforward neural network with the proposed algorithmfor the problem of brain astrocytomas grading was trained and compared the resultswith those achieved by a probabilistic neural network [42].The dissertation is concluded with the Appendix A’, where the training problemsused for the evaluation of the proposed training algorithms are presented.

EuqaristÐecH paroÔsa didaktorik diatrib ja tan adÔnato na ekponhjeÐ qwrÐc thn kajoristikbo jeia kai sumparˆstash poll¸n anjr¸pwn. Ja jela katarq n na euqarist swjermˆ thn epiblèpousa thc didaktorik c diatrib c, epÐkouro kajhg tria ka. JeodoÔlaGrˆya. H parousÐa thc tan kajoristik se ìlh th diˆrkeia foÐths c mou stoTm ma Majhmatik¸n tou PanepisthmÐou Patr¸n. H kajod ghs thc tan ousiastikkai me bo jhse na xeperˆsw tic diˆforec duskolÐec pou antimet¸pisa katˆ thn èreunakai suggraf thc didaktorik c diatrib c. Ja jela, epÐshc, na euqarist sw kai taupìloipa mèlh thc trimeloÔc sumbouleutik c epitrop c, touc kajhghtèc k.k. KosmˆIordanÐdh kai Qarˆlampo Mpìtsarh gia thn bo jeia pou mou prosèferan gia thn oloklrwsh thc diatrib c aut c, allˆ kai gia tic polÔtimec gn¸seic pou mou prosèferankatˆ thn diˆrkeia thc foÐths c mou.Sto shmeÐo autì, ja jela na euqarist sw thn kajhg tria kai prÔtanh tou TampereUniversity of Technology ka. Ulla Ruotsalainen, ton didˆktora k. Antti Happonen,allˆ kai ta upìloipa mèlh thc ereunhtik c omˆdac M 2 oBSI, me touc opoÐouceÐqa thn tim na sunergast¸ se jèmata epexergasÐac shmˆtwn sta plaÐsia thc upotrofÐacMarie Curie thc Eurwpaðk c 'Enwshc. EpÐshc, ja jela na euqarist sw gia thnanektÐmhth bo jeia pou mou prosèferan touc upìloipouc sunergˆtec mou, k.k. IwˆnnhNÐka, Gerˆsimo Antzoulˆto, QristÐna Nikolakˆkou, Aristotèlh Klamargiˆ, Dhm trioGklìtso kai Stulianì D ma.Epiplèon, ja tan parˆleiy mou na mhn euqarist sw thn dieujÔntria thc Bibliojkhc kai Kèntrou Plhrofìrhshc tou PanepisthmÐou Patr¸n ka. AikaterÐnh Sunèllhallˆ kai to upìloipo proswpikì gia thn bo jeia kai thn st rixh touc katˆ thn diˆrkeiathc foÐthshc mou. Ja jela, epÐshc, na apeujÔnw euqaristÐec kai sto proswpikì touErgasthrÐou Hlektronik¸n Upologist¸n kai Plhroforik c tou Tm matoc Majhmatik¸ngia th diˆjesh twn aparaÐthtwn upologistik¸n pìrwn.Tèloc, ja jela na euqarist sw jermˆ touc goneÐc mou, thn aderf mou kai thsÔzugì mou oi opoÐoi me st rixan me ìlh touc th dÔnamh kai ìla ta mèsa katˆ thdiˆrkeia twn spoud¸n mou.7

PerieqìmenaI Eisagwgh 19Kefˆlaio 1 Teqnhtˆ Neurwnikˆ DÐktua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.1 Ofèlh twn Neurwnik¸n DiktÔwn . . . . . . . . . . . . . . . . . . . . . . 221.2 BiologikoÐ Neur¸nec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241.3 TeqnhtoÐ Neur¸nec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.3.1 TÔpoi Sunart sewn EnergopoÐhshc . . . . . . . . . . . . . . . 301.4 Arqitektonikèc DiktÔwn . . . . . . . . . . . . . . . . . . . . . . . . . . . 331.4.1 MonoepÐpeda Emprìsjia TrofodotoÔmena DÐktua . . . . . . . . 331.4.2 PoluepÐpeda Emprìsjia TrofodotoÔmena DÐktua . . . . . . . . 341.5 H IstorÐa twn Teqnht¸n Neurwnik¸n DiktÔwn . . . . . . . . . . . . . . . 361.6 Efarmogèc twn Teqnht¸n Neurwnik¸n DiktÔwn . . . . . . . . . . . . . . 38Kefˆlaio 2 EkpaÐdeush twn Teqnht¸n Neurwnik¸n DiktÔwn . . . . . . . . . . . . . . . . . . 412.1 Mèjodoi EkpaÐdeushc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422.2 Epifˆneia Sfˆlmatoc kai o Q¸roc twn Bar¸n . . . . . . . . . . . . . . . 472.2.1 ArqikopoÐhsh twn Bar¸n . . . . . . . . . . . . . . . . . . . . . 499

10II Neoi Algorijmoi Ekpaideushc TND 53Kefˆlaio 3 Algìrijmoc EkpaÐdeushc Prosèggishc DÔo ShmeÐwn . . . . . . . . . . . . . . 553.1 H mèjodoc twn Barzilai kai Borwein . . . . . . . . . . . . . . . . . . . . . 553.2 Nèa 'ExÐswsh Qord c kai Paragwg Nèwn Bhmˆtwn . . . . . . . . . . . . 593.3 Nèoc Algìrijmoc EkpaÐdeushc kai h Olik SÔgklish tou . . . . . . . . . 623.4 Peiramatikˆ Apotelèsmata . . . . . . . . . . . . . . . . . . . . . . . . . . 653.4.1 Apokleistikì-EITE . . . . . . . . . . . . . . . . . . . . . . . . 673.4.2 IsotimÐa twn 3-bit . . . . . . . . . . . . . . . . . . . . . . . . . 683.4.3 Anagn¸rish KefalaÐwn Grammˆtwn . . . . . . . . . . . . . . . 693.4.4 Prosèggish SuneqoÔc Trigwnometrik c Sunˆrthshc . . . . . . 703.5 Sumperˆsmata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70Kefˆlaio 4 Nèa Oikogèneia AlgorÐjmwn EkpaÐdeushc Suzug¸n KlÐsewn . . . . . . . 734.1 Algìrijmoi EkpaÐdeushc Suzug¸n KlÐsewn . . . . . . . . . . . . . . . . . 744.2 KlimakwtoÐ Algìrijmoi EkpaÐdeushc Suzug¸n KlÐsewn . . . . . . . . . . 764.2.1 Parˆmetroc Klimˆkwshc . . . . . . . . . . . . . . . . . . . . . 774.3 Genikìc Algìrijmoc EkpaÐdeushc . . . . . . . . . . . . . . . . . . . . . . 784.3.1 DiadikasÐec EpanekkÐnhshc . . . . . . . . . . . . . . . . . . . . 794.3.2 DiadikasÐec Grammik c Anaz thshc . . . . . . . . . . . . . . . 804.3.3 Arqikìc Rujmìc EkpaÐdeushc . . . . . . . . . . . . . . . . . . 824.3.4 Perigraf tou AlgorÐjmou EkpaÐdeushc . . . . . . . . . . . . 824.3.5 Olik SÔgklish . . . . . . . . . . . . . . . . . . . . . . . . . . 844.4 Peiramatikˆ Apotelèsmata . . . . . . . . . . . . . . . . . . . . . . . . . . 864.4.1 Apokleistikì-EITE . . . . . . . . . . . . . . . . . . . . . . . . 884.4.2 IsotimÐa twn 3-bit . . . . . . . . . . . . . . . . . . . . . . . . . 90

114.4.3 N − M − N Kwdikopoiht c/Apokwdikopoiht c . . . . . . . . . 924.4.4 Anagn¸rish KefalaÐwn Grammˆtwn . . . . . . . . . . . . . . . 1024.4.5 Anagn¸rish Arijm¸n . . . . . . . . . . . . . . . . . . . . . . . 1044.4.6 Taxinìmhsh twn Fut¸n thc Oikogèneiac Iris . . . . . . . . . . . 1064.4.7 Optik Anagn¸rish Qeirìgrafwn Arijm¸n . . . . . . . . . . . 1084.4.8 Anagn¸rish Fwnhèntwn . . . . . . . . . . . . . . . . . . . . . . 1104.5 Sumperˆsmata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Kefˆlaio 5 Mh Monìtonoi Algìrijmoi EkpaÐdeushc Suzug¸n KlÐsewn . . . . . . . . 1155.1 O Algìrijmoc tou Perry . . . . . . . . . . . . . . . . . . . . . . . . . . . 1155.2 Mh Monìtonoc Klimakwtìc Algìrijmoc EkpaÐdeushc tou Perry . . . . . 1165.3 Peiramatikˆ Apotelèsmata . . . . . . . . . . . . . . . . . . . . . . . . . . 1215.3.1 Apokleistikì-EITE . . . . . . . . . . . . . . . . . . . . . . . . 1225.3.2 IsotimÐa twn 3-bit . . . . . . . . . . . . . . . . . . . . . . . . . 1235.3.3 Anagn¸rish KefalaÐwn Grammˆtwn . . . . . . . . . . . . . . . 1245.3.4 Prosèggish SuneqoÔc Trigwnometrik c Sunˆrthshc . . . . . . 1255.3.5 Taxinìmhsh Karkinik¸n Kuttˆrwn Egkefˆlou . . . . . . . . . 1265.4 Sumperˆsmata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129III Pararthmata Bibliografia 131Kefˆlaio Aþ Probl mata EkpaÐdeushc Neurwnik¸n DiktÔwn . . . . . . . . . . . . . . . . . . . . 133Aþ.1 Apokleistikì-EITE (XOR) . . . . . . . . . . . . . . . . . . . . . . . . . 133Aþ.2 IsotimÐa twn 3-bit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134Aþ.3 Anagn¸rish KefalaÐwn Grammˆtwn . . . . . . . . . . . . . . . . . . . . . 135Aþ.4 Prosèggish SuneqoÔc Trigwnometrik c Sunˆrthshc . . . . . . . . . . . . 136

12Aþ.5 N − M − N Kwdikopoiht c/Apokwdikopoiht c . . . . . . . . . . . . . . . 136Aþ.6 Anagn¸rish Arijm¸n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137Aþ.7 Taxinìmhsh twn Fut¸n thc Oikogèneiac Iris . . . . . . . . . . . . . . . . . 138Aþ.8 Optik Anagn¸rish Qeirìgrafwn Arijm¸n . . . . . . . . . . . . . . . . . 139Aþ.8.1 Anagn¸rish Fwnhèntwn . . . . . . . . . . . . . . . . . . . . . . 140Kefˆlaio Bþ DhmosieÔseic - Anaforèc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Katˆlogoc Sqhmˆtwn1.1 Sqèdio enìc biologikoÔ neur¸na. . . . . . . . . . . . . . . . . . . . . . . 251.2 Metˆdosh s matoc prosunaptikoÔ ˆxona mèsw sunaptik c sqism c ston metasunaptikìneur¸na. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.3 Mh grammikì montèlo enìc neur¸na. . . . . . . . . . . . . . . . . . . . . 271.4 Ommoparallhlikìc metasqhmatismìc paragìmenoc apì thn parousÐa merolhyÐac.Shmei¸netai ìti v k = b k sto u k = 0. . . . . . . . . . . . . . . . . . 281.5 'Ena ˆllo mh grammikì montèlo neur¸na. . . . . . . . . . . . . . . . . . . 301.6 Sunˆrthsh KatwflioÔ. . . . . . . . . . . . . . . . . . . . . . . . . . . . 311.7 Katˆ Tm mata Grammik Sunˆrthsh. . . . . . . . . . . . . . . . . . . . . 321.8 Sigmoeid c Sunˆrthsh. . . . . . . . . . . . . . . . . . . . . . . . . . . . 331.9 Emprìsjia trofodotoÔmeno dÐktuo me èna monì epÐpedo apì neur¸nec. . . . 341.10 Pl rwc diasundedemèno emprìsjia trofodotoÔmeno dÐktuo me èna krufì epÐpedokai èna epÐpedo exìdou. . . . . . . . . . . . . . . . . . . . . . . . . 352.1 H epifˆneia thc sunˆrthshc sfˆlmatoc pˆnw ston q¸ro twn bar¸n se ènaneurwnikì dÐktuo me èna mìno neur¸na. . . . . . . . . . . . . . . . . . . . 472.2 H epifˆneia thc sunˆrthshc sfˆlmatoc pˆnw ston q¸ro twn bar¸n se ènaneurwnikì dÐktuo me èna mìno neur¸na upì thn epÐdrash: a. enìc protÔpou,b. dÔo protÔpwn, g. tri¸n protÔpwn kai d. tessˆrwn protÔpwn. . . . . . . 482.3 IsoôyeÐc thc epifˆneiac thc sunˆrthshc sfˆlmatoc kai troqiˆ tou algìrijmouekpaÐdeushc. Aristerˆ o algìrijmoc sugklÐnei se èna topikì elˆqisto, en¸dexiˆ o algìrijmoc sugklÐnei sto olikì elˆqisto. . . . . . . . . . . . . . 494.1 Mèsoc qrìnoc gia to prìblhma tou apokleistikoÔ-EITE . . . . . . . . 8913

144.2 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmatou apokleistikoÔ-EITE . . . . . . . . . . . . . . . . . . . . . . . . . 894.3 Mèsoc qrìnoc gia to prìblhma thc isotimÐac twn 3-bit . . . . . . . . . 914.4 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmathc isotimÐac twn 3-bit . . . . . . . . . . . . . . . . . . . . . . . . . . 914.5 Mèsoc qrìnoc gia to prìblhma tou 4 − 2 − 4 Kwdikopoiht / Apokwdikopoiht. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944.6 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmatou 4 − 2 − 4 Kwdikopoiht / Apokwdikopoiht . . . . . . . . . . . . 954.7 Mèsoc qrìnoc gia to prìblhma tou 8 − 3 − 8 Kwdikopoiht / Apokwdikopoiht. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964.8 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmatou 8 − 3 − 8 Kwdikopoiht / Apokwdikopoiht . . . . . . . . . . . . 964.9 Mèsoc qrìnoc gia to prìblhma tou 16 − 4 − 16 Kwdikopoiht / Apokwdikopoiht. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974.10 Mèsoc qrìnoc gia to prìblhma tou 32 − 5 − 32 Kwdikopoiht / Apokwdikopoiht. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 984.11 Mèsoc qrìnoc gia to prìblhma tou 64 − 6 − 64 Kwdikopoiht / Apokwdikopoiht. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.12 Mèsoc qrìnoc gia to prìblhma tou 128 − 7 − 128 Kwdikopoiht /Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1004.13 Mèsoc qrìnoc gia to prìblhma tou 256 − 8 − 256 Kwdikopoiht /Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1014.14 Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc kefalaÐwn grammˆtwn 1024.15 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmathc anagn¸rishc kefalaÐwn grammˆtwn . . . . . . . . . . . . . . . . . 1034.16 Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc arijm¸n . . . . . . . 1054.17 KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhmathc anagn¸rishc arijm¸n . . . . . . . . . . . . . . . . . . . . . . . . . 1054.18 Mèsoc qrìnoc gia to prìblhma thc taxinìmhshc twn fut¸n thc oikogèneiacIris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

154.19 Mèsoc qrìnoc gia to prìblhma thc optik c anagn¸rishc qeirìgrafwnarijm¸n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1094.20 Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc fwnhèntwn . . . . . . 1115.1 ParadeÐgmata qamhloÔ (epˆnw) kai uyhloÔ bajmoÔ (kˆtw) egkefalik¸nastrokutwmˆtwn mazÐ me thn tmhmatopoihmènh eikìna. . . . . . . . . . 1275.2 KalÔterec epidìseic gia to emprìsjia trofodotoÔmeno neurwnikì dÐktuogia diaforetikì arijmì neur¸nwn sto krufì epÐpedo. . . . . . . . 130Aþ.1 To prìblhma tou ApokleistikoÔ-EITE sto epÐpedo. Apì to sq ma faÐnetaikajarˆ ìti ta prìtupa ekpaÐdeushc den eÐnai grammikˆ diaqwrÐsima. . . . . 134Aþ.2 H kwdikopoÐhsh twn grammˆtwn A, B, C kai D kai oi antÐstoiqec duadikèckwdikopoi seic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135Aþ.3 H grafik parˆstash thc f(x) = sin(x) cos(2x) sto [0, 2π]. Ta 20 prìtupaekpaÐdeushc apeikonÐzontai pˆnw sth grafik parˆstash me kìkkinec koukÐdec.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136Aþ.4 H kwdikopoÐhsh twn arijm¸n 0 kai 1 kai oi antÐstoiqec duadikèc kwdikopoi seic.138Aþ.5 Iris Setosa. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Aþ.6 Iris Versicolor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139Aþ.7 Iris Virginica. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Katˆlogoc Pinˆkwn3.1 Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITE . . 673.2 Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bit . . . 683.3 Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.4 Sugkritikˆ apotelèsmata gia to prìblhma thc prosèggishc suneqoÔctrigwnometrik c sunˆrthshc. . . . . . . . . . . . . . . . . . . . . . . . 704.1 Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITE . . 904.2 Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bit . . . 924.3 Prodiagrafèc twn problhmˆtwn N−M−N Kwdikopoiht /Apokwdikopoiht .Oi anaferìmenec parˆmetroi eÐnai: Dokimèc, o arijmìc twn exomoi¸sewn,'Orio Ep., o mègistoc epitreptìc arijmìc sunarthsiak¸n upologism¸n,Bˆrh, o arijmìc twn sunaptik¸n bar¸n, MerolhyÐec, o arijmìctwn merolhyi¸n, kai e, g, h sunj kh termatismoÔ. . . . . . . . . . . . . 934.4 Sugkritikˆ apotelèsmata gia to prìblhma tou 4 − 2 − 4 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . 944.5 Sugkritikˆ apotelèsmata gia to prìblhma tou 8 − 3 − 8 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . 954.6 Sugkritikˆ apotelèsmata gia to prìblhma tou 16 − 4 − 16 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . 974.7 Sugkritikˆ apotelèsmata gia to prìblhma tou 32 − 5 − 32 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . 984.8 Sugkritikˆ apotelèsmata gia to prìblhma tou 64 − 6 − 64 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . . . 9917

184.9 Sugkritikˆ apotelèsmata gia to prìblhma tou 128 − 7 − 128 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . 1004.10 Sugkritikˆ apotelèsmata gia to prìblhma tou 256 − 8 − 256 Kwdikopoiht/Apokwdikopoiht . . . . . . . . . . . . . . . . . . . . . . . . . 1014.11 Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034.12 Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc arijm¸n . 1064.13 Sugkritikˆ apotelèsmata gia to prìblhma thc taxinìmhshc twn fut¸nthc oikogèneiac Iris . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074.14 Sugkritikˆ apotelèsmata gia to prìblhma thc optik c anagn¸rishc qeirìgrafwnarijm¸n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1094.15 Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc fwnhèntwn 1105.1 Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITE . . 1235.2 Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bit . . . 1235.3 Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1245.4 Sugkritikˆ apotelèsmata gia to prìblhma thc prosèggishc suneqoÔctrigwnometrik c sunˆrthshc. . . . . . . . . . . . . . . . . . . . . . . . 1255.5 Sugkritikˆ apotelèsmata gia to prìblhma thc Taxinìmhshc Karkinik¸nKuttˆrwn Egkefˆlou metaxÔ tou emprìsjia trofodotoÔmenou neurwnikoÔdiktÔou me 7 neur¸nec (MLP) sto krufì epÐpedo kai tou pijanotikoÔneurwnikoÔ diktÔou (PNN). . . . . . . . . . . . . . . . . . . . . 129Aþ.1 Ta prìtupa ekpaÐdeushc thc IsotimÐac twn 3-bit. . . . . . . . . . . . . 135Aþ.2 Ta prìtupa ekpaÐdeushc 4 − 2 − 4 Kwdikopoiht /Apokwdikopoiht . . 137Aþ.3 Ta prìtupa ekpaÐdeushc 8 − 3 − 8 Kwdikopoiht /Apokwdikopoiht . . 137

Mèroc IEisagwgh19

Kefalaio 1Teqnhtˆ Neurwnikˆ DÐktuaTa Teqnhtˆ Neurwnikˆ DÐktua (TND) èqoun empneusteÐ apì thn diapÐstwsh ìti o anjr¸pinocegkèfaloc kˆnei upologismoÔc me èna entel¸c diaforetikì trìpo apì toucsumbatikoÔc yhfiakoÔc upologistèc. O anjr¸pinoc egkèfaloc eÐnai ftiagmènoc apìèna terˆstio dÐktuo apì upologistikˆ stoiqeÐa, pou kaloÔntai neur¸nec, zeugarwmèname aisjht riouc dèktec kai epidrastèc. O mèsoc anjr¸pinoc egkèfaloc upologÐzetaiìti perièqei perÐpou 100 disekatommÔria kÔttara poikÐlou eÐdouc. 'Enac neur¸nac eÐnaièna eidikì kÔttaro pou metafèrei èna hlektrikì s ma. Upˆrqoun perÐpou 10 disekatommÔrianeur¸nec ston anjr¸pino egkèfalo. Ta enapomeÐnanta 90 disekatommÔriakÔttara onomˆzontai neurogloiakˆ koll¸dh kÔttara, kai autˆ uphretoÔn wc kÔttaraupost rixhc gia touc neur¸nec. Oi neur¸nec allhlepidroÔn mèsw epaf¸n pouonomˆzontai sunˆyeic. Katˆ mèso ìro kˆje neur¸nac lambˆnei s mata mèsw qiliˆdwnsunˆyewn.O egkèfaloc organ¸nei autìn ton terˆstio arijmì apì neur¸nec se èna mazikìparˆllhlo perÐploko dÐktuo, sto opoÐo autoÐ oi neur¸nec allhlepidroÔn metaxÔ toucdunamikˆ gia na parˆgoun èna panÐsquro epexergast plhroforÐac. Tupikˆ, oi neur¸neceÐnai pènte e¸c èxi tˆxeic megèjouc argìteroi apì tic logikèc pÔlec puritÐou.O montèrnoc upologist c eÔkola upertereÐ tou anjr¸pou ston programmatismì epanalhptik¸nupologism¸n. Wstìso, h katanìhsh omilÐac se pragmatikì qrìno kai hoptik antÐlhyh, tic opoÐec o ˆnjrwpoc ulopoieÐ qwrÐc kìpo, eÐnai akìmh pèra apìthn ikanìthta twn yhfiak¸n upologist¸n. Gia parˆdeigma, o anjr¸pinoc egkèfalocepitugqˆnei anagn¸rish gnwst¸n pros¸pwn se ˆgnwsta peribˆllonta se perÐpou 100me 200 qiliostˆ tou deuterolèptou, en¸ h Ðdia leitourgÐa se èna sumbatikì upologistja qreiazìtan arketèc hmèrec gia na ektelesteÐ.O lìgoc pou o anjr¸pinoc egkèfaloc uperisqÔei ofeÐletai sthn ekplhktik domtou kai sthn ikanìthtˆ tou na sqhmatÐzei touc dikoÔc tou kanìnec mèsw miac diadikasÐacgnwst c wc empeirÐa. H empeirÐa qtÐzetai me thn pˆrodo tou qrìnou. H pio dramatik21

22 Teqnhta Neurwnika Diktuaanˆptux thc lambˆnei q¸ra katˆ ta duo pr¸ta èth apì thn gènnhsh, allˆ suneqÐzetaiki argìtera.To anaptussìmeno neurikì sÔsthma qarakthrÐzetai apì mia idiìthta pou onomˆzetaiplastikìthta, h opoÐa tou epitrèpei na prosarmìzetai sto peribˆllon. H plastikìthtaeÐnai ousi¸dhc sth leitourgÐa twn neur¸nwn wc monˆdec epexergasÐac plhrofori¸nston anjr¸pino egkèfalo. Me ton Ðdio trìpo h plastikìthta emfanÐzetai na eÐnaiousi¸dhc kai sta neurwnikˆ dÐktua me teqnhtoÔc neur¸nec. 'Ena neurwnikì dÐktuosthn aploÔsterh morf tou eÐnai mia mhqan pou eÐnai sqediasmènh na prosomoi¸neiton trìpo me ton opoÐo o egkèfaloc ekteleÐ mia sugkekrimènh ergasÐa. Ta neurwnikˆdÐktua gia na mporèsoun na ektelèsoun qr simouc upologismoÔc qrhsimopoioÔn miadiadikasÐa pou onomˆzetai mˆjhsh. Gia na epitÔqoun kal apìdosh, qrhsimopoioÔnèna terˆstio arijmì diasundedemènwn apl¸n upologistik¸n kuttˆrwn pou onomˆzontaineur¸nec. SÔmfwna me ta parapˆnw, èna neurwnikì dÐktuo orÐzetai wc ex c:Orismìc 1.1. 'Ena neurwnikì dÐktuo eÐnai ènac mazikìc parˆllhloc katanemhmènocepexergast c apoteloÔmenoc apì aplèc epexergastikèc monˆdec, touc neur¸nec, pouèqoun th fusik rop na apojhkeÔoun empeirik gn¸sh kai na th diajètoun proc qr sh.Moiˆzoun me ton egkèfalo se dÔo shmeÐa:1. To dÐktuo apoktˆ thn gn¸sh apì to peribˆllon mèsw miac diadikasÐac mˆjhshc.2. Oi sÔndesmoi metaxÔ twn neur¸nwn pou onomˆzontai sunaptikˆ bˆrh aplˆ bˆrh,qrhsimopoioÔntai gia thn apoj keush thc gn¸shc.H diadikasÐa pou qrhsimopoieÐtai gia thn ektèlesh thc diadikasÐac mˆjhshc kaleÐtaialgìrijmoc ekpaÐdeushc. O algìrijmoc ekpaÐdeushc ousiastikˆ tropopoieÐ ta bˆrh touneurwnikoÔ diktÔou me mejodikì trìpo gia thn epÐteuxh tou epijumhtoÔ sqediastikoÔstìqou.1.1 Ofèlh twn Neurwnik¸n DiktÔwnH qr sh twn neurwnik¸n diktÔwn prosfèrei tic akìloujec qr simec idiìthtec kaidunatìthtec:1. Mh grammikìthta. 'Enac teqnhtìc neur¸nac mporeÐ na eÐnai grammikìc mhgrammikìc. 'Ena neurwnikì dÐktuo apoteloÔmeno apì mh grammikoÔc neur¸neceÐnai mh grammikì. Epiplèon, h mh grammikìthta eÐnai idiaÐterh upì thn ènnoia ìtieÐnai katanemhmènh se olìklhro to dÐktuo. H mh grammikìthta eÐnai mia exairetikˆshmantik idiìthta, eidikìtera ìtan o fusikìc mhqanismìc pou eÐnai upeÔjunocgia thn dhmiourgÐa tou s matoc eisìdou eÐnai ek fÔsewc mh grammikìc.

Eisagwgh 232. Susqètish eisìdou exìdou. H epiblepìmenh mˆjhsh, h opoÐa apoteleÐ è-na dhmofilèc prìtupo mˆjhshc, tropopoieÐ ta sunaptikˆ bˆrh tou neurwnikoÔdiktÔou qrhsimopoi¸ntac èna sÔnolo apì prìtupa ekpaÐdeushc. Kˆje prìtupoapoteleÐtai apì èna monadikì s ma eisìdou kai thn antÐstoiqh epijumht èxodo.Ta sunaptikˆ bˆrh tou diktÔou tropopoioÔntai ètsi ¸ste h diaforˆ metaxÔthc epijumht c exìdou kai thc pragmatik c exìdou pou parˆgete apì to s maeisìdou na elaqistopoieÐtai sÔmfwna me èna statistikì krit rio. H ekpaÐdeushtou diktÔou suneqÐzetai gia pollˆ prìtupa ekpaÐdeushc mèqri to dÐktuo na ftˆseise mia stajer katˆstash ìpou h tropopoÐhsh twn sunaptik¸n bar¸n namhn eÐnai plèon shmantik . 'Etsi to dÐktuo majaÐnei apì ta prìtupa ekpaÐdeushcdhmiourg¸ntac mia susqètish eisìdou exìdou gia to dojèn prìblhma.3. Prosarmostikìthta. Ta neurwnikˆ dÐktua èqoun thn ikanìthta na prosarmìzounta sunaptikˆ touc bˆrh ìtan to peribˆllon touc allˆzei. Pio sugkekrimèna,èna neurwnikì dÐktuo pou ekpaideÔetai se èna sugkekrimèno peribˆllonmporeÐ eÔkola na epanekpaideuteÐ ¸ste na antimetwpÐsei mikrèc allagèc sto peribˆllon.Epiplèon, ìtan to dÐktuo leitourgeÐ se mh stajerì peribˆllon, mporeÐna sqediasteÐ ètsi ¸ste na prosarmìzei ta sunaptikˆ tou bˆrh se pragmatikìqrìno.4. Apìkrish basismènh se endeÐxeic. 'Ena neurwnikì dÐktuo mporeÐ na sqediasteÐètsi ¸ste na parèqei plhroforÐec ìqi mìno gia thn taxinìmhsh twn eisìdwnse klˆseic, allˆ kai gia ton bajmì empistosÔnhc aut c thc taxinìmhshc. HteleutaÐa plhroforÐa mporeÐ na qrhsimopoihjeÐ gia thn taxinìmhsh nèwn agn¸stwnprotÔpwn katˆ thn diˆrkeia thc ekpaÐdeushc.5. SunafeÐc plhroforÐec. H gn¸sh antiproswpeÔetai apì thn dom kai thnkatˆstash energopoÐhshc tou neurwnikoÔ diktÔou. Kˆje neur¸nac tou diktÔoueÐnai endeqomènwc ephreasmènoc apì thn olik drasthriìthta ìlwn twn upìloipwnneur¸nwn tou diktÔou. Sunep¸c, sunafeÐc plhroforÐec antimetwpÐzontai mefusikì trìpo apì to neurwnikì dÐktuo.6. Anoq se sfˆlmata. 'Ena neurwnikì dÐktuo pou èqei ulopoihjeÐ se hlektronikmorf ulikoÔ (hardware) èqei thn dunatìthta na eÐnai eggen¸c anektikì sesfˆlmata, me thn ènnoia ìti h apìdosh tou mei¸nete omalˆ kˆtw apì antÐxoecsunj kec leitourgÐac. Gia parˆdeigma, an ènac neur¸nac tou diktÔou uposteÐzhmiˆ, h apìdosh tou diktÔou ja meiwjeÐ poiotikˆ. Wstìso, lìgw thc katanemhmènhcfÔsewc thc plhroforÐac pou eÐnai apojhkeumènh sto dÐktuo, h zhmiˆja prèpei na eÐnai ektetamènh gia na parathrhjoÔn shmantikˆ sfˆlmata sthnapìkrish tou diktÔou.7. UlopoÐhsh VLSI. H mazikˆ parˆllhlh fÔsh tou neurwnikoÔ diktÔou to kajistˆikanì gia gr gorouc upologismoÔc. Autì to qarakthristikì kajistˆ taneurwnikˆ dÐktua idanikˆ gia ulopoÐhsh se ulikì me qr sh thc teqnologÐac oloklrwshc polÔ megˆlhc klÐmakac (VLSI). Mia shmantik aret thc ulopoÐhshc

24 Teqnhta Neurwnika DiktuaVLSI eÐnai ìti parèqei thn dunatìthta na sullambˆnei pragmatikˆ polÔplokhsumperiforˆ me èna ierarqikì trìpo.8. OmoiomorfÐa anˆlushc kai sqediasmoÔ. Ta neurwnikˆ dÐktua moirˆzontaiÐdia shmeiografÐa se ìlec tic efarmogèc touc. Aut h idiìthta ekdhl¸netaime diaforetikoÔc trìpouc:– Oi neur¸nec apoteloÔn èna koinì sustatikì se ìla ta neurwnikˆ dÐktua.– Ta koinˆ qarakthristikˆ dÐnoun thn dunatìthta gia koinoÔc algìrijmoucekpaÐdeushc se diaforetikèc efarmogèc twn neurwnik¸n diktÔwn.9. Biologik analogÐa. O sqediasmìc twn neurwnik¸n diktÔwn eÐnai empneusmènocapì ton anjr¸pino egkèfalo. Oi neurobiolìgoi suqnˆ qrhsimopoioÔnta neurwnikˆ dÐktua wc èna ergaleÐo gia thn ex ghsh neurobiologik¸n fainomènwn.Apì thn ˆllh meriˆ, oi mhqanikoÐ qrhsimopoioÔn thn neurobiologÐa giana afomoi¸soun nèec idèec pou ja touc bohj soun sthn epÐlush polÔplokwnproblhmˆtwn.1.2 BiologikoÐ Neur¸necUpˆrqoun polloÐ diaforetikoÐ tÔpoi biologik¸n neur¸nwn. Oi basikèc idiìthtèc toucìmwc moirˆzontai apì polloÔc neur¸nec. 'Enac neur¸nac (Eikìna 1.1) apoteleÐte apìta parakˆtw kÔria mèrh:1. Pur na2. S¸ma3. DendrÐtec4. Axonikì lofÐsko5. 'Axona6. TermatikoÐ bolboÐKˆje neur¸nac eÐnai èna kÔttaro to opoÐo qrhsimopoieÐ bioqhmikèc antidrˆseic gia thl yh, epexergasÐa kai metˆdosh plhrofori¸n.To dendritikì dèntro enìc neur¸na eÐnai sundedemèno me qiliˆdec geitonikoÔc neur¸nec.'Otan ènac apì autoÔc touc neur¸nec ekpèmpei s ma, èna arnhtikì jetikìfortÐo lambˆnetai apì ènan dendrÐth. Ta dunamikˆ ìlwn twn lambanìmenwn fortÐwn

Eisagwgh 25prostÐjentai mèsw miac diadikasÐac qwrik c kai qronik c prìsjeshc. H qwrik prìsjeshlambˆnei q¸ra ìtan pollˆ adÔnama s mata metatrèpontai se èna megˆlo, en¸h qronik prìsjesh metatrèpei mia seirˆ apì taqeÐa adÔnama s mata apì mia phg seèna megˆlo s ma. H sunolik eÐsodoc sth sunèqeia eisèrqetai sto s¸ma. To s¸ma kaio pur nac den paÐzoun kanèna rìlo sthn epexergasÐa to eiserqomènwn kai exerqomènwndedomènwn. H leitourgÐa touc periorÐzetai sthn sunt rhsh tou kuttˆrou ¸ste oneur¸nac na paramènei leitourgikìc. To mèroc tou s¸matoc to opoÐo eÐnai shmantikìgia thn diˆdosh tou s matoc eÐnai o axonikìc lofÐskoc. An h sunolik eÐsodoc eÐnaimegalÔterh apì thn tim tou katwflioÔ tou axonikoÔ lofÐskou, tìte o neur¸nac ekpèmpeikai to s ma tou metadÐdetai ston ˆxona. To dunamikì thc exìdou eÐnai stajerìeÐte an to s ma eisìdou eÐnai mìlic epˆnw apì thn tim tou katwflioÔ, ekatì forècpio pˆnw. To dunamikì tou s matoc exìdou den ephreˆzetai apì tic pollèc diaklad¸seictou ˆxona kai ftˆnei stouc termatikoÔc bolboÔc me thn Ðdia èntash pou eÐqe stonaxonikì lofÐsko.Sq ma 1.1: Sqèdio enìc biologikoÔ neur¸na.'Enac neur¸nac eÐnai egkleismènoc apì thn membrˆnh tou. H diègersh enìc neur¸naephreˆzei ˆllouc neur¸nec mèsw twn sundèsewn twn termatik¸n bolb¸n me tic membrˆnectwn ˆllwn neur¸nwn. Tètoiec sundèseic onomˆzontai sunˆyeic (Eikìna 1.2).H membrˆnh tou termatikoÔ bolboÔ onomˆzetai prosunaptik membrˆnh. H membrˆnhpˆnw sthn opoÐa epidrˆ o termatikìc bolbìc onomˆzetai metasunaptik membrˆnh. Omikrìc q¸roc metaxÔ twn prosunaptik¸n kai twn metasunaptik¸n membran¸n onomˆzetaisunaptik sqism . H epikoinwnÐa metaxÔ twn neur¸nwn dieukolÔnetai apì thnapeleujèrwsh mikr¸n pakètwn qhmik¸n mèsa sto kenì autì. 'Enac neur¸nac mporeÐna epikoinwneÐ me pˆnw apì 100000 ˆllouc neur¸nec.'Otan èna dunamikì drˆshc ftˆsei ston termatikì bolbì thc prosunaptik c membrˆnhc,proxeneÐ mia allag dunamikoÔ apènanti sthn metasunaptik membrˆnh. H ˆmesh

26 Teqnhta Neurwnika DiktuaaÐtia thc allag c sto hlektrikì dunamikì thc metasunaptik c membrˆnhc eÐnai arqikˆqhmik kai ìqi hlektrik . H hlektrik ¸sh pou metadÐdetai diamèsw kˆje sunaptikoÔkenoÔ eÐnai qhmik¸c parakinoÔmenh kai elegqìmenh.Sq ma 1.2: Metˆdosh s matoc prosunaptikoÔ ˆxona mèsw sunaptik c sqism c ston metasunaptikìneur¸na.'Ena dunamikì drˆshc pou diadÐdetai ston termatikì bolbì proxeneÐ thn apeleujèrwshqhmik¸n ousi¸n pou onomˆzontai neurodiabibastèc apì mikrˆ pakèta (sunaptikˆkustÐdia) sthn prosunaptik membrˆnh. Oi neurodiabibastèc diaqèontai diamèsou thcsunaptik c sqism c sth metasunaptik membrˆnh kai dhmiourgoÔn mia allag sto hlektrikìdunamikì apènanti sth metasunaptik membrˆnh.1.3 TeqnhtoÐ Neur¸nec'Enac teqnhtìc neur¸nac eÐnai mia monˆda epexergasÐac plhroforÐac h opoÐa eÐnai jemeli¸dhcgia thn leitourgÐa enìc neurwnikoÔ diktÔou. To diˆgramma thc Eikìnac 1.3deÐqnei to montèlo enìc teqnhtoÔ neur¸na, o opoÐoc sqhmatÐzei thn bˆsh gia ton sqediasmìtwn teqnht¸n neurwnik¸n diktÔwn. Ed¸ anagnwrÐzontai trÐa basikˆ stoiqeÐatwn neurwnik¸n montèlwn:1. 'Ena sÔnolo apì sunˆyeic, kajemÐa apì tic opoÐec qarakthrÐzetai apì èna bˆroc.Sugkekrimèna, èna s ma x j sthn eÐsodo thc sÔnayhc j sundedemènh me ton neur¸nak pollaplasiˆzetai me to sunaptikì bˆroc w kj . EÐnai shmantikì na gÐnei miashmeÐwsh gia ton trìpo me ton opoÐo grˆfontai oi upodeÐktec twn sunaptik¸nbar¸n w kj . O pr¸toc upodeÐkthc anafèretai ston neur¸na upì exètash kai odeÔteroc upodeÐkthc anafèretai sto tèloc thc eisìdou thc sÔnayhc sthn opoÐaanafèretai to bˆroc. AntÐjeta me mia sÔnayh ston egkèfalo, to sunaptikì bˆroc

Eisagwgh 27enìc teqnhtoÔ neur¸na mporeÐ na brÐsketai se èna pedÐo pou perièqei arnhtikèc,kaj¸c epÐshc kai jetikèc timèc.2. 'Enac ajroist c gia thn ˆjroish twn shmˆtwn eisìdou, bebarhmènwn apì tic antÐstoiqecsunˆyeic tou neur¸na. Oi leitourgÐec pou perigrˆfontai ed¸ sunistoÔnèna grammikì sunduast .3. Mia sunˆrthsh energopoÐhshc gia ton periorismì tou plˆtouc thc exìdou enìcneur¸na. H sunˆrthsh energopoÐhshc epÐshc anafèretai wc sunˆrthsh poltopoÐhshcepeid poltopoieÐ (periorÐzei) to epitrepìmeno plˆtoc tou pedÐou tous matoc exìdou se kˆpoia peperasmènh tim . Tupik¸c, to kanonikopoihmènoplˆtoc tou pedÐou thc exìdou enìc neur¸na grˆfetai wc to kleistì monadiaÐodiˆsthma [0, 1] enallaktikˆ [−1, 1].To neurwnikì montèlo thc Eikìnac 1.3 epÐshc perilambˆnei mia exwterikˆ efarmozìmenhmerolhyÐa, pou dhl¸netai wc b k . H merolhyÐa b k èqei epÐdrash sthn aÔxhsh sthnmeÐwsh thc kajar c eisìdou thc sunˆrthshc energopoÐhshc, exart¸menh apì pìte eÐnaijetik arnhtik , antÐstoiqa.Μεροληψίαb kx 1w k1x 2......w k2υ k∑ f(·)Έξοδο̋y kx kΕίσοδοιw kmΒάρηΑθροιστικήσυνάρτησηΣυνάρτησηενεργοποίηση̋Sq ma 1.3: Mh grammikì montèlo enìc neur¸na.

28 Teqnhta Neurwnika DiktuaSe majhmatikoÔc ìrouc, mporeÐ na perigrafeÐ ènac neur¸nac k grˆfontac to akìloujozeÔgoc exis¸sewn:kaiu k =m∑w kj x j (1.1)j=1y k = f(u k + b k ) (1.2)ìpou x 1 , x 2 , . . . , x m eÐnai ta s mata eisìdou, w k1 , w k2 , . . . , w km eÐnai ta sunaptikˆ bˆrhtou neur¸na k, u k eÐnai h èxodoc tou grammikoÔ susqetist lìgw twn shmˆtwneisìdou, b k eÐnai h merolhyÐa, f(·) eÐnai h sunˆrthsh energopoÐhshc kai y k eÐnai tos ma exìdou tou neur¸na. H qr sh thc merolhyÐac b k èqei wc skopì thn efarmogenìc omoparallhlikoÔ metasqhmatismoÔ sthn èxodo u k tou grammikoÔ sunduast stomontèlo thc Eikìnac 1.3, ìpwc faÐnetai apì thnv k = u k + b k (1.3)Επηρεασμένοτοπικόπεδίο υ kΜεροληψία b k >0b k =0b k

Eisagwgh 29grammikoÔ susqetist , tropopoieÐtai me èna trìpo pou faÐnetai sthn Eikìna 1.4. Shmei¸netaiìti wc èna apotèlesma autoÔ tou omoparallhlikoÔ metasqhmatismoÔ, togrˆfhma tou v k katˆ tou u k den pernˆei plèon apì thn arq twn suntetagmènwn.H merolhyÐa b k eÐnai mia exwterik parˆmetroc tou neur¸na k. IsodÔnama, mporeÐna diatupwjeÐ o sunduasmìc twn exis¸sewn (1.1) kai (1.3) ìpwc akoloujeÐ:v k =m∑w kj x j (1.4)j=0kaiy k = f(v k ) (1.5)Sthn exÐswsh (1.4) prostèjhke mia nèa sÔnayh. H eÐsodoc thc eÐnaix 0 = +1 (1.6)kai to bˆroc thcw k0 = b k (1.7)Gi' autì to lìgo, mporeÐ na epanadiatupwjeÐ to montèlo tou neur¸na k ìpwc sthnEikìna 1.5. Se aut thn eikìna h epÐdrash thc merolhyÐac exhgeÐtai kˆnontac dÔoprˆgmata:(1) prosjètontac èna nèo s ma eisìdou stajerì sthn tim +1, kai(2) prosjètontac èna nèo sunaptikì bˆroc Ðso me thn merolhyÐa b k .Parìlo pou ta montèla twn Eikìnwn 1.3 kai 1.5 eÐnai diaforetikˆ sthn emfˆnish, eÐnaimajhmatikˆ isodÔnama.

30 Teqnhta Neurwnika DiktuaW k0 =b k (μεροληψία)x 0 =+1w k0x 1w k1x 2......w k2υ k∑ f(·)Έξοδο̋y kx kΕίσοδοιw kmΒάρη καιμεροληψίαΑθροιστικήσυνάρτησηΣυνάρτησηενεργοποίηση̋Sq ma 1.5: 'Ena ˆllo mh grammikì montèlo neur¸na.1.3.1 TÔpoi Sunart sewn EnergopoÐhshcH sunˆrthsh energopoÐhshc, pou dhl¸netai apì thn f(v), orÐzei thn èxodo enìc neur¸nase sqèsh me to ephreasmèno topikì pedÐo. Sto shmeÐo autì parousiˆzontai treicbasikoÐ tÔpoi sunart sewn energopoÐhshc.Sunˆrthsh KatwflioÔGi' autì ton tÔpo sunˆrthshc energopoÐhshc, pou perigrˆfetai sthn Eikìna 1.6, èqoume:{1 an v ≥ 0f(v) =(1.8)0 an v < 0Sthn bibliografÐa thc mhqanik c, aut h morf miac sunˆrthshc katwflioÔ koinˆanafèretai wc mia Heaviside sunˆrthsh. AntistoÐqwc, h èxodoc enìc neur¸na k poukatèqei mia tètoia sunˆrthsh katwflioÔ, ekfrˆzetai wc{1 an v k ≥ 0y k =(1.9)0 an v k < 0

Eisagwgh 31ìpou v k eÐnai to ephreasmèno topikì pedÐo tou neur¸na, dhladm∑v k = w kj x j + b k (1.10)j=1'Enac tètoioc neur¸nac anafèretai sthn bibliografÐa wc to montèlo McCulloch–Pittsse anagn¸rish thc prwtopìrac douleiˆc pou ègine apì touc McCulloch kai Pitts(1943). Se autì to montèlo, h èxodoc enìc neur¸na paÐrnei thn tim 1 an to ephreasmènotopikì pedÐo tou neur¸na eÐnai mh arnhtikì, kai 0 diaforetikˆ. Autìc oisqurismìc perigrˆfei thn idiìthta ìla tÐpota tou montèlou McCulloch–Pitts.10.8f(υ)0.60.40.20−2 −1.5 −1 −0.5 0 0.5 1 1.5 2υSq ma 1.6: Sunˆrthsh KatwflioÔ.Katˆ Tm mata Grammik SunˆrthshGia thn katˆ tm mata grammik sunˆrthsh pou perigrˆfetai sthn Eikìna 1.7 èqoume⎧⎪⎨ 1, v ≥ + 1 2f(v) = v, + 1 2 ⎪⎩> v > − 1 (1.11)20, v ≤ − 1 2ìpou o parˆgontac enÐsqushc mèsa sthn grammik perioq thc drˆshc upotÐjetai ìtieÐnai monadiaÐoc. Autìc o tÔpoc sunˆrthshc energopoÐhshc mporeÐ na jewrhjeÐ wc miaprosèggish enìc mh grammikoÔ enisqut . Oi akìloujec dÔo peript¸seic mporoÔn najewrhjoÔn wc eidikèc morfèc thc katˆ tm mata grammik c sunˆrthshc:• 'Enac grammikìc susqetist c parousiˆzetai an h grammik perioq drˆshc diathreÐtaiqwrÐc na trèqei se koresmì.• H katˆ tm mata grammik sunˆrthsh anˆgetai se mia sunˆrthsh katwflioÔ an oparˆgontac enÐsqushc thc grammik c perioq c gÐnetai apeÐrwc megˆloc.

32 Teqnhta Neurwnika Diktua1.210.8f(υ)0.60.40.20−2 −1.5 −1 −0.5 0 0.5 1 1.5 2υSq ma 1.7: Katˆ Tm mata Grammik Sunˆrthsh.Sigmoeid c SunˆrthshH sigmoeid c sunˆrthsh, thc opoÐac to grˆfhma èqei sq ma s, eÐnai makrˆn o piokoinìc tÔpoc sunˆrthshc energopoÐhshc pou qrhsimopoieÐtai sthn kataskeu teqnht¸nneurwnik¸n diktÔwn. OrÐzetai wc mia austhrˆ aÔxousa sunˆrthsh h opoÐa epideiknÔeimia {qaritwmènh} isorropÐa metaxÔ thc mh grammik c kai grammik c sumperiforˆc. 'Enaparˆdeigma thc sigmoeid c sunˆrthshc eÐnai h logistik sunˆrthsh, pou orÐzetai apìf(v) =11 + exp(−αv)(1.12)ìpou α eÐnai h parˆmetroc klÐshc thc sigmoeidoÔc sunˆrthshc. Diaforopoi¸ntac thnparˆmetro α, epitugqˆnoume sigmoeideÐc sunart seic me diaforetikèc klÐseic, ìpwc faÐnetaisthn Eikìna 1.8. Sthn pragmatikìthta, h klÐsh sthn arq twn suntetagmènwneÐnai Ðsh me α/4. Sto ìrio, kaj¸c h parˆmetroc klÐshc plhsiˆzei sto ˆpeiro, h sigmoeidc sunˆrthsh gÐnetai apl¸c mia sunˆrthsh katwflioÔ. En¸ mia sunˆrthsh katwflioÔlambˆnei thn tim 0 1, mia sigmoeid c sunˆrthsh lambˆnei èna suneqèc pedÐo tim¸napì 0 e¸c to 1. Shmei¸netai, epÐshc, ìti h sigmoeid c sunˆrthsh eÐnai diaforÐsimh, en¸h sunˆrthsh katwflioÔ den eÐnai.Oi sunart seic energopoÐhshc pou orÐsthkan stic exis¸seic (1.8), (1.11) kai (1.12)diakumaÐnontai apì to 0 sto +1. EÐnai epijumhtì merikèc forèc na èqoume to pedÐo thcsunˆrthshc energopoÐhshc apì to −1 sto +1. Sthn perÐptwsh aut , h sunˆrthshenergopoÐhshc lambˆnei èna antisummetrikì tÔpo se sqèsh me thn arq twn suntetagmènwn.Dhlad , h sunˆrthsh energopoÐhshc eÐnai mia peritt sunˆrthsh tou ephreasmènoutopikoÔ pedÐou. Sugkekrimèna, h sunˆrthsh katwflioÔ thc exÐswshc (1.8)

Eisagwgh 331.210.8f(υ)0.60.40.20−10 −8 −6 −4 −2 0 2 4 6 8 10υSq ma 1.8: Sigmoeid c Sunˆrthsh.orÐzetai t¸ra wc⎧⎪⎨ 1, an v > 0f(v) = 0, an v = 0⎪⎩−1, an v < 0(1.13)h opoÐa koinˆ anafèretai wc sunˆrthsh pros mou (signum function). Gia ton antÐstoiqotÔpo miac sigmoeidoÔc sunˆrthshc mporeÐ na qrhsimopoihjeÐ h sunˆrthshuperbolik c efaptomènhc (hyperbolic tangent function), pou orÐzetai apìf(v) = tanh(v) (1.14)Epitrèpontac mia sunˆrthsh sigmoeidoÔc tÔpou na lˆbei arnhtikèc timèc ìpwc perigrˆfhkeapì thn exÐswsh (1.14) èqei analutikˆ pleonekt mata.1.4 Arqitektonikèc DiktÔwnO trìpoc me ton opoÐo oi neur¸nec enìc neurwnikoÔ diktÔou eÐnai domhmènoi eÐnaistenˆ sundedemènoc me ton algìrijmo mˆjhshc pou qrhsimopoieÐtai gia thn ekpaÐdeushtou diktÔou. Upˆrqoun pollèc arqitektonikèc neurwnik¸n diktÔwn, allˆ se aut thdiatrib ja asqolhjoÔme me ta emprìsjia trofodotoÔmena neurwnikˆ dÐktua.1.4.1 MonoepÐpeda Emprìsjia TrofodotoÔmena DÐktuaSe èna neurwnikì dÐktuo oi neur¸nec organ¸nontai me thn morf epipèdwn. SthnaploÔsterh morf enìc diktÔou, èqoume èna epÐpedo eisìdou pou probˆlletai pˆnw se

34 Teqnhta Neurwnika Diktuaèna epÐpedo exìdou apì neur¸nec, allˆ ìqi to antÐstrofo. Me ˆlla lìgia, autì todÐktuo eÐnai emprìsjia trofodotoÔmeno. 'Ena tètoio dÐktuo parousiˆzetai sthn Eikìna1.9 gia thn perÐptwsh twn m neur¸nwn sto epÐpedo eisìdou kai twn n neur¸nwn stoepÐpedo exìdou. 'Ena tètoio dÐktuo kaleÐtai monoepÐpedo dÐktuo, me ton qarakthrismì{monoepÐpedo} na anafèretai sto epÐpedo exìdou twn neur¸nwn. Den upologÐzoume toepÐpedo eisìdou giatÐ den ekteleÐtai kanènac upologismìc ekeÐ.Μεροληψίανευρώνων εξόδου1Είσοδοιx 1x 2x 3y 1y 2y 3......x my nΕξοδοιΕπίπεδοΕισόδουΕπίπεδοΕξόδουSq ma 1.9: Emprìsjia trofodotoÔmeno dÐktuo me èna monì epÐpedo apì neur¸nec.1.4.2 PoluepÐpeda Emprìsjia TrofodotoÔmena DÐktuaTa poluepÐpeda emprìsjia trofodotoÔmena neurwnikˆ dÐktua diakrÐnontai apì thn parousÐaenìc perissotèrwn kruf¸n epipèdwn, twn opoÐwn oi neur¸nec kaloÔntai antistoÐqwckrufoÐ neur¸nec. H leitourgÐa twn kruf¸n neur¸nwn eÐnai na mesolabeÐmetaxÔ thc exwterik c eisìdou kai thc exìdou tou diktÔou me kˆpoio qr simo trìpo.Prosjètontac èna perissìtera krufˆ epÐpeda, to dÐktuo kajÐstatai ikanì na exˆgeistatistikˆ uyhlìterhc tˆxhc. Me mia mˆllon asaf ènnoia to dÐktuo apoktˆ mia olik

Eisagwgh 35ìyh parˆ thn topik tou sundesimìthta lìgw tou epiplèon sunìlou apì sunaptikècsundèseic kai thn epiplèon diˆstash twn neurwnik¸n allhlepidrˆsewn (Churchlandkai Sejnowski, 1992). H ikanìthta twn kruf¸n neur¸nwn na exˆgoun statistikˆ u-yhlìterhc tˆxhc eÐnai idiaitèrwc polÔtimh ìtan to mègejoc tou epipèdou eisìdou eÐnaimegˆlo.To epÐpedo eisìdou tou diktÔou parèqei ta antÐstoiqa stoiqeÐa tou protÔpou energopoÐhshc,ta opoÐa apoteloÔn ta s mata eisìdou pou efarmìzontai stouc neur¸necsto deÔtero epÐpedo (dhlad sto pr¸to krufì epÐpedo). Ta s mata exìdou tou deÔterouepipèdou qrhsimopoioÔntai wc eÐsodoi gia to trÐto epÐpedo, kai autì epanalambˆnetaigia to upìloipo dÐktuo. Tupik¸c oi neur¸nec se kˆje epÐpedo tou diktÔou èqounwc eisìdouc touc ta s mata exìdou tou prohgoÔmenou epipèdou mìno. To sÔnolo twnshmˆtwn exìdou twn neur¸nwn sto epÐpedo exìdou tou diktÔou apoteleÐ thn sunolikapìkrish tou diktÔou gia to pareqìmeno prìtupo energopoÐhshc sto epÐpedo eisìdou.Μεροληψίακρυφών νευρώνωνΜεροληψίανευρώνων εξόδου1 1Είσοδοιx 1x 2x 3y 1y 2y 3.........y nΕξοδοιΕπίπεδοΕισόδουΚρυφόΕπίπεδοΕπίπεδοΕξόδουSq ma 1.10: Pl rwc diasundedemèno emprìsjia trofodotoÔmeno dÐktuo me èna krufì epÐpedokai èna epÐpedo exìdou.O arqitektonikìc grˆfoc thc Eikìnac 1.10 parousiˆzei th diˆtaxh tou poluepÐpedou

36 Teqnhta Neurwnika Diktuaemprìsjia trofodotoÔmenou neurwnikoÔ diktÔou gia thn perÐptwsh enìc mìno krufoÔepipèdou. Gia suntomÐa to dÐktuo sthn Eikìna 1.10 anafèretai wc èna m l n dÐktuogiatÐ èqei m eisìdouc, l krufoÔc neur¸nec kai n neur¸nec exìdou. Wc èna ˆlloparˆdeigma, èna emprìsjia trofodotoÔmeno dÐktuo me m eisìdouc, h 1 neur¸nec stopr¸to krufì epÐpedo, h 2 neur¸nec sto deÔtero krufì epÐpedo, kai q neur¸nec stoepÐpedo exìdou anafèretai wc èna m h 1 h 2 q dÐktuo.To neurwnikì dÐktuo sthn Eikìna 1.10 lègetai ìti eÐnai pl rwc diasundedemèno methn ènnoia ìti kˆje kìmboc se kˆje epÐpedo tou diktÔou sundèetai me kˆje ˆllo kìmbotou diplanoÔ emprìsjiou epipèdou. An, wstìso, kˆpoiec apì tic sunaptikèc sundèseicapousiˆzoun apì to dÐktuo, lème ìti to dÐktuo eÐnai merik¸c diasundedemèno.1.5 H IstorÐa twn Teqnht¸n Neurwnik¸n DiktÔwnH nèa epoq twn neurwnik¸n diktÔwn xekÐnhse me thn prwtoporiak douleÐa twnMcCulloch kai Pitts (1943) [50]. Sthn ergasÐa aut perigrˆfoun mia logik anˆlushtwn neurwnik¸n diktÔwn pou sunènwse tic melètec thc neurofusiologÐac kai thmajhmatik logik . To tupikì touc montèlo enìc neur¸na upotèjhke ìti akoloujeÐènan {ìla- -tÐpota} kanìna. Me èna ikanì arijmì apì tètoiec monˆdec, kai me swstˆefarmosmènec sunaptikèc sundèseic, oi McCulloch kai Pitts èdeixan ìti èna tètoiodÐktuo ja mporoÔse, kat' arq n, na upologÐsei opoiad pote upologÐsimh sunˆrthsh.Autì tan èna polÔ shmantikì apotèlesma kai me autì, genik¸c, sumfwn jhke ìtigenn jhkan oi arqèc twn neurwnik¸n diktÔwn kai thc teqnht c nohmosÔnhc.H epìmenh megˆlh anˆptuxh sta neurwnikˆ dÐktua rje to 1949 me th dhmosÐeushtou biblÐou tou Hebb The Organization of Behavior [32], sto opoÐo parousiˆsthke miaanamfÐbolh anaforˆ enìc fusiologikoÔ kanìna mˆjhshc gia sunaptik tropopoÐhshgia pr¸th forˆ. Eidikìtera, o Hebb prìteine ìti h sundetikìthta tou egkefˆlouallˆzei suneq¸c wc ènac organismìc pou majaÐnei diaforetikèc leitourgikèc ergasÐec,kai ìti oi neurwnikèc sunarmolog seic dhmiourgoÔntai apì tètoiec allagèc. O Hebbakolouj¸ntac mia prohgoÔmenh prìtash apì ton Ramón y Cajál [68] eis gage tot¸ra diˆshmo axÐwma thc mˆjhshc, to opoÐo dhl¸nei ìti h apotelesmatikìthta miacmetablht c sÔnayhc metaxÔ dÔo neur¸nwn auxˆnetai apì thn suneqìmenh energopoÐhshtou enìc neur¸na apì ton ˆllo dia mèsou ekeÐnhc thc sÔnayhc. To biblÐo tou Hebb eÐqepˆra polÔ megˆlh epirro stouc yuqolìgouc, allˆ dustuq¸c eÐqe lÐgh èwc kajìlouepÐdrash sthn koinìthta thc mhqanik c.PerÐpou 15 qrìnia metˆ thn dhmosÐeush thc klassik c ergasÐac twn McCullochkai Pitts, mia nèa prosèggish sto prìblhma thc anagn¸rishc protÔpou eis qjhke apìton Rosenblatt (1958) sthn ergasÐa tou pˆnw sto perceptron, mia nèa mèjodo sthnepiblepìmenh mˆjhsh [71]. To anagnwrismèno katìrjwma thc douleiˆ tou Rosenblatt

Eisagwgh 37tan to eponomazìmeno je¸rhma sÔgklishc tou perceptron, h pr¸th apìdeixh gia toopoÐo perigrˆfhke apì ton Rosenblatt (1960) [72]. To 1960, oi Widrow kai Hoffeis gagan ton algìrijmo twn elaqÐstwn mèswn tetrag¸nwn (least mean square (LMS)algorithm) kai ton qrhsimopoÐhsan gia na diatup¸soun to grammikì prosarmostikìstoiqeÐo Adaline (adaptive linear element) [88]. H diaforˆ metaxÔ tou perceptron kaitou Adaline entopÐzetai sthn diadikasÐa ekpaÐdeushc. 'Ena apì ta pr¸ima ekpaideÔsimaneurwnikˆ dÐktua me epÐpeda me pollaplˆ prosarmostikˆ stoiqeÐa tan h dom Madaline(multiple–adaline) pou protˆjhke apì ton Widrow kai touc majhtèc tou (Widrow,1962 [87]). Katˆ thn diˆrkeia thc klassik c periìdou tou perceptron sthn dekaetÐatou 1960, fainìtan ìti ta neurwnikˆ dÐktua mporoÔsan na kˆnoun ta pˆnta. Allˆ tìterje to biblÐo apì touc Minsky kai Papert (1969) [51], pou qrhsimopoÐhsan majhmatikˆgia na deÐxoun ìti upˆrqoun jemeli¸dec ìria gia to ti mporoÔn na upologÐsoun tamonoepÐpeda perceptron. Se èna sunoptikì kefˆlaio sto poluepÐpeda perceptron,d lwsan ìti den up rqe lìgoc na upojèsoun ìti kˆpoioi apì touc periorismoÔc twnmonoepÐpedwn perceptron mporoÔsan na xeperastoÔn sthn poluepÐpedh èkdosh.H ergasÐa aut suneisèfere me ton èna trìpo me ton ˆllo ston metriasmì touendiafèrontoc gia ta neurwnikˆ dÐktua sthn dekaetÐa tou 1970. PolloÐ apì toucereunhtèc, ektìc apì ekeÐnouc thc yuqologÐac kai twn neuroepisthm¸n, egkatèleiyanthn perioq katˆ thn diˆrkeia aut c thc dekaetÐac. 'Ontwc, mìno mia qoÔfta apì toucpr¸imouc prwtopìrouc diat rhsan thn afosÐws touc sta neurwnikˆ dÐktua.Sthn dekaetÐa tou 1980 èginan shmantikèc suneisforèc sthn jewrÐa kai ton sqediasmìtwn neurwnik¸n diktÔwn se pollˆ mètwpa, kai exaitÐac tou lìgou autoÔ, up rxeanagènnhsh tou endiafèrontoc gia ta neurwnikˆ dÐktua. 'Iswc perissìtero apì kˆjeˆllh dhmosÐeush, h anˆptuxh tou algorÐjmou opisjodromik c diˆdoshc tou sfˆlmatoc(backpropagation) twn Rumelhart, Hinton kai Williams (1986) [73] tan h pio shmantikdhmosÐeush, upeÔjunh gia thn anagènnhsh tou endiafèrontoc sta neurwnikˆdÐktua. Se autì ton Ðdio qrìno, dhmosieÔthke to pasÐgnwsto dÐtomo biblÐo, ParallelDistributed Processing: Explorations in the Microstructures of Cognition pou suntˆqjhkeapì touc Rumelhart kai McClelland [74]. Autì to prìsfato biblÐo eÐqe miamegˆlh epirro sth qr sh thc mˆjhshc opisjodromik c diˆdoshc tou sfˆlmatoc, hopoÐa emfanÐsthke sto prosk nio wc o pio dhmofil c algìrijmoc mˆjhshc gia thnekpaÐdeush twn poluepÐpedwn perceptron. Sthn pragmatikìthta h mˆjhsh opisjodromikc diˆdoshc tou sfˆlmatoc anakalÔfjhke anexˆrthta se dÔo ˆllec perioqècperÐpou ton Ðdio kairì (Parker, 1985 [58], LeCun, 1985 [44]). Metˆ thn anakˆluyhtou algorÐjmou opisjodromik c diˆdoshc tou sfˆlmatoc sta mèsa thc dekaetÐac tou1980, apodeÐqjhke ìti eÐqe perigrafeÐ nwrÐtera apì ton Werbos sthn didaktorik toudiatrib sto Panepist mio Harvard ton AÔgousto to 1974 [86]. H basik idèa thc o-pisjodromik c diˆdoshc tou sfˆlmatoc mporeÐ na entopisteÐ akìmh pio pÐsw sto biblÐoApplied Optimal Control apì touc Bryson kai Ho (1969) [13]. Se teleutaÐa anˆlush,wstìso, h perissìterh anagn¸rish gia ton algìrijmo opisjodromik c diˆdoshc tousfˆlmatoc prèpei na dojeÐ stouc Rumelhart, Hinton kai Williams (1986) [73] gia thnprìtas touc na qrhsimopoihjeÐ sthn mˆjhsh mhqan c kai gia thn epÐdeixh tou trìpou

38 Teqnhta Neurwnika Diktuapou mporeÐ na gÐnei.Ta neurwnikˆ dÐktua èqoun plèon dianÔsei èna makrÔ drìmo apì tic pr¸imec mèrectwn McCulloch kai Pitts. Pragmatikˆ, kajier¸jhkan wc èna perilambˆnwn diaforetikècarqèc jèma me bajièc rÐzec stic neuroepist mec, thn yuqologÐa, ta majhmatikˆ,stic fusikèc epist mec kai thc mhqanologÐac. PolloÐ ereunhtèc plèon asqoloÔntai meta neurwnikˆ dÐktua tìso se jewrhtikì ìso kai praktikì epÐpedo. Den qreiˆzetai naanaferjeÐ ìti eÐnai ed¸ gia na meÐnoun kai na suneqÐsoun na anaptÔssontai se jewrÐa,sqediasmì kai efarmogèc.1.6 Efarmogèc twn Teqnht¸n Neurwnik¸n DiktÔwnS mera, probl mata megˆlhc oikonomik c shmasÐac pou den mporoÔsan na proseggistoÔnprohgoumènwc me kanèna praktikì trìpo, mporoÔn t¸ra na epilujoÔn me tateqnhtˆ neurwnikˆ dÐktua. Probl mata sthn epist mh twn majhmatik¸n, thc fusik c,allˆ kai thc iatrik c, thc mhqanik c kai thc oikonomÐac antimetwpÐzontai me thn qr shtwn neurwnik¸n diktÔwn. H efarmog touc se mia megˆlh poikilÐa problhmˆtwn sepollˆ pedÐa ta èkane polÔ elkustikˆ. EpÐshc, oi grhgorìteroi hlektronikoÐ upologistèckai oi grhgorìteroi algìrijmoi èkanan dunat th qr sh twn neurwnik¸n diktÔwngia thn epÐlush polÔplokwn biomhqanik¸n problhmˆtwn pou mèqri t¸ra apaitoÔsanpolÔ megˆlo upologistikì kìpo.H melèth DARPA gia ta neurwnikˆ dÐktua tou 1988, katagrˆfei poikÐlec efarmogèctwn neurwnik¸n diktÔwn, arqÐzontac apì to prosarmostikì kanˆli exiswt perÐpouto 1984. Aut h suskeu , h opoÐa eÐqe mia kataplhktik emporik epituqÐa, eÐnai ènadÐktuo monoÔ neur¸na pou qrhsimopoieÐtai sta thlefwnikˆ sust mata makrin c apìstashcgia thn stajeropoÐhsh twn fwnhtik¸n shmˆtwn. H anaforˆ DARPA suneqÐzeina katagrˆfei ˆllec emporikèc efarmogèc, sumperilambanomènou enìc mikroÔ anagnwristlèxewn, enìc elegkt diadikasi¸n, enìc taxinomht sìnar kai enìc sust matocanˆlushc kindÔnou.Ta neurwnikˆ dÐktua èqoun efarmosteÐ se pollˆ pedÐa apì tìte pou grˆfhke hanaforˆ DARPA. Sth sunèqeia akoloujeÐ mia lÐsta apì efarmogèc pou anafèrontaisthn bibliografÐa [31].Aerodiasthmik . Uyhl c apìdoshc autìmatoi pilìtoi aeroskaf¸n, exomoiwtècmonopati¸n pt shc, sust mata elègqou aeroskaf¸n, emploutismìc autìmatwnpilìtwn, exomoi¸seic sustatik¸n aeroskaf¸n, aniqneutèc sfalmˆtwn sustatik¸n aeroskaf¸n.AutokinhtobiomhqanÐa. Autìmata sust mata kateÔjunshc autokin twn, analutèceggÔhshc kÐnhshc.

Eisagwgh 39Trapezik LeitourgÐa. Anagn¸stec epitag¸n kai ˆllwn eggrˆfwn, apotimhtècefarmog¸n pÐstwshc.'Amuna. PhdalioÔqhsh ìplou, egklwbismìc stìqou, diˆkrish antikeimènwn, a-nagn¸rish pros¸pou, nèa eÐdh aisjht rwn, sìnar, rantˆr kai epexergasÐa shmˆtwneikìnac sumperilambanomènou sumpÐeshc dedomènwn, exagwg qarakthristik¸n kai katastoljorÔbou, anagn¸rish s matoc/eikìnac.Hlektronik . Prìbleyh akoloujÐac kwdikoÔ, diˆtaxh tsip oloklhrwmènou kukl¸matoc,èlegqoc diadikasÐac, anˆlush apotuqÐac tsip, ìrash mhqan c, sÔnjesh fwnc, mh grammik montelopoÐhsh.Diaskèdash. KinoÔmena sqèdia, eidikˆ efè, prìbleyh agorˆc.Oikonomikˆ. EktÐmhsh akÐnhthc periousÐac, sÔmbouloc daneÐou, parakoloÔjhshupoj khc, apotÐmhsh axÐac etairikoÔ grammatÐou, shmeÐwma anaforˆc phg c eÐdhshcqrhsimopoi¸ntac anˆlush, qartofulˆkio emporikoÔ progrˆmmatoc, anˆlush etairik¸noikonomik¸n, prìbleyh tim c sunallˆgmatoc.Asfˆleia. Politik efarmog c apotÐmhshc, beltistopoÐhsh proðìntoc.Kataskeuèc. 'Elegqoc diadikasÐac kataskeu c, sqedÐash kai anˆlush proðìntoc,diˆgnwsh diadikasÐac kai mhqan c, anagn¸rish swmatidÐwn se pragmatikì qrìno,sust mata optik c epije¸rhshc poiìthtac, dokim mpÔrac, anˆlush poiìthtac sugkìllhshc,prìbleyh poiìthtac qartioÔ, anˆlush poiìthtac tsip upologist¸n, anˆlushleitourgi¸n alèsmatoc, anˆlush sqediasmoÔ qhmik¸n proðìntwn, anˆlush sunt rhshcmhqan c, dunamik montelopoÐhsh susthmˆtwn qhmik c diadikasÐac, sqediasmìckai diaqeÐrish.Iatrik . Anˆlush karkinik¸n ist¸n st jouc, anˆlush EEG kai ECG, anˆlushteqnhtoÔ mèlouc, beltistopoÐhsh qrìnwn metamìsqeushc, meÐwsh exìdwn nosokomeÐou,beltÐwsh poiìthtac nosokomeÐwn.Petrèlaio kai Aèrio. ExereÔnhsh.Rompotik . 'Elegqoc troqiˆc, peronofìro anuywtikì rompìt, elegktèc paraplˆnhshc,sust mata ìrashc.OmilÐa. Anagn¸rish omilÐac, sumpÐesh omilÐac, taxinìmhsh fwnhèntwn, sÔnjeshomilÐac apì keÐmeno.ThlepikoinwnÐec. SumpÐesh eikìnac kai dedomènwn, autìmatec uphresÐec plhrofìrhshc,metˆfrash pragmatikoÔ qrìnou omiloÔmenhc gl¸ssac, sust mata epexergasÐacplhrwm c pelat¸n.Metaforikˆ Mèsa. Sust mata diˆgnwshc frènwn forthgoÔ, qronodiˆgrammaoqhmˆtwn.

40 Teqnhta Neurwnika DiktuaO arijmìc twn efarmog¸n twn neurwnik¸n diktÔwn, ta qr mata ta opoÐa èqounependujeÐ sto logismikì kai sthn ulik anˆptuxh twn neurwnik¸n diktÔwn, kai tobˆjoc kai to plˆtoc tou endiafèrontoc gia autèc tic suskeuèc, auxˆnontai taqÔtata.

Kefalaio 2EkpaÐdeush twn Teqnht¸nNeurwnik¸n DiktÔwnOi mèjodoi ekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn qwrÐzontai se dÔo kÔriec kathgorÐec,tic mejìdouc ekpaÐdeushc me epÐbleyh (supervised learning) kai tic mejìdoucekpaÐdeushc qwrÐc epÐbleyh (unsupervised learning). Se aut thn ergasÐa antimetopÐzontaimìno mèjodoi ekpaÐdeushc me epÐbleyh.Oi mèjodoi ekpaÐdeushc me epÐbleyh proôpojètoun thn parousÐa enìc {daskˆlou}.MporeÐ na jewrhjeÐ ìti o dˆskaloc èqei gn¸sh tou peribˆllontoc pou anaparistˆtaiapì èna sÔnolo paradeigmˆtwn eisìdou exìdou. To peribˆllon eÐnai, wstìso, ˆgnwstosto upì exètash neurwnikì dÐktuo. Ac upotejeÐ t¸ra ìti o dˆskaloc kai to neurwnikìdÐktuo eÐnai kai ta duo ektejeimèna se èna diˆnusma ekpaÐdeushc (parˆdeigma) exagìmenoapì to peribˆllon. Apì to protèrhma thc tmhmatikˆ auxanìmenhc gn¸shc, odˆskaloc eÐnai ikanìc na parèqei sto neurwnikì dÐktuo tic epijumhtèc apokrÐseic gia todiˆnusma ekpaÐdeushc. Pragmatikˆ, oi epijumhtèc apokrÐseic apoteloÔn thn bèltisthenèrgeia pou prèpei na ektelesteÐ apì to neurwnikì dÐktuo. Oi parˆmetroi tou diktÔourujmÐzontai kˆtw apì thn sunduasmènh epirro tou dianÔsmatoc ekpaÐdeushc kai tous matoc sfˆlmatoc. To s ma sfˆlmatoc orÐzetai wc h diaforˆ metaxÔ thc epijumht capìkrishc kai thc trèqousac apìkrishc tou diktÔou. Aut h rÔjmish ekteleÐtai epanalhptikˆb ma b ma me th bo jeia thc opoÐac telikˆ to neurwnikì dÐktuo exomoi¸nei todˆskalo. H exomoÐwsh sunepˆgetai ìti eÐnai h bèltisth me mia statistik ènnoia. Meautì ton trìpo diajèsimh gn¸sh tou peribˆllontoc ston dˆskalo, metafèretai stoneurwnikì dÐktuo mèsw thc ekpaÐdeushc ìso to dunatì plhrèstera. 'Otan h sunj khaut ikanopoieÐtai, mporeÐ tìte na apallageÐ o dˆskaloc kai na afejeÐ to neurwnikìdÐktuo na sunallˆssetai me to peribˆllon entel¸c mìno tou.H morf thc ekpaÐdeushc me epÐbleyh pou mìlic perigrˆfhke eÐnai h ekpaÐdeushsfˆlmatoc - diìrjwshc. EÐnai èna sÔsthma anˆdrashc kleistoÔ brìgqou ìpou to ˆ-41

42 Ekpaideush twn Teqnhtwn Neurwnikwn Diktuwngnwsto peribˆllon den eÐnai mèsa ston brìgqo. Wc èna mètro apìdoshc gia to sÔsthmamporoÔme na jewr soume to mèso tetragwnikì sfˆlma to ˆjroisma twn tetrag¸nwntwn sfalmˆtwn pˆnw sto deÐgma ekpaÐdeushc, orismèno wc mia sunˆrthsh twneleÔjerwn paramètrwn tou sust matoc. Aut h sunˆrthsh mporeÐ na apeikonisteÐ wcmia poludiˆstath epifˆneia sfˆlmatoc - apìdoshc aplˆ epifˆneia sfˆlmatoc, me ticeleÔjerec paramètrouc wc suntetagmènec. H pragmatik epifˆneia sfˆlmatoc eÐnaistroggulemènh katˆ mèso ìro pˆnw se ìla ta pijanˆ paradeÐgmata eisìdou exìdou.Kˆje dojeÐsa leitourgÐa tou sust matoc kˆtw apì thn epÐbleyh tou daskˆlou anaparÐstataiwc èna shmeÐo sthn epifˆneia sfˆlmatoc. Gia na belti¸sei to sÔsthma thnapìdosh tou sto qrìno kai epomènwc na mˆjei apì ton dˆskalo, to shmeÐo leitourgÐacprèpei na metakinhjeÐ proc ta kˆtw diadoqikˆ proc èna elˆqisto shmeÐo thc epifˆneiacsfˆlmatoc. To elˆqisto shmeÐo mporeÐ na eÐnai èna topikì elˆqisto èna olikì elˆqisto.'Ena sÔsthma ekpaÐdeushc me epÐbleyh eÐnai ikanì na kˆnei aut thn enèrgeiame thn qr simh plhroforÐa pou èqei gia thn klÐsh thn epifˆneiac sfˆlmatoc sqetikˆme thn trèqousa sumperiforˆ tou sust matoc. H klÐsh miac epifˆneiac sfˆlmatocse kˆje shmeÐo eÐnai èna diˆnusma pou deÐqnei sthn kateÔjunsh thc pio apìtomhc kajìdou(steepest descent). Sthn pragmatikìthta, sthn perÐptwsh thc ekpaÐdeushc meepÐbleyh, to sÔsthma mporeÐ na qrhsimopoi sei mia stigmiaÐa ektÐmhsh tou dianÔsmatocthc klÐshc, paÐrnontac wc dedomènouc touc deÐktec twn paradeigmˆtwn se ekeÐnoto qrìno. H qr sh miac tètoiac ektÐmhshc èqei wc apotèlesma thn kÐnhsh tou shmeÐouleitourgÐac pˆnw sthn epifˆneia sfˆlmatoc pou eÐnai tupikˆ thc morf c tou {tuqaÐouperipˆtou}. Wstìso, dojèntoc enìc algorÐjmou sqediasmènou gia thn elaqistopoÐhshthc sunˆrthshc kìstouc, enìc eparkoÔc sunìlou apì paradeÐgmata eisìdou exìdou,kai arketoÔ epitrepìmenou qrìnou gia na gÐnei h ekpaÐdeush, èna sÔsthma ekpaÐdeushcme epÐbleyh eÐnai sun jwc ikanì na ektelèsei leitourgÐec ìpwc taxinìmhsh protÔpwnkai prosèggish sunart sewn.2.1 Mèjodoi EkpaÐdeushcSÔmfwna me ton arqitektonikì grˆfo tou poluepÐpedou neurwnikoÔ diktÔou thc Eikìnac1.10, dÐnetai to majhmatikì montèlo thc metˆdoshc tou s matoc eisìdou sto dÐktuo.'Estw èna poluepÐpedo neurwnikì dÐktuo tou opoÐou to l epÐpedo perièqei N l neur¸nec,ìpou l = 1, . . . , M. Tìte to neurwnikì dÐktuo perigrˆfetai apì tic akìloujec sqèseic:u l j =N∑l−1i=1w l−1,lij y l−1i + b j , yj l = f(u l j)ìpou u l j eÐnai h eÐsodoc tou j neur¸na sto l epÐpedo (j = 1, . . . , N j), w l−1ij , eÐnai tabˆrh apì ton i neur¸na sto (l − 1) epÐpedo ston j neur¸na sto l epÐpedo, b j eÐnai hmerolhyÐa tou j neur¸na, yj l eÐnai h èxodoc tou j neur¸na pou an kei sto l epÐpedo kaif(u l j) eÐnai h sunˆrthsh energopoÐhshc tou j neur¸na.

Eisagwgh 43Gia thn aplopoÐhsh twn exis¸sewn, jewroÔme ìti w eÐnai to diˆnusma n diastˆsewcpou perièqei ìla ta sunaptikˆ bˆrh kai tic merolhyÐec. H diadikasÐa ekpaÐdeushc touneurwnikoÔ diktÔou mporeÐ na pragmatopoihjeÐ t¸ra elaqistopoi¸ntac thn sunˆrthshsfˆlmatoc E, dhlad anazht¸ntac èna bèltisto diˆnusma w ∗ ∈ R n ètsi ¸steìpouE(w) = 1 2w ∗ = min E(w) (2.1)w ∗ ∈Rn P∑N M∑p=1 j=1(y M j,p − t j,p ) 2 =P∑E p (2.2)ìpou (yj,p M − t j,p ) 2 eÐnai to tetrˆgwno thc diaforˆc metaxÔ thc tim c thc trèqousacapìkrishc ston j neur¸na tou epipèdou exìdou gia to prìtupo p kai thc tim c thcepijumht c apìkrishc tou diktÔou. To p eÐnai ènac deÐkthc gia ta zeÔgh eisìdouexìdou.'Estw ìti to epanalhptikì sq ma ananèwshc twn bar¸n èqei thn akìloujh morfw k+1 = w k + α k d k , k = 0, 1, 2, . . . (2.3)ìpou to w 0 ∈ R n eÐnai to tuqaÐo arqikì shmeÐo, α k eÐnai o rujmìc ekpaÐdeushc meα k > 0 kai d k eÐnai h kateÔjunsh anaz thshc pou ikanopoieÐ thn sunj kh kajìdou,dhlad gk T d k ≤ 0. H klÐsh g k = ∇E(w k ) mporeÐ eÔkola na upologisteÐ efarmìzontacton kanìna thc alusÐdac sta epÐpeda tou neurwnikoÔ diktÔou. Pollèc mèjodoi topik cbeltistopoÐhshc èqoun efarmosteÐ sthn ekpaÐdeush twn teqnht¸n neurwnik¸n diktÔwn[31].O pio gnwstìc algìrijmoc ekpaÐdeushc eÐnai h opisjodromik diˆdosh tou sfˆlmatoc(back–propagation) [74], h opoÐa elaqistopoieÐ thn sunˆrthsh sfˆlmatoc qrhsimopoi¸ntacthn kateÔjunsh thc pio apìtomhc kajìdou (steepest descent), dhladd k = −g k , kai èna stajerì (euretikˆ kajorismèno) rujmì ekpaÐdeushc α. Sthn prˆxh,h parˆmetroc tou rujmoÔ ekpaÐdeushc α orÐzetai se mia mikr tim mèsa sto diˆsthma(0, 1) ètsi ¸ste na exasfalisteÐ h sÔgklish tou algorÐjmou thc opisjodromik cdiˆdoshc tou sfˆlmatoc kai na apofeuqjoÔn oi talant¸seic ìtan h sunˆrthsh sfˆlmatoceÐnai apìtomh kai sten . Wstìso, akìmh kai me èna mikrì rujmì ekpaÐdeushc, oalgìrijmoc thc opisjodromik c diˆdoshc tou sfˆlmatoc mporeÐ na parousiˆsei talant¸seicìtan sunantˆ apìtomec koilˆdec. Epiplèon, h qr sh mikroÔ rujmoÔ ekpaÐdeushcepibradÔnei shmantikˆ thn diadikasÐa ekpaÐdeushc afoÔ mporeÐ na mhn eÐnai katˆllhlocgia ìlec tic perioqèc thc epifˆneiac sfˆlmatoc. Akìmh, o algìrijmoc sun jwcsugklÐnei se topikì elˆqisto parˆ se èna olikì, afoÔ h epifˆneia pou orÐzetai apìthn sunˆrthsh sfˆlmatoc ston q¸ro twn bar¸n mporeÐ na eÐnai an¸malh me pollˆtopikˆ elˆqista. 'Allwste, se pragmatikˆ probl mata, h diˆstash tou dianÔsmatoctwn bar¸n w mporeÐ na eÐnai tìso megˆlh ¸ste o qrìnoc pou apaiteÐ o algìrijmocthc opisjodromik c diˆdoshc tou sfˆlmatoc gia thn ekpaÐdeush tou diktÔou na eÐnaiuperbolikˆ megˆloc, gegonìc pou kajistˆ ton algìrijmo mh praktikì.p=1

44 Ekpaideush twn Teqnhtwn Neurwnikwn DiktuwnProkeimènou na perioristeÐ to eggenèc prìblhma tou stajeroÔ rujmoÔ ekpaÐdeushctou algorÐjmou thc opisjodromik c diˆdoshc tou sfˆlmatoc, ereun jhke h perÐptwshthc allag c tou rujmoÔ ekpaÐdeushc dunamikˆ katˆ thn diˆrkeia thc ekpaÐdeushc. Pollècstrathgikèc gia thn prosarmog tou rujmoÔ ekpaÐdeushc ereun jhkan apì toucereunhtèc thc perioq c. O Vogl [84] prìteine èna prosarmostikì algìrijmo ekpaÐdeushcopisjodromik c diˆdoshc tou sfˆlmatoc ìpou h diadikasÐa ekpaÐdeushc xekinˆ meèna mikrì rujmì ekpaÐdeushc o opoÐoc auxˆnetai ekjetikˆ eˆn diadoqikèc epanal yeicmei¸noun thn tim thc sunˆrthshc sfˆlmatoc, mei¸netai gr gora eˆn shmeiwjeÐ miashmantik aÔxhsh thc tim c thc sunˆrthshc sfˆlmatoc. Oi Chan kai Fallside [15] qrhsimopoÐhsanèna prosarmostikì sq ma gia ton rujmì ekpaÐdeushc ìpou h diadikasÐaekpaÐdeushc xekinˆ me èna mikro rujmì ekpaÐdeushc o opoÐoc auxˆnetai eˆn diadoqikècepanal yeic diathroÔn thn klÐsh thc kateÔjunshc arkoÔntwc stajer , mei¸netaigr gora eˆn h klÐsh thc kateÔjunshc poikÐllei se megˆlo bajmì se kˆje epanˆlhyh.'Allh mia prosèggish tou jèmatoc pou qrhsimopoi jhke stic ergasÐec [4], [3], [35], [36],[79], [62] kai [70] tan h sÔndesh enìc atomikoÔ rujmoÔ ekpaÐdeushc se kˆje bˆroc.O atomikìc rujmìc ekpaÐdeushc auxˆnetai sthn perÐptwsh ìpou diadoqikèc allagècsta bˆrh sumbaÐnoun sthn Ðdia kateÔjunsh, h mei¸netai diaforetikˆ. EpÐshc, kleistoÐtÔpoi gia ton upologismì thc paramètrou tou rujmoÔ ekpaÐdeushc protˆjhkan sticergasÐec [34], [48] kai [63]. KleistoÐ tÔpoi gia ton upologismì tou atomikoÔ rujmoÔekpaÐdeushc gia kˆje bˆroc protˆjhkan stic ergasÐec [23], [49] kai [85]. Wstìso,oi perissìterec apì tic parapˆnw strathgikèc qrhsimopoioÔn euretikèc paramètrouc¸ste na epibˆlloun meÐwsh sthn sunˆrthsh sfˆlmatoc se kˆje epanˆlhyh kai na e-xasfalÐsoun thn asfal sÔgklish tou algorÐjmou ekpaÐdeushc. Autèc oi euretikècparˆmetroi eÐnai exart¸menec apì ton qr sth kai sun jwc prèpei na rujmistoÔn giakˆje prìblhma.'Allec proseggÐseic gia thn apotelesmatik ekpaÐdeush twn neurwnik¸n diktÔwnèqoun protajeÐ apì thn jewrÐa thc arijmhtik c beltistopoÐhshc. Pio sugkekrimèna,èqoun qrhsimopoihjeÐ mèjodoi deÔterhc tˆxhc. Stic ergasÐec [8], [59], kai [6] èqounprotajeÐ algìrijmoi ekpaÐdeushc basismènec sthn mèjodo Newton kai se diˆforectropopoi seic thc. AutoÐ oi algìrijmoi apodÐdoun kalˆ ìtan to arqikì shmeÐo brÐsketaimèsa se kurt perioq kai sugklÐnoun gr gora an h perioq eÐnai tetragwnik ,perÐpou tetragwnik . Sthn prˆxh ìmwc upˆrqoun pollˆ meionekt mata sthn qr shmejìdwn Newton pou tic kˆnoun akatˆllhlec gia thn ekpaÐdeush neurwnik¸n diktÔwn.Gia na sugklÐnoun oi mèjodoi Newton apaiteÐtai mia kal arqik ektÐmhsh thc lÔshc,h opoÐa den eÐnai diajèsimh. Epiplèon, kˆje epanˆlhyh apaiteÐ ton upologismì touEssianoÔ pÐnaka kai tou antistrìfou tou, to opoÐo eÐnai upologistikˆ polÔ akribì(O(n 3 ) poluplokìthta anˆ epanˆlhyh) kai oi apait seic se q¸ro apoj keushc eÐnaieÐnai polÔ megˆlec (O(n 2 ) ekqwr seic mn mhc), eidikˆ se megˆlhc klÐmakac probl mata.Tèloc, gia mia mh kurt sunˆrthsh, ìpwc h sunˆrthsh sfˆlmatoc (2.2), oi mèjodoiNewton mporeÐ na sugklÐnoun se èna topikì mègisto, se èna sagmatikì shmeÐo seèna topikì elˆqisto [57].Oi mèjodoi yeudì Newton (Quasi–Newton) èqoun epÐshc qrhsimopoihjeÐ gia thn

Eisagwgh 45ekpaÐdeush twn teqnht¸n neurwnik¸n diktÔwn. Sunduˆzoun thn mèjodo Newton meèna algìrijmo olik c sÔgklishc ìpwc h grammik anaz thsh [57]. H pio gnwst kaieurèwc qrhsimopoioÔmenh mèjodoc aut c thc kathgorÐac eÐnai h Broyden–Fletcher–Goldfarb–Shanno (BFGS) [8], [6]. Aut h mèjodoc èqei pollˆ pleonekt mata. 'Eqeikalèc idiìthtec sÔgklishc tìso sthn jewrÐa ìso kai sthn prˆxh. UpologÐzei mìnomia prosèggish tou antistrìfou tou EssianoÔ pÐnaka, pou shmaÐnei ìti den apaiteÐtaio upologismìc twn parag¸gwn deutèrac tˆxewc. EpÐshc, o proseggistikìc antÐstrofocEssianìc pÐnakac thc BFGS eÐnai summetrikìc kai jetikˆ orismènoc, gegonìc poukajistˆ ton algìrijmo arijmhtikˆ stajerì. Wstìso, h BFGS kai oi upìloipoi yeudìNewton mèjodoi èqoun epÐshc merikˆ sobarˆ meionekt mata sthn ekpaÐdeush twn neurwnik¸ndiktÔwn. 'Ena apì autˆ eÐnai h anˆgkh gia ton upologismì thc prosèggishctou antistrìfou tou EssianoÔ pÐnaka se kˆje epanˆlhyh, pou kajistˆ tic mejìdoucautèc upologistikˆ akribèc (O(n 2 ) poluplokìthta anˆ epanˆlhyh). EpÐshc h apoj -keush tou pÐnaka se megˆlhc klÐmakac probl mata auxˆnei tic apait seic se mn mh toualgorÐjmou shmantikˆ (O(n 2 ) ekqwr seic mn mhc). Gi' autoÔc touc lìgouc oi mèjodoiyeudì Newton mporoÔn na efarmostoÔn se mikrˆ kai mesaÐac tˆxewc probl mata [10].Sthn ergasÐa [6], o Battiti prìteine mia nèa yeudì Newton mèjodo qwrÐc mn mh,pou thn onìmase One Step Secant (OSS). Aut h mèjodoc eÐnai ulopoihmènh qwrÐc thnqr sh pÐnaka. Gi' autì to lìgo, h apoj keush kai to upologistikì kìstoc, eidikˆ seprobl mata megˆlhc klÐmakac, èqei meiwjeÐ shmantikˆ (O(n) ekqwr seic mn mhc kaipoluplokìthta anˆ epanˆlhyh). Sthn ergasÐa [6] faÐnetai ìti h mèjodoc OSS mporeÐna eÐnai exairetikˆ antagwnistik me thn klasik BFGS mèjodo sthn ekpaÐdeush megˆlwnteqnht¸n neurwnik¸n diktÔwn. Wstìso, exaitÐac thc fÔsewc thc qwrÐc mn mhcprosèggishc, h posìthta thc plhroforÐac deutèrac tˆxewc sthn OSS èqei meiwjeÐ sesqèsh me thn klasik BFGS mèjodo. Basismènoc sthn idèa thc OSS mejìdou, o DiFiore sthn ergasÐa [21] parousÐase mia genÐkeush thc mejìdou OSS. Pio sugkekrimèna,prìteine duo algorÐjmouc pou touc onìmase Generalized Battiti 1 kai 2 (GB1kai GB2). Oi mèjodoi GB perièqoun perissìterh plhroforÐa deutèrac tˆxewc kai suqnˆparousiˆzontai an¸terec thc klasik c mejìdou OSS. 'Allh mia tropopoÐhsh thcBFGS mejìdou eÐnai oi periorismènhc mn mhc BFGS mèjodoi (L–BFGS) [2], [46], [57].Oi L–BFGS mèjodoi enhmer¸noun suneq¸c mia prosèggish tou EssianoÔ pÐnaka qrhsimopoi¸ntacthn pio prìsfath plhroforÐa deutèrac tˆxewc pou eÐnai diajèsimh me thmorf dianusmˆtwn. O rujmìc sÔgklishc twn mejìdwn L–BFGS mporeÐ na beltiwjeÐan qrhsimopoihjeÐ perissìterh plhroforÐa (megalÔteroc arijmìc dianusmˆtwn). Wstìso,perissìtera dianÔsmata auxˆnoun tic apait seic se mn mh twn mejìdwn (O(mn)ekqwr seic mn mhc, ìpou m eÐnai o arijmìc twn dianusmˆtwn). O Bortoletti sthn ergasÐa[11] eis gage mia nèa kathgorÐa mejìdwn yeudì Newton, tic LQN. Autèc oimèjodoi qrhsimopoioÔn katˆllhlec proseggÐseic olìklhrou tou EssianoÔ pÐnaka poukajorÐzontai apì mia ˆlgebra L pinˆkwn diagwnopoihmènwn apì gr gorouc monadiaÐoucmetasqhmatismoÔc [20]. Sthn ergasÐa aut parousiˆsthke ìti oi mèjodoi autècqreiˆzontai O(n) ekqwr seic mn mhc kai parousiˆzoun O(n log n) poluplokìthta anˆepanˆlhyh. Autì epitrèpei thn epÐlush problhmˆtwn ekpaÐdeushc megˆlhc klÐmakac,

46 Ekpaideush twn Teqnhtwn Neurwnikwn Diktuwnse antÐjesh me thn mèjodo BFGS.Mia enallaktik prìtash stouc algorÐjmouc ekpaÐdeushc pou protˆjhkan prohgoumènwceÐnai oi gnwstèc mèjodoi suzug¸n klÐsewn (Conjugate Gradient (CG)).EÐnai koinˆ apodektì sthn koinìthta thc arijmhtik c anˆlushc ìti oi mèjodoi suzug¸nklÐsewn eÐnai oi pio katˆllhlec mèjodoi gia thn epÐlush problhmˆtwn megˆlhcklÐmakac giatÐ apaitoÔn O(n) ekqwr seic mn mhc kai èqoun O(n) poluplokìthta anˆepanˆlhyh [10]. Pollèc mèjodoi suzug¸n klÐsewn èqoun protajeÐ prìsfata wc algìrijmoiekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn [37], [38], [52]. Oi pio dhmofileÐc kaieurèwc qrhsimopoioÔmenoi algìrijmoi ekpaÐdeushc apì aut thn kathgorÐa mejìdwneÐnai oi Hestenes-Stiefel (HS) [33], Fletcher-Reeves (FR) [26] kai Polak-Ribière (PR)[65]. 'Eqei apodeiqjeÐ ìti autoÐ oi algìrijmoi parèqoun stajer ekpaÐdeush, anjektikìthtastic talant¸seic kai beltiwmènouc rujmoÔc sÔgklishc. Autì proèrqetai apìto gegonìc ìti h kateÔjunsh anaz thshc suzug¸n klÐsewn lambˆnei upìyin thn prohgoÔmenhkateÔjunsh kai ìti o rujmìc ekpaÐdeushc kajorÐzetai apì mia apotelesmatikmèjodo grammik c anaz thshc [57]. H qrhsimopoÐhsh twn teqnik¸n grammik c anaz -thshc proôpojètei merikoÔc epiprìsjetouc upologismoÔc thc sunˆrthshc sfˆlmatockai tic klÐshc thc se kˆje epanˆlhyh, allˆ o upologistikìc autìc kìpoc eÐnai sqetikˆmikrìc akìmh kai se probl mata megˆlhc klÐmakac. Sthn ergasÐa [52] parousiˆzetaimia ˆllh ekdoq thc mejìdou suzug¸n klÐsewn ìpou h grammik anaz thsh antikajÐstataiapì mia klimakopoÐhsh thc paramètrou tou rujmoÔ ekpaÐdeushc pou exartˆtaiapì thn epituqhmènh meÐwsh tou sfˆlmatoc kai thn pistìthta miac monodiˆstathc tetragwnikc prosèggishc. Aut h mèjodoc kaleÐtai Scaled Conjugate Gradient (SCG)kai enswmat¸nei idèec apì tic mejìdouc empistosÔnhc perioq c trust region kai kˆpoiecdiadikasÐec asfaleÐac pou apousiˆzoun apì tic klasikèc mejìdouc suzug¸n klÐsewn.Wstìso, akìmh kai qwrÐc thn qr sh teqnik¸n grammik c anaz thshc, o algìrijmocekpaÐdeushc SCG den eÐnai an¸teroc twn klasik¸n algorÐjmwn ekpaÐdeushc suzug¸nklÐsewn ìtan qrhsimopoieÐtai ènac katˆllhloc arqikìc rujmìc ekpaÐdeushc kai miaapotelesmatik teqnik grammik c anaz thshc.'Opwc mporeÐ na gÐnei eÔkola antilhptì, apì thn epoq thc dhmosÐeushc thc ergasÐactwn Rumelhart, Hinton kai Williams [73] to 1986, ìpou parousiˆsthke o algìrijmocekpaÐdeushc thc opisjodromik c diˆdoshc tou sfˆlmatoc, oi ereunhtèc thcperioq c èqoun anaptÔxei pˆra pollèc mejìdouc gia thn ekpaÐdeush twn neurwnik¸ndiktÔwn. Mèqri s mera den èqei upˆrxei kˆpoia mèjodoc h opoÐa na mporeÐ na epilÔseiìlou tou eÐdouc ta probl mata ekpaÐdeushc me bèltisto trìpo. Kˆje mèjodoc mporeÐna epilÔsei kˆpoia upokathgorÐa problhmˆtwn kalÔtera apì tic upìloipec. SthnparoÔsa diatrib ja parousiastoÔn merikoÐ nèoi algìrijmoi ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn oi opoÐoi faÐnetai na mporoÔn na epilÔsoun orismèna probl mata meton kalÔtero dunatì trìpo.

Eisagwgh 472.2 Epifˆneia Sfˆlmatoc kai o Q¸roc twn Bar¸nH ekpaÐdeush twn teqnht¸n neurwnik¸n diktÔwn, dhlad h elaqistopoÐhsh thc sunˆrthshcsfˆlmatoc (2.2), eÐnai èna polÔ dÔskolo prìblhma. Ac jewr soume thn aplperÐptwsh, ìpou to neurwnikì dÐktuo apoteleÐtai apì èna mìno neur¸na me sigmoeidsunˆrthsh energopoÐhshc. To dÐktuo dèqetai mia mìno eÐsodo, pou shmaÐnei ìti todiˆnusma twn bar¸n w ∈ R 2 . O skopìc tou diktÔou eÐnai na mˆjei 8 zeÔgh eisìdouexìdou. Sthn Eikìna 2.1 apeikonÐzetai h epifˆneia thc sunˆrthshc sfˆlmatoc pˆnwston q¸ro twn bar¸n.4.543.532.521.5−4−2024 −505Sq ma 2.1: H epifˆneia thc sunˆrthshc sfˆlmatoc pˆnw ston q¸ro twn bar¸n se ènaneurwnikì dÐktuo me èna mìno neur¸na.To epijumhtì elˆqisto brÐsketai sto kèntro tou sq matoc en¸ upˆrqoun dÔo koilˆdec(dexiˆ kai aristerˆ tou sq matoc) pou odhgoÔn se topikˆ elˆqista. Katˆ thnekpaÐdeush tou neurwnikoÔ diktÔou, upˆrqei pijanìthta o algìrijmoc ekpaÐdeushc nasugklÐnei sta anepijÔmhta topikˆ elˆqista. Sto parˆdeigma autì o q¸roc twn bar¸nkai h epifˆneia thc sunˆrthshc sfˆlmatoc eÐnai sqetikˆ aplèc. Se probl mata polÔ

48 Ekpaideush twn Teqnhtwn Neurwnikwn Diktuwnmegˆlhc klÐmakac o q¸roc twn bar¸n gÐnetai pio polÔplokoc. H sunˆrthsh sfˆlmatocplèon parousiˆzei stenèc perioqèc me pollˆ topikˆ elˆqista ìpou eÐnai pio eÔkolo napagideutoÔn oi mèjodoi ekpaÐdeushc.EpÐshc shmantikì rìlo paÐzei kai o arijmìc twn protÔpwn ekpaÐdeushc. 'Oso perissìteraeÐnai ta prìtupa ekpaÐdeushc tìso pio polÔplokoc gÐnetai o q¸roc twn bar¸nkai h sunˆrthsh sfˆlmatoc. DhmiourgoÔntai megˆlec epÐpedec perioqèc me sqedìnmhdenik klÐsh kai apìtomec qarˆdrec me pollˆ topikˆ elˆqista pou kˆnoun thn ekpaÐdeushtwn teqnht¸n neurwnik¸n diktÔwn pragmatik prìklhsh. Sthn Eikìna 2.3faÐnetai h epÐdrash pou èqei o arijmìc twn protÔpwn sthn epifˆneia sfˆlmatoc sthnperÐptwsh tou aploÔ neurwnikoÔ diktÔou, apoteloÔmeno apì èna neur¸na.11.50.510.505050−5−5050−5−505αβ1.53120.5105050−550−50−550−5γδSq ma 2.2: H epifˆneia thc sunˆrthshc sfˆlmatoc pˆnw ston q¸ro twn bar¸n se ènaneurwnikì dÐktuo me èna mìno neur¸na upì thn epÐdrash: a. enìc protÔpou, b. dÔo protÔpwn,g. tri¸n protÔpwn kai d. tessˆrwn protÔpwn.

Eisagwgh 492.2.1 ArqikopoÐhsh twn Bar¸nOi arqikèc timèc twn sunaptik¸n bar¸n paÐzoun polÔ shmantikì rìlo sthn epituqÐathc ekpaÐdeushc tou teqnhtoÔ neurwnikoÔ diktÔou. Ac jewr soume gia mia akìmh forˆthn apl perÐptwsh, ìpou to neurwnikì dÐktuo apoteleÐtai apì èna mìno neur¸na kai hepifˆneia thc sunˆrthshc sfˆlmatoc dÐnetai apì thn Eikìna 2.1. Xekin¸ntac apì dÔodiaforetikˆ shmeÐa pˆnw ston q¸ro twn bar¸n, to neurwnikì dÐktuo ekpaideÔetai dÔoforèc qrhsimopoi¸ntac ton algìrijmo ekpaÐdeushc thc opisjodromik c diˆdoshc tousfˆlmatoc stajeropoi¸ntac kai tic dÔo forèc ìlec tic upìloipec paramètrouc. 'OpwcfaÐnetai apì thn Eikìna 2.3, o algìrijmoc ekpaÐdeushc sumperifèretai diaforetikˆ,anˆloga me to arqikì shmeÐo.44332211b0b0−1−1−2−2−3−3−4−3 −2 −1 0 1 2 3w−4−3 −2 −1 0 1 2 3wSq ma 2.3: IsoôyeÐc thc epifˆneiac thc sunˆrthshc sfˆlmatoc kai troqiˆ tou algìrijmouekpaÐdeushc. Aristerˆ o algìrijmoc sugklÐnei se èna topikì elˆqisto, en¸ dexiˆ o algìrijmocsugklÐnei sto olikì elˆqisto.Sthn pr¸th perÐptwsh (aristerˆ), o algìrijmoc ekpaÐdeushc apoklÐnei apì to olikìelˆqisto kai pagideÔetai se mia koilˆda pou odhgeÐ se èna topikì elˆqisto. SthndeÔterh perÐptwsh (dexiˆ) o algìrijmoc ekpaÐdeushc sugklÐnei gr gora sto olikìelˆqisto.H epilog arqik¸n tim¸n gia ta sunaptikˆ bˆrh eÐnai èna polÔ dÔskolo kai anoiqtìjèma. 'Otan ta sunaptikˆ bˆrh arqikopoioÔntai me megˆlec timèc, eÐnai polÔ pijanìoi neur¸nec tou diktÔou na odhghjoÔn se koresmì. Se aut thn perÐptwsh, h klÐshthc sunˆrthshc paÐrnei polÔ mikrèc timèc gegonìc pou epibradÔnei ton algìrijmo

50 Ekpaideush twn Teqnhtwn Neurwnikwn DiktuwnekpaÐdeushc shmantikˆ. Wstìso, ìtan ta sunaptikˆ bˆrh arqikopoioÔntai me mikrèctimèc o algìrijmoc ekpaÐdeushc mporeÐ na leitourgeÐ se mia polÔ epÐpedh perioq kontˆsthn arq twn suntetagmènwn thc epifˆneiac sfˆlmatoc, eidikˆ sthn perÐptwshpou qrhsimopoioÔntai antisummetrikèc sunart seic energopoÐhshc, ìpwc h uperbolikefaptomènh. Dustuq¸c ìmwc, h arq twn suntetagmènwn eÐnai èna sagmatikì shmeÐo(saddle point), to opoÐo anafèretai se èna stˆsimo shmeÐo ìpou h kampulìthta thcepifˆneiac sfˆlmatoc egkˆrsia tou sˆgmatoc eÐnai arnhtik kai katˆ m koc tou sˆgmatoceÐnai jetik . Gi' autoÔc touc lìgouc prèpei na apofeÔgetai h qr sh megˆlwn kaimikr¸n tim¸n gia thn arqikopoÐhsh twn sunaptik¸n bar¸n. H swst epilog gia thnarqikopoÐhsh brÐsketai kˆpou metaxÔ aut¸n twn duo akraÐwn peript¸sewn.Ac upojèsoume ìti èna poluepÐpedo neurwnikì dÐktuo qrhsimopoieÐ wc sunˆrthshenergopoÐhshc gia touc neur¸nec tou thn uperbolik efaptomènh. 'Estw ìti h merolhyÐapou efarmìzetai se kˆje neur¸na tou diktÔou eÐnai Ðsh me mhdèn. MporoÔme tìtena perigrˆyoume to ephreasmèno topikì pedÐo tou neur¸na j wcv j =m∑w ji y ii=1'Estw ìti oi eÐsodoi pou efarmìzontai se kˆje neur¸na sto dÐktuo èqoun mhdenikìmèso kai monadiaÐa diakÔmansh, ìpwc faÐnetai apì thnkai thnµ y = E[y i ] = 0 gia ìla ta iσ 2 y = E[(y i − µ i ) 2 ] = E[y 2 i ] = 1 gia ìla ta i'Estw ìti upojètoume epiplèon ìti oi eÐsodoi eÐnai mh susqetizìmenec, ìpwc faÐnetaiapì thn{1 gia k = iE[y i y k ] =0 gia k ≠ ikai ìti ta sunaptikˆ bˆrh epilègontai apì èna omoiìmorfa katanemhmèno sÔnolo apìarijmoÔc me mhdenikì mèsokai diakÔmanshµ w = E[w ji ] = 0 gia ìla ta zeÔgh (j, i)σ 2 w = E[(w ji − µ w ) 2 ] = E[w 2 ji] gia ìla ta zeÔgh (j, i)Epomènwc mporoÔme na ekfrˆsoume ton mèso kai thn diakÔmansh tou ephreasmènoutopikoÔ pedÐou v j wc[ m]∑µ v = E[v j ] = E w ji y i =i=1m∑E[w ji ]E[y i ] = 0i=1

Eisagwgh 51kaiσv 2 = E[(v j − m v ) 2 ] = E[vj 2 [ m]∑ m∑= E w ji w jk y i y k==m∑i=1m∑i=1= mσ 2 wi=1k=1m∑E[w ji w jk ]E[y i y k ] (2.4)k=1E[w 2 ji]ìpou m eÐnai o arijmìc twn sunaptik¸n sundèsewn enìc neur¸na.SÔmfwna me ton parapˆnw apotèlesma, mporoÔme na perigrˆyoume mia kal strathgikgia thn arqikopoÐhsh twn sunaptik¸n bar¸n ètsi ¸ste h tupik apìklish touephreasmènou topikoÔ pedÐou enìc neur¸na na brÐsketai sthn perioq metˆbashc metaxÔtou grammikoÔ kai twn koresmènwn tmhmˆtwn thc sigmoeidoÔc sunˆrthshc energopoÐhshctou. Gia parˆdeigma, gia thn perÐptwsh miac sunˆrthshc uperbolik c efaptomènhcthc morf cf(v) = α tanh(bv)ìpou α = 1.7159 kai b = 2/3, autìc o stìqoc ikanopoieÐtai jètontac σ v = 1 sthnexÐswsh (2.4), sthn opoÐa perÐptwsh paÐrnoume (LeCun, 1993 [45])σ w = m −1/2'Etsi eÐnai epijumhtì gia thn omoiìmorfh katanom , apì thn opoÐa epilèqjhkan tasunaptikˆ bˆrh, na èqei mhdenikì mèso kai diakÔmansh Ðsh me ton antÐstrofo touarijmoÔ twn sunaptik¸n sundèsewn enìc neur¸na.Mia ˆllh idiaÐtera apotelesmatik teqnik arqikopoÐhshc twn bar¸n protˆjhkeapì touc Nguyen kai Widrow [55]. Aut h teqnik apotrèpei ton prìwro koresmìstouc krufoÔc neur¸nec upologÐzontac to diˆsthma sto opoÐo lambˆnontai ta bˆrhpou sundèoun tic eisìdouc me touc krufoÔc neur¸nec sÔmfwna me ton arijmì twneisìdwn N kai twn arijmì twn kruf¸n neur¸nwn M. Gia thn ulopoÐhsh thc teqnik c,pr¸ta upologÐzetai h parˆmetroc ρ:ρ = 0.7(M 1/N )Sth sunèqeia epilègontai ta bˆrh w r = (w11, r . . . , wnm, r . . . , wN r M ) tuqaÐa mèsa apì todiˆsthma [−1, 1]. Tèloc ta bˆrh metaxÔ thc eisìdou kai tou krufoÔ epipèdou arqikopoioÔntaisÔmfwna me th sqèshw 0 nm = ρwr nm∥w r ∥

52 Ekpaideush twn Teqnhtwn Neurwnikwn DiktuwnAut h diadikasÐa arqikopoÐhshc twn sunaptik¸n bar¸n èqei wc apotèlesma thnkatanom twn bar¸n twn kruf¸n neur¸nwn me tètoio trìpo ¸ste na eÐnai pio pijanìkˆje prìtupo na prokaleÐ apodotik ekpaÐdeush twn kruf¸n neur¸nwn, gia naepitaqÔnetai h sÔgklish kai na apofeÔgete o prìwroc koresmìc. Se aut thn diatrib, qrhsimopoieÐtai h teqnik arqikopoÐhshc twn sunaptik¸n bar¸n twn Nguyen kaiWidrow ektìc ki an shmeiwjeÐ diaforetikˆ.

Mèroc IINeoi Algorijmoi EkpaideushcTND53

Kefalaio 3Algìrijmoc EkpaÐdeushcProsèggishc DÔo ShmeÐwnSe autì to kefˆlaio parousiˆzetai ènac nèoc algìrijmoc ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn basismènoc ston algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatockai thn autìmath prosarmog tou rujmoÔ ekpaÐdeushc qrhsimopoi¸ntac plhroforÐadÔo shmeÐwn. H kateÔjunsh anaz thshc eÐnai pˆnta h kateÔjunsh thc pio apìtomhckajìdou, allˆ gia ton prosdiorismì tou rujmoÔ ekpaÐdeushc qrhsimopoioÔntaiproseggÐseic dÔo shmeÐwn thc exÐswshc qord c twn mejìdwn yeudì Newton [5]. Epiplèon,parˆgoume èna nèo rujmì ekpaÐdeushc proseggÐzontac mia nèa exÐswsh qord c,pou protˆjhke apì ton Zhang [90], h opoÐa qrhsimopoieÐ plhroforÐa parag¸gwn kaisunarthsiak¸n tim¸n. Sthn sunèqeia, ènac katˆllhloc mhqanismìc epilog c tou rujmoÔekpaÐdeushc enswmat¸netai ston algìrijmo ekpaÐdeushc ¸ste na epilègetai kˆjeforˆ o katˆllhloc rujmìc ekpaÐdeushc. Tèloc, gÐnetai melèth thc sÔgklishc toualgorÐjmou ekpaÐdeushc kai parousiˆzontai ta peiramatikˆ apotelèsmata gia diˆforaprobl mata ekpaÐdeushc.3.1 H mèjodoc twn Barzilai kai BorweinAc jewr soume to prìblhma thc beltistopoÐhshc qwrÐc periorismoÔcmin f(x), x ∈ R n (3.1)ìpou h f eÐnai suneq c me diajèsimec tic parag¸gouc pr¸thc tˆxewc. O algìrijmoc thcpio apìtomhc kajìdou gia thn epÐlush tou probl matoc (3.1) eÐnai èna epanalhptikìsq ma thc morf cx k+1 = x k − α k g k (3.2)55

56 Neoi Algorijmoi Ekpaideushc TNDìpou g k = ∇f(x k ) kai α k eÐnai èna b ma.EÐnai gnwstì ìti h kateÔjunsh thc pio apìtomhc kajìdou èqei thn akìloujh bèltisthidiìthta−g k = min lim [f(x k) − f(x k + αd/∥d∥ 2 2)]/α (3.3)a→0+d∈R nSthn klasik mèjodo thc pio apìtomhc kajìdou [14], to b ma prosdiorÐzetai qrhsimopoi¸ntacakrib c grammik anaz thsh, dhladα k = arg minαf(x k − αg k ) (3.4)Wstìso, parìlec tic bèltistec idiìthtec (3.3) kai (3.4), h mèjodoc thc pio apìtomhckajìdou sugklÐnei polÔ argˆ kai ephreˆzetai shmantikˆ apì thn kak katˆstash touprobl matoc [1].Oi Barzilai kai Borwein [5], prìteinan mia tropopoÐhsh tou algorÐjmou thc pio apìtomhckajìdou ìpou to b ma upologÐzetai qrhsimopoi¸ntac proseggÐseic dÔo shmeÐwnthc exÐswshc qord c twn mejìdwn yeudì Newton. H mèjodoc aut onomˆzetai BB.H exÐswsh (3.2) mporeÐ na grafeÐ wcx k+1 = x k − B k g k (3.5)ìpou B k = α k I. Gia na èqei o pÐnakac B k thn idiìthta yeudì Newton, to α k upologÐzetaiètsi ¸ste na elaqistopoieÐtai h posìthtaWc sunèpeia thc apaÐthshc aut c prokÔptei∥s k−1 − B k y k−1 ∥ (3.6)α k = sT k−1 y k−1y T k−1 y k−1(3.7)ìpou s k−1 = x k − x k−1 kai y k−1 = g k − g k−1 .Summetrikˆ, mporoÔme na lÔsoume to prìblhma min ∥B −1ks k−1 − y k−1 ∥ wc proc toα k , to opoÐo dÐnei wc lÔsh to b maα k = sT k−1 s k−1s T k−1 y k−1(3.8)Epomènwc, h mèjodoc twn Barzilai kai Borwein dÐnetai apì ton parakˆtw algìrijmo.

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 57Algìrijmoc 1 H mèjodoc twn Barzilai kai Borwein.BB(x 0 , ϵ, µ, x ⋆ , f(x ⋆ ))1: Gia k := 0 èwc µ Kˆne2: Upolìgise to f k = f(x k ) kai to g k = ∇f(x k ).3: An ∥g k ∥ ≤ ϵ Tìte4: Epèstreye x ⋆ = x k , f(x ⋆ ) = f(x k ).5: Alli¸c6: Jèse d k = −g k .7: An k := 0 Tìte8: Brec to α 0 qrhsimopoi¸ntac grammik anaz thsh.9: Alli¸c10: Upolìgise to α k qrhsimopoi¸ntac thn (3.7) thn (3.8).11: Tèloc12: Upolìgise x k+1 = x k + α k d k .13: Tèloc14: Tèloc15: Epèstreye To elˆqisto den brèjhke.Apì ton parapˆnw algìrijmo faÐnetai ìti den qreiˆzontai kajìlou upologismoÐpinˆkwn. EpÐshc den qreiˆzontai grammikèc anazht seic, me exaÐresh to pr¸to b ma(k = 0). H mèjodoc twn Barzilai kai Borwein eÐnai mia mèjodoc klÐshc pou apaiteÐmikrìtero upologistikì kìpo kai epitaqÔnei shmantikˆ thn sÔgklish thc mejìdou thcpio apìtomhc kajìdou. Oi Barzilai kai Borwein èdeixan ìti o parapˆnw algìrijmoceÐnai R upergrammik c sÔgklishc gia thn perÐptwsh twn tetragwnik¸n sunart sewn.Mia pio leptomer c exètash thc mejìdou twn Barzilai kai Borwein faner¸nei ìti tab mata α k eÐnai proseggÐseic tou phlÐkou Rayleigh miac tim c 1/λ i . 'Eqontac sto mualìthn exÐswsh qord c, B k s k−1 = y k−1 , mporeÐ na epalhjeuteÐ ìti to phlÐko Rayleigh[∫ 1]/R q = s T k−1 ∇ 2 f(x k−1 + τs k−1 ) dτ s k−1 sTk−1 s k−1antistoiqeÐ ston mèso ìro tou EssianoÔ pÐnaka0∫ 10∇ 2 f(x k−1 + ts k )dtsto eujÔgrammo tm ma metaxÔ tou x k−1 kai x k . Epomènwc to R q eÐnai mia kal prosèggishthc idiotim c λ i gia thn opoÐa to s k−1 eÐnai to antÐstoiqo idiodiˆnusma. EpomènwceÐnai profanèc ìti to b ma α k sqetÐzetai me tic idiotimèc tou EssianoÔ pÐnaka sthn timtou elaqÐstou kai ìqi me thn sunarthsiak tim .Sthn genik perÐptwsh twn mh tetragwnik¸n sunart sewn, o Raydan [69] prìteinethn qrhsimopoÐhsh miac mh monìtonhc strathgik c grammik c anaz thshc pou tairiˆzeime thn mèjodo twn Barzilai kai Borwein, apodeiknÔontac thn olik sÔgklish thc

58 Neoi Algorijmoi Ekpaideushc TNDmejìdou. H mh monìtonh sunj kh tou Armijo, pou qrhsimopoieÐtai sthn grammikanaz thsh, dÐnetai apì thn sqèshf (x k − α k g k ) ≤max f(x k−j ) − σ α k gk T g k (3.9)0≤j≤min{k,M}ìpou M eÐnai ènac akèraioc arijmìc pou elègqei to pl joc twn pio prìsfatwn tim¸nthc antikeimenik c sunˆrthshc kai to σ ∈ (0, 1) eÐnai ènac mikrìc jetikìc arijmìc. Auth sunj kh, pou protˆjhke apì touc Grippo, Lampariello kai Lucidi [29], epitrèpei thnapodoq opoioud pote shmeÐou pou belti¸nei ikanopoihtikˆ thn megalÔterh apì tic pioprìsfatec sunarthsiakèc timèc.Epiplèon to b ma α k pou upologÐzetai apì tou tÔpouc (3.7) kai (3.8) mporeÐ naeÐnai aparˆdekta megˆlo mikrì. Se aut thn perÐptwsh, prèpei na upojèsoume ìti tob ma α k ikanopoieÐ th sunj kh0 < α (l) ≤ α k ≤ α (u) gia ìla ta kìpou α (l) kai α (u) eÐnai prokajorismènoi arijmoÐ.An qrhsimopoi soume thn epanˆlhyhx k+1 = x k − 1 α kg k = x k − λ k g k (3.10)meα k = sT k−1 y k−1s T k−1 s , λ k = 1 (3.11)k−1 α ktìte, èqontac upìyin ìti s k = − 1α kg k = −λ k g k , èqoumeα k+1 = sT k y ks T k s k= −λ kg T k y kλ 2 k gT k g k= − gT k y kλ k g T k g kH mèjodoc twn Barzilai kai Borwein me thn qr sh thc mh monìtonhc strathgik canaz thshc twn Grippo, Lampariello kai Lucidi dÐnetai apì ton parakˆtw algìrijmo.Shmei¸netai ìti M ≥ 0, σ ∈ (0, 1), δ > 0 kai 0 < σ 1 < σ 2 < 1.

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 59Algìrijmoc 2 H mèjodoc twn Barzilai kai Borwein me thn qr sh thc mh monìtonhcstrathgik c anaz thshc.NMBB(x 0 , ϵ, µ, M, σ, δ, σ 1 , σ 2 , α (l) , α (u) , x ⋆ , f(x ⋆ ))1: Gia k := 0 èwc µ Kˆne2: Upolìgise to f k = f(x k ) kai to g k = ∇f(x k ).3: An ∥g k ∥ ≤ ϵ Tìte4: Epèstreye x ⋆ = x k , f(x ⋆ ) = f(x k ).5: Alli¸c6: An α k ≤ α (l) α k ≥ α (u) Tìte7: Jèse α k = δ.8: Tèloc9: Jèse λ = 1/α k .10: 'Oso f (x k − λg k ) > max 0≤j≤min{k,M} f(x k−j ) − σ λgk T g k Kˆne11: Epèlexe σ ∈ [σ 1 , σ 2 ] kai jèse λ = σλ.12: Tèloc13: Jèse λ k = λ kai x k+1 = x k − λ k g k .14:15:Jèse α k+1 = −(gk T y k)/(λ k gk T g k)Tèloc16: Tèloc17: Epèstreye To elˆqisto den brèjhke.3.2 Nèa 'ExÐswsh Qord c kai Paragwg NèwnBhmˆtwnOi mèjodoi yeudì Newton proseggÐzoun ton Essianì pÐnaka ∇ 2 f(x k ) me èna n × nsummetrikì pÐnaka B k o opoÐoc upologÐzetai enhmer¸nontac ton pÐnaka B k−1 kai axiopoi¸ntacthn diajèsimh plhroforÐa. H enhmèrwsh basÐzetai sthn exÐswsh qord c pouikanopoieÐtai apì ton pÐnaka B k . H klasik exÐswsh qord c prokÔptei akolouj¸ntacthn parakˆtw diadikasÐa.'Estw g(x) = ∇f(x) kai s k−1 = x k − x k−1 . Tìte èqoume[∫ 1g k − g k−1 =0]∇ 2 f(x k−1 + τs k−a )dτ s k−1 (3.12)AfoÔ o B k prèpei na proseggÐzei ton Essianì pÐnaka, h exÐswsh qord c gÐnetaiìpou y k−1 = g k − g k−1 .B k s k−1 = y k−1 (3.13)

60 Neoi Algorijmoi Ekpaideushc TNDProfan¸c, mìno dÔo diadoqikèc klÐseic qrhsimopoioÔntai sthn exÐswsh qord c. OZhang [90] par gage mia nèa exÐswsh qord c h opoÐa qrhsimopoieÐ plhroforÐa klÐsewnkai sunarthsiak¸n tim¸n. Pio sugkekrimèna, prospˆjhse na proseggÐsei to∇ 2 f(x k )s k−1 qrhsimopoi¸ntac tic timèc g k−1 , g k , f k−1 kai f k .OrÐzontacx(τ) = x k−1 + τs k−1 /∥s k−1 ∥ (3.14)èqoume(d/dτ)g(x(τ))| τ=∥sk−1 ∥ = ∇ 2 f(x(τ))s k−1 /∥s k−1 ∥| τ=∥sk−1 ∥= ∇ 2 f(x k )s k−1 /∥s k−1 ∥∇ 2 f(x k )s k−1 = ∥s k−1 ∥(d/dτ)g(x(τ))| τ=∥sk−1 ∥ (3.15)H exÐswsh (3.15) upodhl¸nei ìti mporoÔme na pˆroume mia kal prosèggish tou∇ 2 f(x k )s k−1 an petÔqoume mia kal prosèggish tou g(x(τ)). Upojètoume ìti h g(x(τ))proseggÐzetai apì èna polu¸numo q(τ). AfoÔ h diajèsimh plhroforÐa perièqei 2n + 2stoiqeÐa, o bajmìc tou poluwnÔmou q(τ) prèpei na eÐnai megalÔteroc thc monˆdac. E-pomènwc, upojètoume ìti to q(τ) eÐnai mia tetragwnik sunˆrthsh pou dÐnetai apì thnparakˆtw sqèshq(τ) = ατ 2 + bτ + c (3.16)ìpou α, b, c ∈ R n . Lambˆnontac upìyin thn diajèsimh plhroforÐa, h proseggistiksunˆrthsh q(τ) apaiteÐtai na ikanopoieÐ tic akìloujec sunj kecq(0) = g(x(0)) = g(x k−1 ) = g k−1 (3.17)q(∥s k−1 ∥) = g(x(∥s k−1 ∥)) = g(x k ) = g k (3.18)∫ ∥sk−1 ∥ìpou h teleutaÐa exÐswsh prokÔptei apì thn tautìthta∫ ∥sk−1 ∥00g(x(τ)) T x ′ (τ)dτ =q(τ) T x ′ (τ)dτ = f k − f k−1 (3.19)∫ ∥sk−1 ∥0g(x(τ)) T dx(τ)= f(x(τ))| ∥s k−1∥0= f(x k ) − f(x k−1 )Oi sunj kec (3.17)-(3.19) mporoÔn na diatupwjoÔn wcc = g k−1 (3.20)∥s k−1 ∥ 2 α + ∥s k−1 ∥b = g k − g k−1 = y k−1 (3.21)∥s k−1 ∥ 2 α T s k−1 = γ (3.22)

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 61ìpouγ = 3(g k − g k−1 ) T s k−1 − 6(f k − f k−1 ) (3.23)H exÐswsh (3.22) apofèrei thn∥s k−1 ∥ 2 α = γs k−1 /∥s k−1 ∥ 2 + υ (3.24)ìpou to υ eÐnai èna opoiod pote diˆnusma ston R n kˆjeto sto s k−1 .sqèseic (3.15), (3.16), (3.21) kai (3.24) èqoume'Etsi apì tic∇ 2 f(x k )s k−1 = ∥s k−1 ∥(d/dτ)g(x(τ))| τ=∥sk−1 ∥≈ ∥s k−1 ∥(d/dτ)q(τ)| τ=∥sk−1 ∥= 2∥s k−1 ∥ 2 α + ∥s k−1 ∥b= y k−1 + γs k−1 /∥s k−1 ∥ 2 + υ (3.25)ìpou to γ kai to υ orÐzontai apì tic (3.23) kai (3.24), antÐstoiqa. Epomènwc h nèaexÐswsh qord c èqei th morfB k s k−1 = y k−1 + γs k−1 /∥s k−1 ∥ 2 + υ (3.26)Gia na kajorÐsoume to diˆnusma υ, parathroÔme ìti h klasik exÐswsh qord c (3.13)leitourgeÐ apotelesmatikˆ sthn prˆxh. Epomènwc, eÐnai logikì na kˆnoume to dexÐmèroc thc exÐswshc (3.26) na plhsiˆsei to dexÐ mèroc thc exÐswshc (3.13). Elaqistopoi¸ntacthn diaforˆ metaxÔ tou y k−1 + γs k−1 /∥s k−1 ∥ 2 + υ kai tou y k−1 èqoumeEpomènwc h nèa exÐswsh qord c dÐnetai apì tic sqèseicυ = 0 (3.27)B k s k−1 = ỹ k−1 (3.28)ỹ k−1 = y k−1 + γs k−1 /∥s k−1 ∥ 2 (3.29)γ = 3(g k − g k−1 ) T s k−1 − 6(f k − f k−1 ) (3.30)'Eqontac prosdiorÐsei t¸ra thn nèa exÐswsh qord c, eÐnai dunatì na upologistoÔnnèa b mata me ton Ðdio trìpo pou oi Barzilai kai Borwein upolìgisan gia thn klasikexÐswsh qord c. Pio sugkekrimèna, upojètoume ìti o pÐnakac B k thc exÐswshc (3.5)mporeÐ na grafeÐ wc B k = ˜α k I. Gia na ikanopoieÐ o pÐnakac B k thn nèa exÐswsh qord c,to ˜α k upologÐzetai ètsi ¸ste na elaqistopoieÐtai h posìthtaH parapˆnw apaÐthsh odhgeÐ sthn epilog∥s k−1 − B k ỹ k−1 ∥ (3.31)˜α k = sT k−1ỹk−1ỹ T k−1ỹk−1(3.32)

62 Neoi Algorijmoi Ekpaideushc TNDìpou s k−1 = x k − x k−1 kai y k−1 = g k − g k−1 .Summetrikˆ, mporeÐ na epilujeÐ to prìblhma min ∥B −1k s k−1 − ỹ k−1 ∥ wc proc to α k ,to opoÐo dÐnei wc lÔsh to b ma˜α k = sT k−1 s k−1s T k−1ỹk−1(3.33)Ta nèa b mata (3.32) kai (3.33) èqoun megalÔterh akrÐbeia apì ta b mata (3.7)kai (3.8) afoÔ proèrqontai apì thn nèa exÐswsh qord c h opoÐa perièqei perissìterhplhroforÐa se sqèsh me thn klasik exÐswsh qord c. An h sunˆrthsh f eÐnai tetragwniksto eujÔgrammo tm ma metaxÔ tou x k−1 kai x k , tìte oi (3.32) kai (3.33)eÐnai isodÔnamec me tic (3.7) kai (3.8), antistoÐqwc. Dhlad , se aut thn perÐptwshèqoume ˜α k = α k . Epiplèon, an h sunˆrthsh f eÐnai austhr¸c kurt , tìte èqoume0 ≤ ˜α k ≤ 2α k .3.3 Nèoc Algìrijmoc EkpaÐdeushc kai h OlikSÔgklish touSe aut thn parˆgrafo, dÐnetai ènac nèoc algìrijmoc ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn o opoÐoc basÐzetai sto b ma twn Barzilai kai Borwein kai to nèo b mapou basÐzetai sthn nèa exÐswsh qord c. Epiplèon, qrhsimopoieÐtai mia teqnik grammikc anaz thshc pou basÐzetai sthn mh monìtonh sunj kh tou Armijo (3.9). Piosugkekrimèna, gÐnetai prospˆjeia na elaqistopoihjeÐ h sunˆrthsh sfˆlmatoc E(w)qrhsimopoi¸ntac to akìloujo epanalhptikì sq maìpou η k eÐnai o rujmìc ekpaÐdeushc kai g k = ∇E(w k ).w k+1 = w k − η k g k (3.34)Gia ton rujmì ekpaÐdeushc η k qrhsimopoioÔntai ta b mata (3.8) kai (3.33). Hepilog tou katˆllhlou rujmoÔ ekpaÐdeushc katˆ thn diadikasÐa ekpaÐdeushc tou neurwnikoÔdiktÔou epitugqˆnetai qrhsimopoi¸ntac ènan mhqanismì allag c tou rujmoÔekpaÐdeushc. Efìson o rujmìc ekpaÐdeushc (3.33) eÐnai uyhlìterhc akrÐbeiac apì tonrujmì ekpaÐdeushc (3.8), qrhsimopoieÐtai mia akèraih parˆmetroc j ston algìrijmo giana apofasisteÐ an axÐzei ton kìpo h qrhsimopoÐhsh tou rujmoÔ ekpaÐdeushc (3.33). Htim tou j exartˆtai apì thn akoloujÐa u k , ìpou u k = | α k˜α k− 1| eÐnai mia posìthta poudeÐqnei pìso kontˆ eÐnai h E(w) se mia tetragwnik sunˆrthsh sto eujÔgrammo tm mapou orÐzetai apì ta w k−1 kai w k . Pio sugkekrimèna, upojètoume ìti c 3 > c 2 > c 1 eÐnaitreic jetikèc stajerèc. An u k ≤ c 1 , max(u k , u k−1 ) ≤ c 2 , max(u k , u k−1 , u k−2 ) ≤ c 3 ,tìte mporeÐ na jewrhjeÐ ìti h E(w) proseggÐzei mia tetragwnik sunˆrthsh sto eujÔgrammotm ma pou orÐzetai apì ta w k−1 kai w k , to opoÐo upodhl¸nei ìti o rujmìc

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 63ekpaÐdeushc uyhlìterhc akribeÐac eÐnai katallhlìteroc. Se aut thn perÐptwsh o a-kèraioc arijmìc j tÐjetai Ðsoc me 1 kai qrhsimopoieÐtai o rujmìc ekpaÐdeushc (3.33).Diaforetikˆ, tÐjetai j = 0 kai qrhsimopoieÐtai o rujmìc ekpaÐdeushc (3.8). O mhqanismìcallag c tou b matoc perigrˆfetai apì ton parakˆtw algìrijmo.Algìrijmoc 3 Mhqanismìc allag c b matoc.SSM(k, δ, w k , w k−1 , g k , g k−1 , f k , f k−1 , c 1 , c 2 , c 3 , lambda)1: An s T k−1 y k−1 ≤ 0 Tìte2: Jèse λ = δ, u k = 1 kai epèstreye to λ.3: Alli¸c4: Upolìgise ta α k kai ˜α k qrhsimopoi¸ntac tic sqèseic (3.8) kai (3.33), antÐstoiqa.5: Upolìgise to u k = | α k˜α k− 1|.6: Jèse j = 0.7: An u k ≤ c 1 , max i=0,1 u k−i ≤ c 2 , max i=0,1,2 u k−i ≤ c 3 Tìte8: Jèse j = 1.9: Tèloc10: An j = 1 Tìte11: Jèse λ = ˜α k .12: Alli¸c13: Jèse λ = max(1/δ, min(α k , δ))14: Tèloc15: Epèstreye to λ.16: TèlocO paronomast c α k tou klˆsmatoc sthn sqèsh tou u k mporeÐ na gÐnei mhdèn. W-stìso, apì tic sqèseic (3.8) kai (3.33) èqoumeα k˜α k= sT k−1ỹk−1s T k−1 y k−1to opoÐo eÐnai kalˆ orismèno afoÔ èqoume ìti s T k−1 y k−1 > 0.O parapˆnw mhqanismìc epilog c tou katˆllhlou rujmoÔ mˆjhshc sunduˆzetai memia strathgik grammik c anaz thshc pou qrhsimopoieÐ thn mh monìtonh sunj kh touArmijo (3.9). H mèjodoc thc grammik c anaz thshc pou qrhsimopoieÐtai ston algìrijmoekpaÐdeushc eÐnai apl kai sthrÐzetai ston upodiplasiasmì tou rujmoÔ ekpaÐdeushcìtan autìc den ikanopoieÐ thn mh monìtonh sunj kh tou Armijo (3.9). H mèjodocekpaÐdeushc prosèggishc dÔo shmeÐwn dÐnetai apì ton parakˆtw algìrijmo.

64 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc 4 Algìrijmoc EkpaÐdeushc Prosèggishc DÔo ShmeÐwn.ABB(x 0 , ϵ, µ, M, σ, δ, c 1 , c 2 , c 3 , w ⋆ , E(w ⋆ ))1: Gia k := 0 èwc µ Kˆne2: Upolìgise to E k = E(w k ) kai to g k = ∇E(w k ).3: An E k ≤ ϵ Tìte4: Epèstreye w ⋆ = w k , E(w ⋆ ) = E(w k ).5: Alli¸c6: An k = 0 Tìte7: Jèse λ = 1/∥g k ∥.8: Alli¸c9: Jèse λ =SSM(k, δ, w k , w k−1 , g k , g k−1 , f k , f k−1 , c 1 , c 2 , c 3 , λ)10: Tèloc11: 'Oso f (x k − λg k ) > max 0≤j≤min{k,M} f(x k−j ) − σ λgk T g k Kˆne12: Jèse λ = λ/2.13: Tèloc14: Jèse η k = λ kai w k+1 = w k − η k g k .15: Tèloc16: Tèloc17: Epèstreye To dÐktuo den ekpaideÔthke.Ac upotejeÐ ìti to ∇E(w) eÐnai Lipschitz suneqèc. Tìte èqoume ìtis T k−1y k−1 ≤ Ls T k−1s k−1 (3.35)ìpou L eÐnai mia jetik stajerˆ. An to s T k−1 y k−1 > 0, tìte apì tic sqèseic (3.8) kai(3.35) èqoume ìtiα k ≥ L −1 (3.36)Apì thn parapˆnw sqèsh kai ton Algìrijmo 3 èqoume ìti, an j = 1, tìte prèpei naèqoume ìtiλ = ˜α k ≥ (1 + c 3 ) −1 α k ≥ [(1 + c 3 )L] −1 (3.37)Diaforetikˆ, an to j = 0 to λ lambˆnetai apì to deÔtero b ma tou Algìrijmou 3,epÐshc èqoumeλ ≥ δ −1 (3.38)Qrhsimopoi¸ntac tic sqèseic (3.37) kai (3.38) mporeÐ na apodeiqjeÐ h olik sÔgklishtou AlgorÐjmou 4 ìpwc sto Je¸rhma 2.1 sthn ergasÐa [69].Je¸rhma 3.1. 'Estw ìti to sÔnolo L 0 = {w ∈ R n : E(w) ≤ E(w 0 )} eÐnai fragmènokai ìti to ∇E(w) eÐnai Lipschitz suneqèc se mia geitoniˆ N tou L 0 . 'Estw ìti {w k } ∞ k=0eÐnai h akoloujÐa pou parˆgetai apì ton Algìrijmo 4. Tìte, eÐte to ∇E(w k ) = 0 giakˆpoio peperasmèno k, isqÔoun oi akìloujec idiìthtec:

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 65(i) lim k→∞ ∥∇E(w k )∥ = 0,(ii) kanèna oriakì shmeÐo thc {w k } eÐnai èna topikì mègisto thc E,(iii) an o arijmìc twn stˆsimwn shmeÐwn thc E sto L 0 eÐnai peperasmènoc, tìte hakoloujÐa {w k } sugklÐnei.Apìdeixh. Gia na apodeiqjeÐ to (i), mporeÐ na qrhsimopoihjeÐ to pr¸to mèroc thc apìdeixhctou jewr matoc sÔgklishc sthn parˆgrafo 3 thc ergasÐac [29].Ac upojèsoume ìti m(k) = min(k, M). EÐnai fanerì ìti m(0) = 0 kai ìti0 ≤ m(k) ≤ min(m(k − 1) + 1, M), gia k ≥ 1.Epiplèon upˆrqei mia jetik stajerˆ λ tètoia ¸ste 0 < η k ≤ λ gia ìla ta k. 'Ontwcgia ton Algìrijmo 4 to λ = max(1/δ, min(α k , δ)). EpÐshc, upˆrqoun jetikoÐ arijmoÐσ 1 kai σ 2 tètoioi ¸ste h kateÔjunsh anaz thshc d k na ikanopoieÐ thn gk T d k ≤ −σ 1 ∥g k ∥ 2kai thn ∥d k ∥ ≤ σ 2 ∥g k ∥. 'Ontwc, ston Algìrijmo 4, h kateÔjunsh anaz thshc d k eÐnaiÐsh me to −g k gia ìla ta k kai epomènwc σ 1 = σ 2 = 1. Epomènwc, paÐrnoume thnexÐswsh (14) apì thn ergasÐa [29], h opoÐa sthn perÐptws mac paÐrnei thn morflim η k∥g k ∥ = 0k→∞Efìson to η k ≥ 1/δ η k ≥ [(1 + c 3 )L] −1 (anˆloga me thn tim tou j) gia ìla ta k,o isqurismìc (i) isqÔei. Oi isqurismoÐ (ii) kai (iii) aporrèoun ˆmesa apì to je¸rhmasÔgklishc sthn ergasÐa [29].3.4 Peiramatikˆ ApotelèsmataSthn parˆgrafo aut axiologeÐtai h epÐdosh tou proteinìmenou algìrijmou ekpaÐdeushcteqnht¸n neurwnik¸n diktÔwn (ABB) [81] kai sugkrÐnetai me thn epÐdosh twnakìloujwn gnwst¸n algorÐjmwn ekpaÐdeushc:1. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me stajerì rujmìekpaÐdeushc (BP) [74].2. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me stajerì rujmìekpaÐdeushc kai orm (BPM) [74].3. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me prosarmostikìrujmì ekpaÐdeushc kai orm (ABP) [84].4. Ton mh monìtono algìrijmo ekpaÐdeushc twn Barzilai kai Borwein (BBP) [64].

66 Neoi Algorijmoi Ekpaideushc TNDGia touc algìrijmouc ekpaÐdeushc BBP kai ABB oi parˆmetroi M, σ kai δ èqoun tictimèc 10, 10 −4 kai 10 3 , antÐstoiqa. Epiplèon, oi parˆmetroi c 1 , c 2 kai c 3 tou algìrijmouABB èqoun tic timèc 0.5 × 10 −4 , 0.1 kai 0.5, antÐstoiqa. Oi timèc twn parapˆnwparamètrwn eÐnai oi Ðdiec gia ìla ta peirˆmata pou pragmatopoi jhkan. Kai oi dÔoalgìrijmoi qrhsimopoioÔn thn Ðdia mh monìtonh strathgik grammik c anaz thshc.Epiplèon, gia thn arqikopoÐhsh twn bar¸n kai twn merolhyi¸n efarmìsthke h teqnikpou protˆjhke apì touc Nguyen kai Widrow [55] gia ìlouc touc algorÐjmouc.Ta probl mata pou qrhsimopoi jhkan gia thn axiolìghsh thc taqÔthtac sÔgklishctou proteinìmenou algìrijmou ekpaÐdeushc ABB se sqèsh me touc proanaferjèntecalgìrijmouc ekpaÐdeushc eÐnai ta akìlouja:1. To prìblhma tou ApokleistikoÔ-EITE.2. To prìblhma thc isotimÐac twn 3-bit.3. To prìblhma thc anagn¸rishc twn kefalaÐwn grammˆtwn thc Agglik c alfab -tou.4. To prìblhma thc prosèggishc thc suneqoÔc trigwnometrik c sunˆrthshc.Gia kˆje prìblhma, paratÐjetai ènac pÐnakac o opoÐoc sunoyÐzei thn apìdosh twnalgorÐjmwn gia prosomoi¸seic pou èftasan se lÔsh entìc enìc prokajorismènou o-rÐou upologism¸n thc sunˆrthshc sfˆlmatoc pou orÐzetai se kˆje prìblhma. Oianaferìmenec stouc pÐnakec parˆmetroi eÐnai: Min o elˆqistoc arijmìc epanal yewn,Max o mègistoc arijmìc epanal yewn, µ h mèsh tim twn epanal yewn, σ h tupikapìklish, kai EpituqÐa o arijmìc twn petuqhmènwn prosomoi¸sewn apì èna sÔnolo1000 dokim¸n. Se perÐptwsh pou ènac algìrijmoc apotÔqei na sugklÐnei mèsa stoprokajorismèno ìrio twn upologism¸n thc sunˆrthshc sfˆlmatoc, jewreÐtai ìti apètuqena ekpaideÔsei to teqnhtì neurwnikì dÐktuo kai oi epanal yeic thc sugkekrimènhcprosomoÐwshc den perilambˆnontai sthn statistik anˆlush tou algorÐjmou.Se autì to shmeÐo, prèpei na epishmanjeÐ ìti genikˆ mia epanˆlhyh enìc algorÐjmouekpaÐdeushc shmaÐnei ìti ìla ta prìtupa ekpaÐdeushc èqoun parousiasteÐ stoteqnhtì neurwnikì dÐktuo. Stouc klasikoÔc algorÐjmouc, ìpwc h BP, kˆje epanˆlhyhantistoiqeÐ se èna sunarthsiakì upologismì thc sunˆrthshc sfˆlmatoc kai ènanupologismì thc klÐshc (parˆgwgoi pr¸thc tˆxewc). Stouc algorÐjmouc BBP kaiABB lambˆnoun q¸ra perissìteroi sunarthsiakoÐ upologismoÐ anˆ epanˆlhyh, gegonìcpou ofeÐletai sthn parousÐa thc mh monìtonhc teqnik c grammik c anaz thshc.ExaitÐac thc fÔsewc thc grammik c anazht sewc pou qrhsimopoieÐtai stouc algorÐjmoucautoÔc, en¸ mporeÐ na apaitoÔntai perissìteroi sunarthsiakoÐ upologismoÐ anˆepanˆlhyh, autì den isqÔei gia ton upologismì thc klÐsewc h opoÐa upologÐzetai miaforˆ anˆ epanˆlhyh. Lambˆnontac upìyin ìti oi upologismoÐ klÐsewc eÐnai akribìteroi

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 67apì touc sunarthsiakoÔc upologismoÔc tou sfˆlmatoc, eÐnai fanerì ìti oi algìrijmoiautoÐ eÐnai grhgorìteroi afoÔ apaitoÔn ligìterec prˆxeic kinht c upodiastol c.Gia touc parapˆnw lìgouc, stouc pÐnakec twn apotelesmˆtwn, gia tic mejìdouc BBPkai ABB anafèrontai dÔo seirèc apotelesmˆtwn, h pr¸th gia touc upologismoÔc twntim¸n thc sunˆrthshc sfˆlmatoc kai h deÔterh gia touc upologismoÔc twn klÐsewn.3.4.1 Apokleistikì-EITEGia to prìblhma tou apokleistikoÔ-EITE (blèpe Parˆrthma A') qrhsimopoi same èna2 − 2 − 1 teqnhtì neurwnikì dÐktuo (6 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn tim 0.1, antÐ thc proepilegmènhc tim c 0.01, ètsi ¸ste na epitaqunjeÐh sÔgklish touc, afoÔ me thn proepilegmènh tim sugklÐnoun polÔ argˆ se autì toprìblhma. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 3.1.Algìrijmoc Min Max µ σ EpituqÐaBP 31 975 100.9 124.8 71.1%BPM 24 962 128.3 161.8 75.7%ABP 19 865 47.3 69.1 70.8%7 998 65.3 131.4BBP 82.7%4 595 40.6 79.47 931 56.1 125.1ABB 82.7%6 556 39.2 77.7PÐnakac 3.1: Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITEO algìrijmoc BPM apèdwse kalÔtera ìson aforˆ to posostì epituqÐac se sqèshme ton algìrijmo BP, allˆ qreiˆsthke perissìterec epanal yeic. O algìrijmocABP, an kai eÐqe shmantikˆ mikrìtero mèso arijmì epanal yewn, tan o qeirìterocalgìrijmoc se posostì epituqÐac. O algìrijmoc ekpaÐdeushc BBP tan shmantikˆkalÔteroc apì touc klasikoÔc algìrijmouc ekpaÐdeushc tìso se posostì epituqÐacìso kai se kìstoc. Pio sugkekrimèna, an kai qreiˆsthke perissìterouc upologismoÔctwn tim¸n thc sunˆrthshc sfˆlmatoc se sqèsh me thn ABP, qreiˆsthke arketˆ ligìteroucupologismoÔc klÐshc. 'Oson aforˆ ton proteinìmeno algìrijmo ekpaÐdeushcABB, parousÐase to Ðdio posostì epituqÐac me ton algìrijmo BBP, allˆ belti¸jhkeìson aforˆ touc sunarthsiakoÔc upologismoÔc kai touc upologismoÔc klÐsewc,gegonìc pou ton kajistˆ kalÔtero.

68 Neoi Algorijmoi Ekpaideushc TND3.4.2 IsotimÐa twn 3-bitGia to prìblhma thc isotimÐac twn 3-bit (blèpe Parˆrthma A') qrhsimopoi same èna3 − 2 − 1 teqnhtì neurwnikì dÐktuo (8 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn tim 0.1, antÐ thc proepilegmènhc tim c 0.01, ètsi ¸ste na epitaqunjeÐh sÔgklish touc, afoÔ me thn proepilegmènh tim sugklÐnoun polÔ argˆ se autì toprìblhma. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 3.2.Algìrijmoc Min Max µ σ EpituqÐaBP - - - - 0.0%BPM 178 995 486.1 203.3 52.6%ABP 426 959 602.9 123.4 48.5%50 999 233.5 192.3BBP 72.9%33 615 147.1 115.350 983 222.4 185.5ABB 76.0%33 619 148.4 114.7PÐnakac 3.2: Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bitO algìrijmoc ekpaÐdeushc BP apètuqe na sugklÐnei mèsa sto ìrio twn upologism¸ntwn tim¸n thc sunˆrthshc sfˆlmatoc se ìlec tic prosomoi¸seic. AxÐzei naanaferjeÐ ìti dokimˆsame diaforetikèc timèc gia thn tim thc paramètrou tou rujmoÔekpaÐdeushc, allˆ to apotèlesma tan to Ðdio. O algìrijmoc BPM apèdwse arketˆkalˆ, en¸ tan kalÔteroc tìso se posostì epituqÐac ìso kai se upologismoÔc tim¸nthc sunˆrthshc sfˆlmatoc (kai upologismoÔc klÐsewc), anaforikˆ me ton algìrijmoekpaÐdeushc ABP. O algìrijmoc BBP apèdwse polÔ kalÔtera ìson aforˆ to posostìepituqÐac afoÔ h diaforˆ tou se posostì epituqÐac apì ton algìrijmo BPM tan20%. Epiplèon, eÐqe polÔ mikrìtero kìstoc se sunarthsiakoÔc upologismoÔc kai u-pologismoÔc klÐsewc. O proteinìmenoc algìrijmoc ekpaÐdeushc ABB parousÐase tokalÔtero posostì epituqÐac sto parˆdeigma autì. EpÐshc, parousÐase mikrìtero kìstocse sunarthsiakoÔc upologismoÔc katˆ mèso ìro apì thn BBP en¸ o mèsoc ìroctwn upologism¸n klÐsewc tan perÐpou o Ðdioc.

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 693.4.3 Anagn¸rish KefalaÐwn GrammˆtwnGia to prìblhma thc anagn¸rishc kefalaÐwn grammˆtwn thc Agglik c alfab tou (blèpeParˆrthma A') qrhsimopoi same èna 35 − 30 − 26 teqnhtì neurwnikì dÐktuo (1830bˆrh kai 56 merolhyÐec). To krufì epÐpedo tou neurwnikoÔ diktÔou eÐnai basismèno seneur¸nec me logistik sunˆrthsh energopoÐhshc, en¸ to epÐpedo exìdou eÐnai basismènose neur¸nec grammik c sunˆrthshc energopoÐhshc. H sunj kh termatismoÔ eÐnaiE ≤ 0.1 kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoceÐnai 2000. Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìcekpaÐdeushc orÐsthke sthn proepilegmènh tim 0.01. H arqikopoÐhsh twn bar¸n ègineme thn teqnik twn Nguyen kai Widrow, allˆ sth sunèqeia ta bˆrh metaxÔ tou krufoÔepipèdou kai tou epipèdou thc exìdou pollaplasiˆsthkan me 0.01. Ta apotelèsmatatwn arijmhtik¸n peiramˆtwn parousiˆzontai ston PÐnaka 3.3.Algìrijmoc Min Max µ σ EpituqÐaBP 1044 1998 1548.4 199.7 75.6%BPM 1199 1899 1560.9 183.5 3.6%ABP 1230 1999 1776.5 157.5 16.9%101 362 178.9 37.5BBP 100.0%64 222 116.1 22.1103 350 168.7 32.2ABB 100.0%68 215 108.2 20.3PÐnakac 3.3: Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwnApì touc klasikoÔc algìrijmouc ekpaÐdeushc, o BP parousiˆzei thn kalÔterh e-pÐdosh tìso se posostì epituqÐac ìso kai se upologismoÔc thc tim c thc sunˆrthshcsfˆlmatoc. Eidikˆ h BPM parousiˆzei polÔ mikrì posostì epituqÐac. O algìrijmocBBP parousiˆzei exairetikì posostì epituqÐac, afoÔ sugklÐnei se ìlec tic prosomoi¸seic.Epiplèon h apìdosh tou se sunarthsiakoÔc upologismoÔc kai upologismoÔcklÐsewc eÐnai exairetik . Wstìso, o proteinìmenoc algìrijmoc, ABB, epÐshc sugklÐneise ìlec tic prosomoi¸seic, allˆ parousiˆzei kalÔterh epÐdosh tìso stouc upologismoÔcthc tim c thc sunˆrthshc sfˆlmatoc ìso kai stouc upologismoÔc klÐsewc.AxÐzei na shmeiwjeÐ ìti o mègistoc arijmìc sunarthsiak¸n upologism¸n pou qreiˆsthkanoi BBP kai ABB eÐnai mikrìteroc apì 1000 to opoÐo shmaÐnei ìti gia autoÔctouc algìrijmouc eÐnai dunatì na mei¸soume to ìrio twn sunarthsiak¸n upologism¸nqwrÐc na ephreasteÐ to posostì epituqÐac. Se aut thn perÐptwsh ìmwc, kanènac apìtouc klasikoÔc algìrijmouc den ja sunèkline, afoÔ o elˆqistoc arijmìc upologism¸nthc tim c thc sunˆrthshc sfˆlmatoc eÐnai megalÔteroc apì 1000.

70 Neoi Algorijmoi Ekpaideushc TND3.4.4 Prosèggish SuneqoÔc Trigwnometrik c SunˆrthshcGia to prìblhma thc prosèggishc miac suneqoÔc trigwnometrik c sunˆrthshc (blèpeParˆrthma A') qrhsimopoi same èna 1 − 15 − 1 teqnhtì neurwnikì dÐktuo (30 bˆrh kai16 merolhyÐec). To krufì epÐpedo tou neurwnikoÔ diktÔou eÐnai basismèno se neur¸necme logistik sunˆrthsh energopoÐhshc, en¸ to epÐpedo exìdou eÐnai basismèno seneur¸nec grammik c sunˆrthshc energopoÐhshc. H sunj kh termatismoÔ eÐnai E ≤ 0.1kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn proepilegmènh tim 0.01. Ta apotelèsmata twn exomoi¸sewn parousiˆzontaiston PÐnaka 3.4.Algìrijmoc Min Max µ σ EpituqÐaBP 318 985 710.4 176.3 12.0%BPM 323 996 688.2 172.6 11.9%ABP 173 996 627.1 221.5 26.7%30 995 238.2 183.7BBP 93.3%25 599 149.6 108.630 988 225.8 183.1ABB 93.9%25 612 150.1 215.2PÐnakac 3.4: Sugkritikˆ apotelèsmata gia to prìblhma thc prosèggishc suneqoÔctrigwnometrik c sunˆrthshc.Se autì to prìblhma, o algìrijmoc ekpaÐdeushc ABP eÐqe thn kalÔterh epÐdoshmetaxÔ twn klasik¸n algorÐjmwn ekpaÐdeushc se posostì epituqÐac kai se mèso ìroupologism¸n twn tim¸n thc sunˆrthshc sfˆlmatoc. O algìrijmoc BBP apèdwse polÔkalÔtera ìson aforˆ to posostì epituqÐac afoÔ h diaforˆ tou se posostì epituqÐacapì ton algìrijmo ABP tan pˆnw apì 60%. Epiplèon, eÐqe polÔ mikrìtero kìstoc sesunarthsiakoÔc upologismoÔc kai upologismoÔc klÐsewc. O proteinìmenoc algìrijmocekpaÐdeushc ABB parousÐase to kalÔtero posostì epituqÐac sto parˆdeigma autì.EpÐshc, parousÐase mikrìtero kìstoc se sunarthsiakoÔc upologismoÔc katˆ mèsoìro apì thn BBP en¸ o mèsoc ìroc twn upologism¸n klÐsewc tan perÐpou o Ðdioc.3.5 SumperˆsmataSe autì to kefˆlaio anaptÔqjhke ènac nèoc rujmìc ekpaÐdeushc proseggÐzontac mianèa exÐswsh qord c, pou protˆjhke apì ton Zhang [90], h opoÐa qrhsimopoieÐ plhroforÐaparag¸gwn kai sunarthsiak¸n tim¸n. Sthn sunèqeia, anaptÔqjhke ènac nèoc

Algorijmoc Ekpaideushc Proseggishc Duo Shmeiwn 71algìrijmoc ekpaÐdeushc o opoÐoc qrhsimopoieÐ wc kateÔjunsh anaz thshc thn kateÔjunshthc pio apìtomhc kajìdou. O algìrijmoc autìc qrhsimopoieÐ ènan katˆllhlomhqanismì epilog c tou rujmoÔ ekpaÐdeushc ¸ste na epilègei kˆje forˆ ton kalÔterorujmì ekpaÐdeushc (metaxÔ tou nèou rujmoÔ ekpaÐdeushc kai tou rujmoÔ ekpaÐdeushctwn Barzilai kai Borwein). Epiplèon qrhsimopoieÐtai mia mh monìtonh teqnik grammikc anaz thshc gia thn apodoq tou rujmoÔ ekpaÐdeushc kai apodeÐqjhke h oliksÔgklish tou algorÐjmou.Sth sunèqeia axiolog jhke h apìdosh tou nèou algorÐjmou sugkrÐnontac ton mediˆforouc gnwstoÔc algìrijmouc thc Ðdiac tˆxhc. Gia ton skopì autì qrhsimopoi jhkangnwstˆ probl mata ekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn. Ta apotelèsmatatwn peiramˆtwn èdeixan ìti o proteinìmenoc algìrijmoc uperèqei tìso se posostˆ e-pituqÐac ìso kai se taqÔthta sÔgklishc. PisteÔoume ìti o sugkekrimènoc algìrijmocmporeÐ na beltiwjeÐ peraitèrw qrhsimopoi¸ntac katallhlìtero mhqanismì epilog c tourujmoÔ ekpaÐdeushc. EpÐshc, qrhsimopoi¸ntac mia pio apodotik teqnik grammik c a-naz thshc eÐmaste bèbaioi ìti h taqÔthta tou algorÐjmou ja beltiwjeÐ perissìtero.

72 Neoi Algorijmoi Ekpaideushc TND

Kefalaio 4Nèa Oikogèneia AlgorÐjmwnEkpaÐdeushc Suzug¸n KlÐsewnSe autì to kefˆlaio parousiˆzontai merikoÐ nèoi apotelesmatikoÐ algìrijmoi ekpaÐdeushcoi opoÐoi basÐzontai stic mejìdouc beltistopoÐhshc suzug¸n klÐsewn. Stoucupˆrqontec algìrijmouc ekpaÐdeushc suzug¸n klÐsewn prostÐjetai ènac nèoc algìrijmocekpaÐdeushc pou basÐzetai sthn mèjodo suzug¸n klÐsewn tou Perry [60]. Hmèjodoc tou Perry èqei apodeiqjeÐ ìti eÐnai mia polÔ apotelesmatik mèjodoc stoplaÐsio thc beltistopoÐhshc qwrÐc periorismoÔc, allˆ den eÐqe efarmosteÐ potè sthnperioq thc ekpaÐdeushc twn teqnht¸n neurwnik¸n diktÔwn. Epiplèon, proteÐnetainèec mèjodoi suzug¸n klÐsewn, pou onomˆzontai klimakwtèc mèjodoi suzug¸n klÐsewn,kai prokÔptoun apì tic Ðdiec arqèc pou proèrqontai oi gnwstèc mèjodoi suzug¸nklÐsewn twn Hestenes-Stiefel, Fletcher-Reeves, Polak-Ribière kai Perry. Aut h kathgorÐamejìdwn basÐzetai sth fasmatik parˆmetro klimˆkwshc pou protˆjhke apìtouc Barzilai kai Borwein [5]. H fasmatik parˆmetroc klimˆkwshc perièqei plhroforÐadeutèrac tˆxewc qwrÐc na apaiteÐtai o upologismìc tou EssianoÔ pÐnaka. Epiplèon,enswmat¸netai stouc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn mia apodotik teqnikgrammik c anaz thshc pou basÐzetai stic sunj kec tou Wolfe kai sthn diasfalismènhkubik parembol [77]. Akìmh, h parˆmetroc tou arqikoÔ rujmoÔ ekpaÐdeushc,pou trofodoteÐtai ston algìrijmo thc grammik c anaz thshc, prosarmìzetai autìmatase kˆje epanˆlhyh sÔmfwna me èna kleistì tÔpo tou protˆjhke stic ergasÐec [77]kai [80]. Sth sunèqeia, efarmìzetai mia apotelesmatik diadikasÐa epanekkÐnhshc ètsi¸ste na beltiwjoÔn peraitèrw oi algìrijmoi ekpaÐdeushc suzug¸n klÐsewn kai na a-podeiqjeÐ h olik touc sÔgklish. Tèloc, parousiˆzontai ta peiramatikˆ apotelèsmatagia diˆfora probl mata ekpaÐdeushc.73

74 Neoi Algorijmoi Ekpaideushc TND4.1 Algìrijmoi EkpaÐdeushc Suzug¸n KlÐsewnOi mèjodoi suzug¸n klÐsewn eÐnai mia polÔ shmantik kathgorÐa mejìdwn gia thnelaqistopoÐhsh suneq¸n sunart sewn, eidikìtera ìtan h diˆstash eÐnai megˆlh [57].MporoÔn na jewrhjoÔn wc mèjodoi suzug¸n kateujÔnsewn ektrop c thc klÐshc, oiopoÐec mporoÔn na kathgoriopoihjoÔn metaxÔ thc mejìdou thc pio apìtomhc kajìdoukai thc mejìdou tou Newton. To kÔrio pleonèkthmˆ touc eÐnai ìti den apaitoÔn thnapoj keush pinˆkwn, ìpwc h mèjodoc Newton, kai ìti eÐnai sqediasmènec ¸ste nasugklÐnoun taqÔtera apì thn mèjodo thc pio apìtomhc kajìdou.Oi mèjodoi suzug¸n klÐsewn sugklÐnoun to polÔ se n epanal yeic gia tetragwnikˆprobl mata beltistopoÐhshc qwrÐc periorismoÔc sto R n ìtan qrhsimopoioÔntaiakribeÐc grammikèc anazht seic. Wstìso, efarmìzontai kai sthn perÐptwsh twn mhtetragwnik¸n problhmˆtwn, afoÔ oi suneqeÐc sunart seic parousiˆzoun tetragwniksumperiforˆ sthn perioq tou bèltistou. Se autèc tic peript¸seic, gia na beltiwjeÐo rujmìc sÔgklishc, o algìrijmoc epanekkineÐtai kˆje n epanal yeic. Pollèc tropopoiseic thc strathgik c twn suzug¸n klÐsewn èqoun protajeÐ sthn bibliografÐabasismènec sthn qalˆrwsh twn tetragwnik¸n upojèsewn thc akriboÔc grammik canaz thshc [25], [56]. EpÐshc, èqoun protajeÐ pollèc sunj kec epanekkÐnhshc toualgorÐjmou twn suzug¸n klÐsewn [17], [40], [66].H mèjodoc suzug¸n klÐsewn dhmiourgeÐ mia kateÔjunsh anaz thshc h opoÐa eÐnaiamoibaÐa suzug c me tic prohgoÔmenec kateujÔnseic anaz thshc se sqèsh me èna dojènjetikˆ orismèno pÐnaka H, kai brÐskei to bèltisto shmeÐo se aut thn kateÔjunshqrhsimopoi¸ntac mia teqnik grammik c anaz thshc. DÔo kateujÔnseic anaz thshc d ikai d j lègetai ìti eÐnai amoibaÐa suzug c wc proc ton H an ikanopoieÐtai h akìloujhsunj khd T i Hd j = 0 ìpou i ≠ j. (4.1)Me ˆlla lìgia, h epìmenh kateÔjunsh anaz thshc upologÐzetai wc ènac grammikìcsunduasmìc thc prohgoÔmenhc kateÔjunshc kai thc trèqousac klÐshc me tètoio trìpo¸ste ta b mata elaqistopoÐhshc se ìlec tic prohgoÔmenec kateujÔnseic na mhnparembˆllontai. H kateÔjunsh anaz thshc mporeÐ na kajoristeÐ wcd k ={−g k , gia k = 0−g k + β k d k−1 , gia k > 0.(4.2)ìpou β k eÐnai mia upì kajorismì parˆmetroc ètsi ¸ste h kateÔjunsh d k na gÐnei hk-ost suzug c kateÔjunsh. Upˆrqoun polloÐ trìpoi gia ton upologismì tou β k .O kajènac dhmiourgeÐ mia xeqwrist mh grammik mèjodo suzug¸n klÐsewn h opoÐaèqei thn dik thc idiìthta sÔgklishc kai arijmhtik apìdosh. PolloÐ tÔpoi gia tonupologismì thc paramètrou β k èqoun protajeÐ sthn bibliografÐa. Oi pio axioshmeÐwtoi

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 75kai eurèwc qrhsimopoioÔmenoi sthn perioq thc ekpaÐdeushc twn teqnht¸n neurwnik¸ndiktÔwn eÐnai oi akìloujoi:• Hestenes–Stiefel (HS) [33]• Fletcher–Reeves (FR) [26]• Polak–Ribière (PR) [65]β HSk = gT k y kd T k−1 y kβ F Rk = gT k g kg T k−1 g k−1β P Rk = gT k y kg T k−1 g k−1(4.3)(4.4)(4.5)ìpou y k = g k − g k−1 . EpÐshc, dhl¸noume ed¸ gia mellontik anaforˆ ìti s k = w k −w k−1 .Sthn perioq thc arijmhtik c beltistopoÐhshc qwrÐc periorismoÔc, o Perry sthnergasÐa [60] prìteine mia nèa apodotik mèjodo suzug¸n klÐsewn h opoÐa upertereÐshmantikˆ apì tic proanaferjeÐsec klasikèc mejìdouc suzug¸n klÐsewn. H mèjodocaut den èqei qrhsimopoihjeÐ potè sthn perioq twn teqnht¸n neurwnik¸n diktÔwn.Epomènwc, h mèjodoc suzug¸n klÐsewn tou Perry parousiˆzetai sth sunèqeia, prosarmosmènhsthn perioq thc ekpaÐdeushc twn teqnht¸n neurwnik¸n diktÔwn.O Perry parat rhse sthn parapˆnw ergasÐa ìti qrhsimopoi¸ntac thn epilog twnHestenes–Stiefel gia thn parˆmetro β k h kateÔjunsh d k , gia k > 0, sthn exÐswsh (4.2)mporeÐ na epanadiatupwjeÐ wc ex c:[d k = − I − s kykT ]yk T s g k ≡ −Dk HS g k (4.6)kShmei¸noume ìti o pÐnakac DkHS eÐnai mia prosèggish tou antistrìfou tou EssianoÔpÐnaka, allˆ ìqi summetrikìc. Wc ek toÔtou, den apoteleÐ mia yeudì Newton qwrÐcmn mh ananèwsh tou pÐnaka [47]. Pio katˆllhla, an o D k upodhl¸netai wc mia prosèggishtou antistrìfou tou EssianoÔ pÐnaka, h sunj kh yeudì Newton apaiteÐ ìtiD k y k = s k bˆsei summetrÐac ìtiy T k D k = s T k (4.7)Ektìc apì thn mh summetrÐa, h exÐswsh (4.7) apofèrei ìti yk T DHS k = 0. O Perry[60] shmei¸nei ìti kˆtw apì mh akribeÐc grammikèc anazht seic, eÐnai pio skìpimo naepilèxoume ìti d k = −Dk P g k, ìpou h prosèggish tou antÐstrofou EssianoÔ pÐnakaepilègetai na ikanopoieÐ thn sunj kh yeudì Newton (4.7) antÐ na ikanopoieÐ aplˆD P k

76 Neoi Algorijmoi Ekpaideushc TNDthn H k -suzugÐa (4.1). 'Etsi, prèpei na isqÔei y T k d k = −y T k DP k g k = −s T k g k. Antikajist¸ntacautì sthn exÐswsh (4.2) o Perry dÐnei thn dik tou epilog gia to β k kai thnantÐstoiqh kateÔjunsh anaz thshc d k . Dhlad ,βk P = (y k − s k ) T g kyk T d (4.8)k−1[d k = − I − s kykTyk T s − s ks T ]kk yk T s g k ≡ −Dk P g k (4.9)kShmei¸netai ìti ènac epiplèon ìroc prostèjhke ston pÐnaka DkHS gia na paraqjeÐ hprosèggish DkP kai h epilog tou Perry eÐnai h Ðdia me thn epilog twn Hestenes–Stiefel/Polak–Ribière an qrhsimopoihjoÔn akribeÐc grammikèc anazht seic. Kˆtw apìautèc tic sunj kec, h tropopoÐhsh tÐjetai se isqÔ ìtan s T k g k ≠ 0.SunoyÐzontac ta prohgoÔmena, se aut thn parˆgrafo kataskeuˆsthke mia pl -rhc kathgorÐa algorÐjmwn ekpaÐdeushc suzug¸n klÐsewn. ApoteleÐtai apì touc dhgnwstoÔc algorÐjmouc ekpaÐdeushc neurwnik¸n diktÔwn HS, FR kai PR kai tou nèoualgorÐjmou ekpaÐdeushc tou Perry pou protˆjhke prohgoumènwc. Sth sunèqeia gÐnetaianaforˆ se aut thn kathgorÐa wc h klasik kathgorÐa algorÐjmwn ekpaÐdeushcsuzug¸n klÐsewn.4.2 KlimakwtoÐ Algìrijmoi EkpaÐdeushc Suzug¸nKlÐsewnSe aut thn parˆgrafo, lambˆnontac upìyin thn ergasÐa twn Birgin kai Martinez [9],upotÐjetai ìti mia pio genik morf thc kateÔjunshc anaz thshc suzug¸n klÐsewndÐnetai apì thn exÐswsh{−g k , gia k = 0d k =(4.10)−ϑ k g k + β k d k−1 , gia k > 0.ìpou ϑ k kai β k eÐnai upì kajorismì parˆmetroi ètsi ¸ste h kateÔjunsh d k na gÐnei h k-ost suzug c kateÔjunsh. Shmei¸netai ìti an ϑ k = 1 tìte h kateÔjunsh anaz thshc(4.10) anˆgetai sthn klasik kateÔjunsh suzug¸n klÐsewn (4.2).Ac upotejeÐ t¸ra ìti h kateÔjunsh d k eÐnai H k -suzug c wc proc thn kateÔjunshd k−1 , epibˆllontac thn sunj kh suzugÐac (4.1), dhlad d T k−1 H kd k = 0, ìpou H k eÐnaio Essianìc pÐnakac thc sunˆrthshc sfˆlmatoc E sto shmeÐo w k . Qrhsimopoi¸ntacmia tetragwnik prosèggish thc sunˆrthshc sfˆlmatoc E sto shmeÐo w k mazÐ me thnexÐswshw k+1 = w k + α k d k , k = 0, 1, 2, . . . (4.11)

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 77èqoume ìti y k = α k−1 H k d k−1 . Epomènwc h sunj kh suzugÐac apaiteÐ ìtiAntikajist¸ntac thn (4.10) sthn (4.12) èqoumed T k y k = 0 (4.12)β k = ϑ kg T k y kd T k−1 y k(4.13)ParathreÐtai ìti sthn perÐptwsh pou to ϑ k = 1 o tÔpoc (4.13) anˆgetai sthn epilogtwn Hestenes-Stiefel (4.3).An upotejeÐ ìti ekteloÔntai akribeÐc grammikèc anazht seic, tìte ikanopoieÐtaih exÐswsh gk T d k−1 = 0. Metˆ apì merikoÔc aploÔc upologismoÔc, o tÔpoc (4.13)metatrèpetai wcβ k =ϑ k gk T y kϑ k−1 gk−1 T g k−1(4.14)o opoÐoc eÐnai h epilog twn Polak–Ribière (4.5) ìtan to ϑ k = ϑ k−1 = 1.Epiplèon, ìtan h sunˆrthsh sfˆlmatoc E eÐnai tetragwnik , ìlec oi kateujÔnseicpou parˆgontai apì thn exÐswsh (4.10) qrhsimopoi¸ntac ton tÔpo (4.14) eÐnai metaxÔtouc suzugeÐc, kai oi klÐseic thc E se diaforetikèc epanal yeic eÐnai metaxÔ toucorjog¸niec. Dhlad isqÔei ìti gk T g k−1 = 0. 'Etsi, gia tetragwnikèc sunart seickai qrhsimopoi¸ntac akribeÐc grammikèc anazht seic o tÔpoc (4.14) metatrèpetai stontÔpoϑ k gk T β k =g kϑ k−1 gk−1 T g (4.15)k−1o opoÐoc eÐnai h epilog twn Fletcher–Reeves (4.4) ìtan to ϑ k = ϑ k−1 = 1.An epiblhjeÐ ìti h genik morf twn kateujÔnsewn anaz thshc suzug¸n klÐsewnpou dÐnetai apì thn exÐswsh (4.10), qrhsimopoi¸ntac thn genÐkeush tou tÔpou HS giato β k (4.13), ikanopoieÐ thn summetrik yeudì-Newton exÐswsh (4.7) parˆ thn sunj khH k -suzugÐac (4.1), tìte kajorÐzetai h genikeumènh epilog tou Perry h opoÐa dÐnetaiapì thn exÐswshβ k = (ϑ ky k − s k ) T g ky T k d k−1(4.16)Shmei¸netai ìti gia ϑ k = 1 o tÔpoc (4.16) anˆgetai sthn klasik epilog tou Perrygia to β k pou dÐnetai apì ton tÔpo (4.8).4.2.1 Parˆmetroc KlimˆkwshcSto shmeÐo autì, o stìqoc eÐnai na brejeÐ mia katˆllhlh epilog gia thn parˆmetroklimˆkwshc ϑ k ètsi ¸ste na beltiwjeÐ h apìdosh kai h taqÔthta sÔgklishc twn klasik¸nalgorÐjmwn ekpaÐdeushc suzug¸n klÐsewn. Mia kal kai apodotik epilog gia

78 Neoi Algorijmoi Ekpaideushc TNDthn parˆmetro klimˆkwshc eÐnai to fasmatikì b ma pou protˆjhke apì touc Barzilaikai Borwein [5], dhladϑ k = sT k s ks T k y k(4.17)Oi Barzilai kai Borwein prìteinan autì to b ma ètsi ¸ste na stajeropoi soun kaina epitaqÔnoun thn mèjodo thc pio apìtomhc kajìdou. Ta peiramatikˆ apotelèsmataèdeixan ìti to fasmatikì b ma belti¸nei shmantikˆ thn apìdosh thc mejìdou thcpio apìtomhc kajìdou. Argìtera, o Raydan sthn ergasÐa [69] sundÔase to fasmatikìb ma me mia mh monìtonh strathgik grammik c anaz thshc [29] kai par gage miaeÔrwsth mèjodo elaqistopoÐhshc pou mporeÐ na sugkrijeÐ me tic klasikèc mejìdoucsuzug¸n klÐsewn. Stic ergasÐec [63] kai [64], h mèjodoc tou Raydan metafèrjhkeepituq¸c sthn perioq twn teqnht¸n neurwnik¸n diktÔwn me polÔ kalˆ apotelèsmata.H epituqÐa tou fasmatikoÔ b matoc èrqetai apì to gegonìc ìti to ϑ k eÐnai proseggÐseictou phlÐkou Rayleigh gia kˆpoia tim 1/λ i . 'Eqontac sto mualì thn exÐswshqord c, B k s k = y k , mporoÔme na epalhjeuteÐ ìti to phlÐko Rayleigh pou dÐnetai apìthn exÐswsh[∫ 1]/R q = s T k ∇ 2 E(w k + τs k ) dτ s k sTk s kantistoiqeÐ ston mèso ìro tou EssianoÔ pÐnaka0∫ 10∇ 2 E(w k + ts k )dtsto eujÔgrammo tm ma metaxÔ tou w k kai w k+1 . Epomènwc to R q eÐnai mia kal prosèggishthc idiotim c λ i gia thn opoÐa to s k eÐnai to antÐstoiqo idiodiˆnusma. EpomènwceÐnai profanèc ìti to b ma (5.9) sqetÐzetai me tic idiotimèc tou EssianoÔ pÐnaka sthntim tou elaqÐstou kai ìqi me thn sunarthsiak tim . Autì shmaÐnei ìti to fasmatikìb ma perièqei plhroforÐa deutèrac tˆxewc qwrÐc na apaiteÐtai o upologismìc touEssianoÔ pÐnaka.H qr sh tou fasmatikoÔ b matoc ϑ k (5.9) wc parˆmetroc klimˆkwshc sthn genikeumènhkateÔjunsh anaz thshc suzug¸n klÐsewn (4.10) kai stouc genikeumènouctÔpouc gia to β k , dhlad gia touc tÔpouc (4.13), (4.15), (4.14) kai (4.16), odhgeÐ semia nèa kathgorÐa algìrijmwn ekpaÐdeushc suzug¸n klÐsewn. To ìnoma thc kathgorÐacaut c eÐnai klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn Self-ScaledConjugate Gradient (S-SCG).4.3 Genikìc Algìrijmoc EkpaÐdeushcSe aut thn parˆgrafo dÐnetai ènac genikìc algìrijmoc ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn pou basÐzetai stic mejìdouc suzug¸n klÐsewn. Prin parousiasteÐ h

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 79diadikasÐa ekpaÐdeushc, gÐnetai mia leptomer c parousÐash ìson aforˆ tic sunj kecepanekkÐnhshc, tic teqnikèc grammik c anaz thshc kai ton arqikì rujmì ekpaÐdeushc.4.3.1 DiadikasÐec EpanekkÐnhshcMia tropopoÐhsh pou qrhsimopoieÐtai suqnˆ stouc mh grammikoÔc algorÐjmouc suzug¸nklÐsewn eÐnai h epanekkÐnhsh tou algorÐjmou kˆje n epanal yeic jètontac thnparˆmetro β k = 0 sthn exÐswsh (4.10), dhlad efarmìzontac mia epanˆlhyh qrhsimopoi¸ntacthn kateÔjunsh thc pio apìtomhc kajìdou. H epanekkÐnhsh uphreteÐ toskopì thc periodik c ananèwshc tou algorÐjmou, diagrˆfontac paliˆ plhroforÐa pouden eÐnai plèon qr simh. 'Eqei apodeiqjeÐ èna isqurì jewrhtikì apotèlesma ìson aforˆthn epanekkÐnhsh. Pio sugkekrimèna, odhgeÐ se tetragwnik sÔgklish n-bhmˆtwn,dhlad∥w k+n − w∥ = O(∥w k − w ∗ ∥ 2 ) (4.18)To apotèlesma autì den prokaleÐ èkplhxh. Ac jewr soume ìti h sunˆrthsh E eÐnaikurt tetragwnik sthn perioq thc upì anaz thsh lÔshc, allˆ mh tetragwnik stoupìloipo pedÐo orismoÔ thc. Upojètontac ìti o algìrijmoc sugklÐnei sthn upì anazthsh lÔsh, oi epanal yeic ja odhghjoÔn telikˆ sthn tetragwnik perioq . Kˆpoiastigm o algìrijmoc ja epanekkinhjeÐ se aut thn perioq kai apì to shmeÐo autìh sumperiforˆ tou ja eÐnai Ðdia me aut thc grammik c mejìdou suzug¸n klÐsewn.Dhlad , o algìrijmoc ja termatÐsei se peperasmèno arijmì epanal yewn mèsa sticepìmenec n epanal yeic apì thn epanekkÐnhsh. H epanekkÐnhsh eÐnai shmantik giatÐh idiìthta tou peperasmènou termatismoÔ thc grammik c mejìdou suzug¸n klÐsewnisqÔei mìno ìtan h arqik kateÔjunsh anaz thshc d 0 eÐnai Ðsh me thn arnhtik klÐsh.Akìmh kai an h sunˆrthsh E den eÐnai akrib¸c tetragwnik sthn perioq thc lÔshc,to je¸rhma tou Taylor upodhl¸nei ìti mporeÐ na proseggisteÐ apì mia tetragwniksunˆrthsh, dedomènou ìti eÐnai omal . Epomènwc, en¸ den anamènetai plèon termatismìcmèsa se n epanal yeic apì thn epanekkÐnhsh, den prokaleÐ èkplhxh ìti gÐnetaishmantik prìodoc gia thn eÔresh thc lÔshc, ìpwc faÐnetai apì thn sqèsh (4.18).Wstìso h teqnik thc epanekkÐnhshc lambˆnontac upìyin ton arijmì twn epanalyewn endèqetai na mhn eÐnai epark c sthn prˆxh, afoÔ oi mh grammikoÐ algìrijmoisuzug¸n klÐsewn qrhsimopoioÔntai kurÐwc gia thn epÐlush problhmˆtwn megˆlhc diˆstashcn. Se tètoiou eÐdouc probl mata mporeÐ o algìrijmoc na mhn epanekkinhjeÐkajìlou afoÔ sun jwc brÐsketai mia proseggistik lÔsh se ligìterec apì n epanalyeic. 'Etsi oi mh grammikèc mèjodoi suzug¸n klÐsewn ulopoioÔntai me strathgikècepanekkÐnhshc pou qrhsimopoioÔn ˆllouc parˆgontec.H pio eurèwc qrhsimopoioÔmenh diadikasÐa epanekkÐnhshc protˆjhke apì ton Powell[66]. H diadikasÐa epanekkÐnhshc tou Powell elègqei eˆn upˆrqei polÔ mikr

80 Neoi Algorijmoi Ekpaideushc TNDorjogwniìthta metaxÔ thc trèqousac kai thc prohgoÔmenhc klÐshc. Eidikìtera, an hsunj kh|g T k g k−1 | ≥ 0.2||g k || 2ikanopoieÐtai, tìte h kateÔjunsh anaz thshc suzug¸n klÐsewn epanekkineÐtai me thnkateÔjunsh thc pio apìtomhc kajìdou −g k .O Shanno, sthn ergasÐa [76], prìteine mia diadikasÐa elègqou gwnÐac gia na prosdiorÐseipìte ja prèpei na epanekkineÐtai h mèjodoc suzug¸n klÐsewn me thn kateÔjunshthc pio apìtomhc kajìdou. O èlegqoc basÐzetai sthn eggÔhsh ìti to sunhmÐtono thcgwnÐac metaxÔ thc kateÔjunshc anaz thshc suzug¸n klÐsewn kai thc arnhtik c klÐshceÐnai entìc enìc stajeroÔ pollaplˆsiou tou sunhmitìnou metaxÔ thc kateÔjunshcanaz thshc suzug¸n klÐsewn twn FR kai thc arnhtik c klÐshc.Mia ˆllh diadikasÐa epanekkÐnhshc, pio apl apì thn prohgoÔmenh pou prosdiorÐsthkeapì ton Shanno, protˆjhke apì touc Birgin kai Martinez sthn ergasÐa [9]. Oalgìrijmoc suzug¸n klÐsewn epanekkineÐtai ìtan h gwnÐa metaxÔ thc trèqousac kateÔjunshcanaz thshc suzug¸n klÐsewn d k kai thc klÐshc g k den eÐnai arketˆ oxeÐa.Dhlad , an h sunj khd T k g k ≤ −10 −3 ||d k ||||g k || (4.19)ikanopoieÐtai, tìte o algìrijmoc epanekkineÐtai qrhsimopoi¸ntac wc kateÔjunsh anazthshc thn fasmatik kateÔjunsh anaz thshc −ϑ k g k .Pio exeligmènec diadikasÐec epanekkÐnhshc èqoun protajeÐ sth bibliografÐa ìpouh kateÔjunsh epanekkÐnhshc eÐnai diaforetik apì thn kateÔjunsh thc pio apìtomhckajìdou [7], [22], [54] kai [66]. Se autèc tic diadikasÐec epanekkÐnhshc, h kateÔjunshepanekkÐnhshc d k orÐzetai qrhsimopoi¸ntac mia epanˆlhyh tri¸n ìrwn basismènwnsthn klÐsh g k , thn prohgoÔmenh kateÔjunsh suzug¸n klÐsewn d k−1 kai mia trÐth kateÔjunshpou perièqei prohgoÔmenh plhroforÐa gia thn sumperiforˆ thc sunˆrthshcsfˆlmatoc. Wstìso, ed¸ qrhsimopoieÐtai h diadikasÐa epanekkÐnhshc pou protˆjhkeapì touc Birgin kai Martinez sthn ergasÐa [9], h opoÐa eÐnai mia apl diadikasÐa, ètsi¸ste na deiqjeÐ h isqÔc twn proteinìmenwn algorÐjmwn suzug¸n klÐsewn qwrÐc naenisqÔontai me exeligmènec teqnikèc epanekkÐnhshc.4.3.2 DiadikasÐec Grammik c Anaz thshcOi algìrijmoi ekpaÐdeushc suzug¸n klÐsewn gia na èqoun thn idiìthta thc olik csÔgklishc, o rujmìc ekpaÐdeushc prèpei na kajorÐzetai apì mia monodiˆstath grammikanaz thsh katˆ m koc thc kateÔjunshc anaz thshc suzug¸n klÐsewn d k [56]. Pollècteqnikèc grammik c anaz thshc èqoun protajeÐ sthn bibliografÐa gia qr sh me toucalgìrijmouc ekpaÐdeushc suzug¸n klÐsewn.

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 81EnÐote, teqnikèc grammik c anaz thshc qwrÐc thn qr sh parag¸gwn èqoun qrhsimopoihjeÐgia ton kajorismì thc paramètrou tou rujmoÔ ekpaÐdeushc. H pio dhmofil cteqnik apì autoÔ tou eÐdouc grammik c anaz thshc eÐnai o algìrijmoc thc qrus ctom c kai h anaz thsh Fibonacci [47]. H diaforˆ se autèc tic dÔo teqnikèc entopÐzetaiston trìpo pou parˆgontai oi dokimastikèc parˆmetroi tou rujmoÔ ekpaÐdeushc.O Brent sthn ergasÐa [12] prìteine mia teqnik grammik c anaz thshc h opoÐa eÐnaisunduasmìc thc anaz thshc thc qrus c tom c kai mia tetragwnik c prosèggishc. Hanaz thsh tou Brent epiqeireÐ na sunduˆsei tic kalÔterec idiìthtec kai apì tic dÔoproseggÐseic kai eÐnai ènac algìrijmoc pou den apaiteÐ thn qr sh parag¸gwn.Wstìso, teqnikèc grammik c anaz thshc pou kˆnoun qr sh thc plhroforÐac pr¸thctˆxewc, dhlad thc klÐshc thc sunˆrthshc sfˆlmatoc, qrhsimopoioÔntai polÔsuqnˆ me touc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn. Pio sugkekrimèna, o rujmìcekpaÐdeushc pou epitugqˆnetai apì mia teqnik grammik c anaz thshc prèpei naikanopoieÐ thc sunj kec tou Wolfe pou dÐnontai apì tic parakˆtw sqèseicE(w k + α k d k ) − E(w k ) ≤ c 1 α k ∇E(w k ) T d k (4.20)∇E(w k + α k d k ) T d k ≥ c 2 ∇E(w k ) T d k (4.21)ìpou 0 < c 1 ≤ c 2 < 1. H pr¸th sunj kh, pou dÐnetai apì thn sqèsh (4.20), epitrèpeiopoiod pote shmeÐo na gÐnei apodektì an belti¸nei ikanopoihtikˆ thn sunarthsiaktim anaforikˆ me thn prohgoÔmenh sunarthsiak tim . H deÔterh sunj kh, pou dÐnetaiapì thn sqèsh (4.21) den epitrèpei ston rujmì ekpaÐdeushc na gÐnei polÔ mikrìc kaiexasfalÐzei ìti o paronomast c thc fasmatik c paramètrou (5.9) eÐnai kalˆ orismènockai pˆntote jetikìc, afoÔ upodhl¸nei ìti s T k y k > 0.PolloÐ algìrijmoi grammik c anaz thshc pou qrhsimopoioÔn thc sunj kec touWolfe, (4.20) kai (4.21), èqoun protajeÐ sthn bibliografÐa. Oi Nocedal kai Wright[57] periègrayan mia teqnik grammik c anaz thshc pou basÐzetai sthn prosèggishthc gnwst c sunˆrthshc sfˆlmatoc kai twn tim¸n thc klÐshc thc. Sto biblÐo [75]perigrˆfetai mia ubridik teqnik grammik c anaz thshc pou basÐzetai sthn diqotìmhshkai thn kubik parembol . O Qaralˆmpouc, sthn ergasÐa [16], anèptuxe mia teqnikgrammik c anaz thshc gia qr sh me touc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn.'Opwc h prohgoÔmenh mèjodoc, efarmìzei mia ubridik anaz thsh. QrhsimopoieÐ miakubik parembol mazÐ me mia morf tmhmatopoÐhshc. Oi Shanno kai Phua, sthnergasÐa [77], prìteinan mia apodotik teqnik grammik c anaz thshc gia qr sh mazÐ meton algìrijmo CONMIN. Autìc o algìrijmoc grammik c anaz thshc basÐzetai se miateqnik egklwbismoÔ pou perigrˆfetai sthn ergasÐa [78] kai mia diasfalismènh kubikparembol ìmoia me aut pou qrhsimopoieÐtai sthn ergasÐa [18].'Olec oi parapˆnw teqnikèc grammik c anaz thshc èqoun qrhsimopoihjeÐ mazÐ metouc algorÐjmouc ekpaÐdeushc suzug¸n klÐsewn apì polloÔc ereunhtèc gia thn epÐlushdiafìrwn problhmˆtwn. Wstìso den eÐnai safèc poia ap' ìlec apodÐdei kalÔteragenikˆ. Dhlad , diaforetikoÐ sunduasmoÐ twn algorÐjmwn ekpaÐdeushc suzug¸n klÐsewnme teqnikèc grammik c anaz thshc eÐnai katˆllhloi gia kˆje prìblhma. Ed¸

82 Neoi Algorijmoi Ekpaideushc TNDepilèqjhke na qrhsimopoihjeÐ h teqnik grammik c anaz thshc pou qrhsimopoieÐtaiston algìrijmo CONMIN.4.3.3 Arqikìc Rujmìc EkpaÐdeushcH epilog thc paramètrou tou arqikoÔ rujmoÔ ekpaÐdeushc gia touc algìrijmoucekpaÐdeushc suzug¸n klÐsewn eÐnai polÔ shmantik gia thn epituqhmènh kai gr gorhsÔgklis touc. Mia dhmofil c strathgik eÐnai na upojèsoume ìti h pr¸thc tˆxewcallag sthn sunˆrthsh sto shmeÐo w k ja eÐnai h Ðdia me tou prohgoÔmenou b matoc[57]. Me ˆlla lìgia, h arqik ektÐmhsh tou rujmoÔ ekpaÐdeushc α k epilègetai naeÐnai tètoia ¸ste α k g T k d k = α k−1 g T k−1 d k−1. Dhlad h arqik parˆmetroc tou rujmoÔekpaÐdeushc dÐnetai apì to tÔpoα k = α k−1g T k−1 d k−1g T k d kMia ˆllh epilog gia ton arqikì rujmì ekpaÐdeushc protˆjhke sto biblÐo [25]. EˆneÐnai diajèsimh mia ektÐmhsh ∆E = E(w k−1 ) − E(w k ), tìte eÐnai dunatì na parembˆlloumemia tetragwnik sunˆrthsh sto g T k d k kai ∆E, h opoÐa dÐnei thn akìloujhektÐmhshα k = −2∆Eg T k d kSthn pr¸th epanˆlhyh tou algorÐjmou ekpaÐdeushc, to ∆E prèpei na oristeÐ apì tonqr sth. Sth sunèqeia, h epilog tou ∆E = max(E(w k−1 ) − E(w k ), ε), ìpou ε eÐnaiènac mikrìc jetikìc arijmìc, leitourgeÐ arketˆ kalˆ, pou shmaÐnei ìti h meÐwsh apìthn prohgoÔmenh epanˆlhyh diasfalÐzetai.Mia dhmofil c epilog tou arqikoÔ rujmoÔ ekpaÐdeushc pou apodedeigmèna sunergˆzetaiepituq¸c me thc mejìdouc suzug¸n klÐsewn èqei protajeÐ apì ton Shannoston algìrijmo CONMIN [77]. O arqikìc rujmìc ekpaÐdeushc tou Shanno dÐnetaiapì ton tÔpoα k ={1||g 0 || 2, an k = 0;α k−1 ||d k ||||d k−1,||diaforetikˆ.(4.22)Ed¸ qrhsimopoieÐtai o arqikìc rujmìc ekpaÐdeushc tou Shanno.4.3.4 Perigraf tou AlgorÐjmou EkpaÐdeushcSe autì to shmeÐo dÐnetai mia leptomer c diatÔpwsh tou klimakwtoÔ algorÐjmou ekpaÐdeushcsuzug¸n klÐsewn. Shmei¸netai ìti gia ϑ k = 1, o algìrijmoc anˆgetai ston

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 83klasikì algìrijmo ekpaÐdeushc suzug¸n klÐsewn. O algìrijmoc ekpaÐdeushc dèqetaiwc eÐsodo to diˆnusma twn arqik¸n bar¸n w 0 , to krit rio termatismoÔ ϵ, ton mègistoarijmì epanal yewn µ, tic paramètrouc c 1 kai c 2 pou qreiˆzontai gia ton algìrijmogrammik c anaz thshc kai epistrèfei to shmeÐo elaqÐstou w ⋆ kai to elˆqisto thcsunˆrthshc sfˆlmatoc E(w ⋆ ).Algìrijmoc 5 O klimakwtìc algìrijmoc ekpaÐdeushc suzug¸n klÐsewn.SCG(w 0 , ϵ, µ, c 1 , c 2 , w ⋆ , E(w ⋆ ))1: Gia k := 0 èwc µ Kˆne2: E k = E(w k ), g k = ∇E(w k ).3: An E k ≤ ϵ Tìte4: Epèstreye w ⋆ = w k , E(w ⋆ ) = E(w k ).5: Tèloc6: An k := 0 Tìte7: d k = −g k .8: α k = 1/||g k ||.9: Alli¸c10: ϑ k = sT k s ks T k y k .11: Upolìgise to β k epilègontac ènan apì touc tÔpouc (4.13), (4.14), (4.15) kai(4.16).12: d = −ϑ k g k + β k d k−1 .13: An d T g k

84 Neoi Algorijmoi Ekpaideushc TNDlogÐzetai h parˆmetroc klimˆkwshc apì ton tÔpo (5.9) sto b ma 10. Sto b ma 11upologÐzetai h parˆmetroc β k epilègontac ènan apì touc tÔpouc (4.13), (4.14), (4.15)(4.16). Sth sunèqeia, sto b ma 12, upologÐzetai h genik kateÔjunsh anaz thshcsuzug¸n klÐsewn qrhsimopoi¸ntac ton tÔpo (4.10). Sto b ma 13, elègqetai to kritrio epanekkÐnhshc gia thn kateÔjunsh anaz thshc d k . An h sunj kh ikanopoieÐtai,tìte h genik kateÔjunsh anaz thshc suzug¸n klÐsewn gÐnetai apodekt , diaforetikˆ,h kateÔjunsh anaz thshc epanekkineÐtai qrhsimopoi¸ntac thn fasmatik kateÔjunshanaz thshc d k = −ϑ k g k . H arqik parˆmetroc tou rujmoÔ ekpaÐdeushc upologÐzetaisto b ma 18. Sto b ma 20, efarmìzetai mia teqnik grammik c anaz thshc giaton rujmì ekpaÐdeushc α k . Sto b ma 21, upologÐzetai to nèo diˆnusma twn bar¸nkai o algìrijmoc epanalambˆnetai mèqri na ikanopoihjeÐ to krit rio termatismoÔ naepiteuqjeÐ o mègistoc arijmìc epanal yewn.4.3.5 Olik SÔgklishSe aut thn parˆgrafo apodeiknÔetai h olik sÔgklish tou AlgorÐjmou 6. Gia tonskopì autì ja qrhsimopoihjeÐ to Je¸rhma tou Zoutendijk.Je¸rhma 4.1. JewroÔme to epanalhptikì sq ma thc morf c (4.11), ìpou d k eÐnaimia kajodik kateÔjunsh kai to α k ikanopoieÐ tic sunj kec tou Wolfe (4.20) kai (4.21).Upojètoume ìti h sunˆrthsh sfˆlmatoc E eÐnai kˆtw fragmènh sto R n kai ìti h E eÐnaisuneq¸c diaforÐsimh se mia geitoniˆ N tou sunìlou L = {w ∈ R n : E(w) ≤ E(w 0 )},ìpou w 0 eÐnai to arqikì shmeÐo tou epanalhptikoÔ sq matoc. Upojètoume epÐshc ìtih klÐsh ∇E eÐnai Lipschitz suneq c sto N , dhlad , ìti upˆrqei mia stajerˆ L > 0tètoia ¸ste∥∇E(w) − ∇E( ˜w)∥ ≤ L∥w − ˜w∥, gia ìla ta w, ˜w ∈ N . (4.23)Tìte èqoumeìpou∑cos 2 θ k ∥∇E k ∥ 2 < ∞ (4.24)k≥0cos θ k =−∇ET k d k∥∇E k ∥∥d k ∥(4.25)Apìdeixh. Apì tic sqèseic (4.21) kai (4.11) èqoume ìtien¸ h sunj kh Lipschitz upodhl¸nei ìti(∇E k+1 − ∇E k ) T d k ≥ (c2 − 1)∇E T k d k(∇E k+1 − ∇E k ) T d k ≤ α k L∥d k ∥ 2

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 85sunduˆzontac tic dÔo autèc sqèseic èqoume ìtiα k ≥ c 2 − 1 ∇Ek T d kL ∥d k ∥ 2Antikajist¸ntac aut thn anisìthta sthn pr¸th sunj kh tou Wolfe (4.20), èqoumeìti1 − c 2 (∇Ek T E k+1 ≤ E k − c d k) 21 .L ∥d k ∥ 2Qrhsimopoi¸ntac thn sqèsh (4.25), mporoÔme na epanadiatup¸soume thn prohgoÔmenhsqèsh wcE k+1 ≤ E k − c cos 2 θ k ∥∇E k ∥ 2 .ìpou c = c 1 (1 − c 2 )/L. AjroÐzontac thn parapˆnw èkfrash gia ìlouc touc deÐktecpou eÐnai mikrìteroi Ðsh tou k, èqoume ìtiE k+1 ≤ E 0 − ck∑cos 2 θ j ∥∇E j ∥ 2 . (4.26)j=0Efìson h E eÐnai kˆtw fragmènh, èqoume ìti h diaforˆ E k+1 − E 0 eÐnai mikrìterh apìmia jetik stajerˆ gia ìla ta k. 'Etsi paÐrnontac to ìrio gia thn sqèsh (4.26) èqoumeìti∑cos 2 θ k ∥∇E k ∥ 2 < ∞k≥0to opoÐo oloklhr¸nei thn apìdeixh.Parat rhsh 4.1. To parapˆnw je¸rhma mporeÐ na efarmosteÐ ston Algìrijmo 6afoÔ h sunˆrthsh sfˆlmatoc E ikanopoieÐ tic apait seic tou jewr matoc. Pio sugkekrimèna,h sunˆrthsh sfˆlmatoc E eÐnai kˆtw fragmènh (E(w) > 0) wc ˆjroismatetrag¸nwn. Epiplèon, gia ta teqnhtˆ neurwnikˆ dÐktua pou qrhsimopoioÔn suneqeÐcsunart seic energopoÐhshc me suneqeÐc parag¸gouc (uperbolik efaptomènh, logistiksunˆrthsh), ìpwc sthn perÐptwsh pou meletˆme, h sunˆrthsh sfˆlmatoc E eÐnaiepÐshc suneq¸c diaforÐsimh.Qrhsimopoi¸ntac to apotèlesma tou Jewr matoc tou Zoutendijk mporeÐ na apodeÐqjeÐthn olik sÔgklish gia ìlouc touc algorÐjmouc oi opoÐoi epanekkinoÔntai periodikˆjètontac thn parˆmetro β k = 0. An k 1 , k 2 , . . . upodhl¸noun tic epanal yeicstic opoÐec lambˆnoun q¸ra oi epanekkin seic, tìte apì thn sunj kh tou Zoutendijk(4.24) èqoume ìti∑∥∇E k ∥ 2 < ∞ (4.27)k=k 1 ,k 2 ,...Eˆn den epitrapoÔn perissìterec apì ¯n epanal yeic metaxÔ dÔo diadoqik¸n epanekkinsewn, h akoloujÐa {k j } ∞ j=1 ja eÐnai mh peperasmènh, kai apì thn sqèsh (4.27)

86 Neoi Algorijmoi Ekpaideushc TNDlambˆnetai ìti to lim j→∞ ∥∇E kj ∥ = 0. Dhlad , mia upakoloujÐa twn klÐsewn thcsunˆrthshc sfˆlmatoc E sugklÐnei sto 0, isodÔnama,lim infk→∞ ∥∇E k∥ = 0 (4.28)To apotèlesma autì isqÔei gia ìlouc touc algorÐjmouc suzug¸n klÐsewn pou enswmat¸nounteqnikèc epanekkÐnhshc kai epomènwc gia ton Algìrijmo 6.4.4 Peiramatikˆ ApotelèsmataSe aut thn parˆgrafo, axiologeÐtai h apìdosh twn proteinìmenwn algorÐjmwn ekpaÐdeushckai sugkrÐnontai me touc klasikoÔc algìrijmouc ekpaÐdeushc suzug¸n klÐsewntwn Hestenes-Stiefel (HS) [33], Fletcher-Reeves (FR) [26] kai Polak–Ribière (PR) [65].EpÐshc, sumperilambˆnontai sta peirˆmata oi algìrijmoi ekpaÐdeushc OSS [6], BFGSperiorismènhc mn mhc me m = 1 (LN) [2, 46], GB1 kai GB2 [21]. 'Oloi oi proanaferjèntecalgìrijmoi ekpaÐdeushc apaitoÔn mia strathgik grammik c anaz thshc gia naèqoun thn idiìthta thc olik c sÔgklishc. Gia na eÐnai dÐkaih h sÔgkrish, oi algìrijmoiekpaÐdeushc OSS, LN, GB1 kai GB2 efodiˆsthkan me thn Ðdia strathgik grammikanaz thshc kai krit rio epanekkÐnhshc pou qrhsimopoi jhke stouc algìrijmouc ekpaÐdeushcsuzug¸n klÐsewn (klasik¸n kai klimakwt¸n). Oi algìrijmoi ulopoi jhkansto upologistikì prìgramma Matlab qrhsimopoi¸ntac to pareqìmeno pakèto gia taneurwnikˆ dÐktua. Gia touc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn (klasikoÔckai klimakwtoÔc) kai gia thc yeudì-Newton mejìdouc, oi timèc twn paramètrwn c 1kai c 2 èqoun kajoristeÐ stic 10 −4 kai 0.5, antÐstoiqa, gia ìla ta peirˆmata ètsi ¸-ste na epalhjeujeÐ ìti oi proteinìmenoi algìrijmoi apodÐdoun arketˆ kalˆ se diˆforaprobl mata qwrÐc na qreiˆzetai rÔjmish aut¸n twn paramètrwn. Epiplèon, gia thnarqikopoÐhsh twn bar¸n kai twn merolhyi¸n efarmìsthke h teqnik pou protˆjhkeapì touc Nguyen kai Widrow [55] gia ìlouc touc algorÐjmouc (ektìc ki an anaferjeÐdiaforetikˆ gia kˆpoio prìblhma).H epituqÐa kai h taqÔthta sÔgklishc twn proteinìmenwn algorÐjmwn ekpaÐdeushcaxiologeÐtai anaforikˆ me touc proanaferjèntec algìrijmouc ekpaÐdeushc qrhsimopoi¸ntacta akìlouja polÔ gnwstˆ probl mata:1. To prìblhma tou ApokleistikoÔ-EITE.2. To prìblhma thc isotimÐac twn 3-bit.3. To prìblhma tou N −M −N kwdikopoiht /apokwdikopoiht (ìpou M = log 2 N,N = 4, 8, 16, 32, 64, 128, 256).4. To prìblhma thc anagn¸rishc twn kefalaÐwn grammˆtwn thc Agglik c alfab -tou.

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 875. To prìblhma thc anagn¸rishc arijm¸n.Epiplèon axiologeÐtai h apìdosh twn algorÐjmwn ekpaÐdeushc qrhsimopoi¸ntac merikˆpragmatikˆ probl mata apì thn apoj kh dedomènwn mhqanik c mˆjhshc tou UCI[53]. Ta probl mata autˆ eÐnai h taxinìmhsh twn fut¸n thc oikogèneiac Iris, h optikanagn¸rish qeirìgrafwn arijm¸n kai h anagn¸rish fwnhèntwn. Gia ta probl mataautˆ axiologeÐtai kai h ikanìthta genÐkeushc twn algorÐjmwn ekpaÐdeushc.Gia thn anˆlush thc statistik c shmasÐac twn apotelesmˆtwn, qrhsimopoi jhke hmèjodoc elègqou Wilcoxon paired two sided sign rank test [89], [27]. Aut eÐnai mia mhparametrik mèjodoc pou jewreÐtai enallaktik lÔsh gia th mèjodo elègqou pairedt-test. O èlegqoc autìc upojètei ìti upˆrqei plhroforÐa sto mègejoc thc diaforˆcmetaxÔ antistoiqismènwn parathr sewn, kaj¸c epÐshc kai sta prìshma. H mhdenikupìjesh tou elègqou eÐnai ìti h diaforˆ metaxÔ twn antistoiqismènwn parathr sewnproèrqetai apì mia katanom thc opoÐac o diˆmesoc eÐnai mhdèn. Oi parathr seic stapeirˆmatˆ mac eÐnai h taqÔthta sÔgklishc (arijmìc upologism¸n sunarthsiak¸n tim¸nkai upologism¸n klÐsewn) kai o èlegqoc pragmatopoi jhke me epÐpedo shmantikìthtac5%. Dhlad an h tim p-value eÐnai mikrìterh apì 0.05 tìte h mhdenik upìjesh mporeÐna aporrifjeÐ. Oi statistik¸c shmantikèc peript¸seic shmei¸nontai me to sÔmbolo(+) en¸ oi statistik¸c as mantec peript¸seic me to sÔmbolo (−). Sta peirˆmatˆ macsugkrÐjhkan oi parathr seic tou algorÐjmou ekpaÐdeushc SPerry ènanti ekeÐnwn twnalgorÐjmwn pou eÐqan qeirìterh apìdosh apì autìn. O lìgoc gia ton opoÐo epilèqjhkeo algìrijmoc ekpaÐdeushc SPerry eÐnai ìti o algìrijmoc autìc tan o apodotikìterocsta perissìtera probl mata pou qrhsimopoi jhkan.Gia kˆje prìblhma, paratÐjetai ènac pÐnaka o opoÐoc sunoyÐzei thn apìdosh twnalgorÐjmwn gia prosomoi¸seic pou èftasan se lÔsh entìc enìc prokajorismènou orÐouupologism¸n thc sunˆrthshc sfˆlmatoc pou orÐzetai se kˆje prìblhma. Oi anaferìmenecstouc pÐnakec parˆmetroi eÐnai: Min o elˆqistoc arijmìc epanal yewn, Max omègistoc arijmìc epanal yewn, µ h mèsh tim twn epanal yewn, σ h tupik apìklish,kai EpituqÐa o arijmìc twn petuqhmènwn prosomoi¸sewn apì èna sÔnolo 1000 dokim¸n(ektìc ki an anaferjeÐ diaforetikˆ gia kˆpoio prìblhma). Se perÐptwsh pou ènac algìrijmocapotÔqei na sugklÐnei mèsa sto prokajorismèno ìrio twn upologism¸n thcsunˆrthshc sfˆlmatoc, jewreÐtai ìti apètuqe na ekpaideÔsei to teqnhtì neurwnikìdÐktuo kai oi epanal yeic thc sugkekrimènhc prosomoÐwshc den perilambˆnontai sthnstatistik anˆlush tou algorÐjmou. Gia ta probl mata pou qrhsimopoi jhkan giathn axiolìghsh thc genÐkeushc twn algorÐjmwn ekpaÐdeushc, prostÐjetai mia epiplèonparˆmetroc stouc pÐnakec, pou onomˆzetai GenÐkeush, h opoÐa parousiˆzei to posostìthc mèshc genÐkeushc gia petuqhmènec prosomoi¸seic twn algorÐjmwn ekpaÐdeushc.Se autì to shmeÐo, prèpei na epishmanjeÐ ìti genikˆ mia epanˆlhyh enìc algorÐjmouekpaÐdeushc shmaÐnei ìti ìla ta prìtupa ekpaÐdeushc èqoun parousiasteÐ stoteqnhtì neurwnikì dÐktuo. Stouc klasikoÔc algorÐjmouc, ìpwc h BP, kˆje epanˆlhyhantistoiqeÐ se èna sunarthsiakì upologismì thc sunˆrthshc sfˆlmatoc kai ènan

88 Neoi Algorijmoi Ekpaideushc TNDupologismì thc klÐshc (parˆgwgoi pr¸thc tˆxewc). Stouc algorÐjmouc ekpaÐdeushcsuzug¸n klÐsewn kai stic yeudì-Newton mejìdouc lambˆnoun q¸ra perissìteroisunarthsiakoÐ upologismoÐ kai upologismoÐ klÐsewn anˆ epanˆlhyh, gegonìc pou o-feÐletai sthn parousÐa thc teqnik c grammik c anaz thshc. ExaitÐac thc fÔsewc thcgrammik c anazht sewc pou qrhsimopoieÐtai stouc algorÐjmouc autoÔc, apaiteÐtai oÐdioc arijmìc sunarthsiak¸n upologism¸n kai upologism¸n klÐsewn anˆ epanˆlhyh.Dhlad metˆ thn ektèlesh thc diadikasÐac thc grammik c anaz thshc se mia epanˆlhyhtou algorÐjmou ekpaÐdeushc o Ðdioc arijmìc sunarthsiak¸n upologism¸n kaiupologism¸n klÐsewn lambˆnei q¸ra (gia parˆdeigma, sthn k epanˆlhyh tou algorÐjmouekpaÐdeushc h diadikasÐa thc grammik c anaz thshc qreiˆsthke r sunarthsiakoÔcupologismoÔc kai upologismoÔc klÐsewn).Sthn statistik anˆlush twn algorÐjmwn ekpaÐdeushc, ektìc apì touc sunarthsiakoÔcupologismoÔc kai touc upologismoÔc klÐsewn, upologÐsthke kai o qrìnoc thckentrik c monˆdac epexergasÐac (CPU) se deuterìlepta (sec) pou qreiˆsthke kˆje algìrijmocekpaÐdeushc gia kˆje èna apì ta probl mata. Sthn sunèqeia, apeikonÐzontaioi mèsoi ìroi twn qrìnwn thc CPU se rabdogrˆmmata gia kˆje algìrijmo ekpaÐdeushcse kˆje prìblhma. Epiplèon, dhmiourg jhkan graf mata ta opoÐa apeikonÐzoun tonmèso ìro twn kampul¸n ekpaÐdeushc gia touc algorÐjmouc ekpaÐdeushc gia kˆpoia apìta probl mata. Ston ˆxona x apeikonÐzetai o mèsoc ìroc tou ajroÐsmatoc twn upologism¸nthc sunˆrthshc sfˆlmatoc kai thc klÐshc thc, en¸ ston ˆxona y apeikonÐzetaih mèsh tim thc sunˆrthshc sfˆlmatoc. O ˆxonac y eÐnai se logarijmik klÐmaka ètsi¸ste oi diaforèc metaxÔ twn algorÐjmwn ekpaÐdeushc na gÐnontai eÔkola antilhptèc.Se autˆ ta graf mata l fjhkan upìyin mìno oi prosomoi¸seic pou èftasan sthn lÔshapì èna sÔnolo 100 dokim¸n.4.4.1 Apokleistikì-EITEGia to prìblhma tou apokleistikoÔ-EITE (blèpe Parˆrthma A') qrhsimopoi same èna2 − 2 − 1 teqnhtì neurwnikì dÐktuo (6 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 4.1 kai stic Eikìnec4.1 kai 4.2.'Oloi oi algìrijmoi ekpaÐdeushc pou dokimˆsthkan parousÐasan kalì posostì epituqÐackai kal taqÔthta sÔgklishc. O kalÔteroc algìrijmoc ìson aforˆ to posostìepituqÐac tan o SPerry. Wstìso, kai oi upìloipoi algìrijmoi ekpaÐdeushc parousÐasanantagwnistikˆ posostˆ epituqÐac, me exaÐresh touc GB1, GB2, FR, SFR kai SPRpou parousÐasan qamhlìtera posostˆ epituqÐac (toulˆqiston 2.6% ligìtero apì ton

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 890.180.160.140.12CPU Time (sec)0.10.080.060.040.020OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.1: Mèsoc qrìnoc gia to prìblhma tou apokleistikoÔ-EITEXOR Problem10 1 Passes (FE+GE)SSE10 0HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 −110 −20 50 100 150 200 250 300 350Sq ma 4.2: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma touapokleistikoÔ-EITE

90 Neoi Algorijmoi Ekpaideushc TNDalgìrijmo SPerry). 'Oson aforˆ thn taqÔthta sÔgklishc, o algìrijmoc GB1 parousÐaseton taqÔtero rujmì sÔgklishc. Oi upìloipec mèjodoi parousÐasan epÐshc kaltaqÔthta sÔgklishc, ektìc apì tic mejìdouc FR kai SFR (blèpe tic Eikìnec 1.2 kai1.3). Sunolikˆ, oi klimakwtoÐ algìrijmoi suzug¸n klÐsewn parousÐasan elafr¸c kalÔterhapìdosh tìso se posostì epituqÐac ìso kai se taqÔthta sÔgklishc se sqèshme touc klasikoÔc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn me kˆpoiec exairèseic (oalgìrijmoc PR tan kalÔteroc apì ton SPR ìson aforˆ to posostì epituqÐac, en¸o algìrijmoc tou Perry tan elafr¸c grhgorìteroc apì ton algìrijmo SPerry). OPÐnakac 4.1 parousiˆzei, epÐshc, ta apotelèsmata tou elègqou Wilcoxon sign rank testgia tic mejìdouc pou tan qeirìterec apì ton algìrijmo ekpaÐdeushc SPerry. 'OpwceÐnai profanèc, h beltÐwsh pou parousiˆzei o algìrijmoc SPerry se sqèsh me autoÔctouc algorÐjmouc eÐnai statistikˆ shmantik ìson aforˆ thn taqÔthta sÔgklishc.Algìrijmoc min max µ σ EpituqÐaOSS 11 990 102.5 (+) 143.9 92.6%GB1 8 993 81.8 158.3 90.9%GB2 8 995 82.5 156.1 87.5%LN 7 997 93.5 159.7 92.4%HS 9 996 101.8 (+) 157.4 92.6%SHS 9 975 101.4 (+) 153.7 92.9%FR 8 961 163.8 (+) 198.2 86.8%SFR 8 945 146.4 (+) 177.6 87.7%PR 8 991 93.4 142.1 92.0%SPR 8 946 87.3 131.1 90.0%Perry 7 981 95.2 146.9 92.9%SPerry 9 993 97.8 153.9 93.5%PÐnakac 4.1: Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITE4.4.2 IsotimÐa twn 3-bitGia to prìblhma thc isotimÐac twn 3-bit (blèpe Parˆrthma A') qrhsimopoi same èna3 − 2 − 1 teqnhtì neurwnikì dÐktuo (8 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 4.2 kai stic Eikìnec4.3 kai 4.4.Se autì to prìblhma. o algìrijmoc ekpaÐdeushc SPerry parousÐase to kalÔteroposostì epituqÐac apì ìlouc touc upìloipouc algìrijmouc. EpÐshc, tan o kalÔteroc

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 910.350.30.25CPU Time (sec)0.20.150.10.050OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.3: Mèsoc qrìnoc gia to prìblhma thc isotimÐac twn 3-bit3−bit Parity Problem10 1 Passes (FE+GE)SSE10 010 −1HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 −20 200 400 600 800 1000 1200Sq ma 4.4: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma thcisotimÐac twn 3-bit

92 Neoi Algorijmoi Ekpaideushc TNDalgìrijmoc ìson aforˆ thn taqÔthta sÔgklishc se sqèsh me touc algìrijmouc ekpaÐdeushcsuzug¸n klÐsewn (klasik¸n kai klimakwt¸n). Oi algìrijmoi ekpaÐdeushcGB1, GB2 kai LN parousÐasan taqÔterh sÔgklish apì ton algìrijmo SPerry, allˆ toposostì epituqÐac tou tan arketˆ qamhlìtero. O èlegqoc Wilcoxon sign rank testèdeixe ìti ta apotelèsmata tou algorÐjmou ekpaÐdeushc SPerry eÐnai statistik¸c shmantikˆse sqèsh me touc ˆllouc algìrijmouc pou apèdwsan qeirìtera. Oi algìrijmoiekpaÐdeushc FR kai SFR eÐqan polÔ kak apìdosh tìso se posostì epituqÐac ìso kaise taqÔthta sÔgklishc. Wstìso, o algìrijmoc ekpaÐdeushc SFR tan beltiwmènocse sqèsh me ton klasikì algìrijmo FR. H mèjodoc PR parousÐase to deÔtero kalÔteroposostì epituqÐac se autì to prìblhma. O antÐstoiqoc klimakwtìc algìrijmocekpaÐdeushc, SPR eÐqe shmantikˆ qamhlìtero posostì epituqÐac, en¸ tan elˆqistataqÔteroc. O algìrijmoc ekpaÐdeushc SHS apèdwse kalÔtera se posostì epituqÐac,allˆ sunèkline bradÔtera apì ton algìrijmo HS.Algìrijmoc min max µ σ EpituqÐaOSS 111 990 347.6 (+) 193.3 65.5%GB1 42 987 191.1 174.4 71.2%GB2 39 975 189.9 178.6 65.9%LN 49 991 228.3 174.7 76.5%HS 67 998 247.9 (+) 167.9 74.6%SHS 67 995 253.7 (+) 178.1 74.9%FR 97 992 468.9 (+) 248.2 23.9%SFR 93 997 445.2 (+) 254.4 28.5%PR 59 998 269.8 (+) 174.9 77.3%SPR 59 993 267.1 (+) 163.6 71.5%Perry 64 995 233.1 (+) 162.7 73.0%SPerry 69 996 232.9 169.1 80.6%PÐnakac 4.2: Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bit4.4.3 N − M − N Kwdikopoiht c/Apokwdikopoiht cGia to prìblhma tou N − M − N Kwdikopoiht /Apokwdikopoiht (blèpe ParˆrthmaA') qrhsimopoi same mia seirˆ apì eptˆ teqnhtˆ neurwnikˆ dÐktua. Ousiastikˆ,pragmatopoi same dokimèc gia N = 4, 8, 16, 32, 64, 128 kai 256. Oi neur¸nec tou krufoÔepipèdou, allˆ kai tou epipèdou exìdou eÐqan wc sunart seic energopoÐhshc thnlogistik sunˆrthsh. Ston PÐnaka 4.3, dÐnontai oi prodiagrafèc twn eptˆ problhmˆtwnKwdikopoÐhshc/ApokwdikopoÐhshc. H arqikopoÐhsh twn bar¸n ègine me thnteqnik twn Nguyen kai Widrow, allˆ sth sunèqeia ta bˆrh metaxÔ tou krufoÔ epipèdoukai tou epipèdou thc exìdou pollaplasiˆsthkan me 0.01. Ta apotelèsmata twnexomoi¸sewn parousiˆzontai stouc PÐnakec 4.4-4.10. EpÐshc, h mèsh tim thc taqÔthtacekpaÐdeushc gia ìlouc touc algorÐjmouc ekpaÐdeushc apeikonÐzetai stic Eikìnec

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 934.5-4.13.Prìblhma Dokimèc 'Orio Ep. Bˆrh MerolhyÐec e.g.4-2-4 1000 1000 16 6 0.18-3-8 1000 1000 48 11 0.116-4-16 1000 1000 128 20 0.132-5-32 1000 1000 320 37 0.164-6-64 1000 2000 768 70 0.1128-7-128 100 10000 1792 135 0.1256-8-256 10 50000 4096 264 0.1PÐnakac 4.3: Prodiagrafèc twn problhmˆtwn N − M − N Kwdikopoiht/Apokwdikopoiht . Oi anaferìmenec parˆmetroi eÐnai: Dokimèc, o arijmìc twn e-xomoi¸sewn, 'Orio Ep., o mègistoc epitreptìc arijmìc sunarthsiak¸n upologism¸n,Bˆrh, o arijmìc twn sunaptik¸n bar¸n, MerolhyÐec, o arijmìc twn merolhyi¸n, kaie, g, h sunj kh termatismoÔ.O algìrijmoc ekpaÐdeushc SPerry tan o kalÔteroc ìson aforˆ to posostì epituqÐacse olìklhrh thn oikogèneia twn problhmˆtwn KwdikopoÐhth/Apokwdikopoiht ,me exaÐresh to prìblhma 8-3-8 ìpou o algìrijmoc ekpaÐdeushc LN eÐqe elˆqista kalÔteroposostì epituqÐac. AxÐzei na shmeiwjeÐ ìti kaj¸c h diˆstash tou probl matocmegal¸nei, o algìrijmoc ekpaÐdeushc SPerry diathreÐ ta uyhlˆ posostˆ epituqÐac,en¸ to posostì epituqÐac twn upoloÐpwn algorÐjmwn ekpaÐdeushc mei¸netai drastikˆ.O algìrijmoc SPerry parousiˆzei polÔ kal taqÔthta sÔgklishc. Eidikìtera sticuyhlèc diastˆseic tou probl matoc tou KwdikopoÐhth/Apokwdikopoiht h taqÔthtasÔgklishc tou algorÐjmou tan makrˆn h taqÔterh. Se merikèc peript¸seic parousÐaseqamhlìterh taqÔthta sÔgklishc se sqèsh me ton algìrijmo ekpaÐdeushc LN (giata probl mata 4-2-4 kai 16-4-16) allˆ h diaforˆ tan polÔ mikr en¸ sunèkline seperissìterec prosomoi¸seic. Oi klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewneÐqan kalÔterh apìdosh tìso se posostì epituqÐac ìso kai se taqÔthta sÔgklishcstic perissìterec peript¸seic se sqèsh me touc antÐstoiqouc klasikoÔc algorÐjmoucekpaÐdeushc suzug¸n klÐsewn. EpÐshc, axÐzei na shmeiwjeÐ ìti sta probl mata KwdikopoÐhth/Apokwdikopoiht128-7-128 kai 256-8-256 oi algìrijmoi ekpaÐdeushc OSS,GB1, GB2, FR kai SFR apètuqan na sugklÐnoun se ìlec tic exomoi¸seic pou epiqeir -jhkan. EÐnai pijanì ìti mia aÔxhsh tou mègistou arijmoÔ epitrepìmenwn sunarthsiak¸nupologism¸n ja eÐqe wc apotèlesma na sugklÐnoun se kˆpoiec exomoi¸seic. Wstìso,to upologistikì touc kìstoc ja tan shmantikˆ uyhlìtero apì touc upìloipouc algorÐjmoucekpaÐdeushc kai eidikìtera ton SPerry. Stouc PÐnakec 4.4-4.10 parousiˆzontaita apotelèsmata tou elègqou Wilcoxon sign rank test. Ta apotelèsmata tou algorÐjmouekpaÐdeushc SPerry ìson aforˆ touc upologismoÔc thc sunˆrthshc sfˆlmatockai thc klÐsewc thc tan statistikˆ shmantikˆ se sqèsh me touc algorÐjmouc pouapèdwsan qeirìtera. Mìno sthn perÐptwsh tou probl matoc 8-3-8 ta apotelèsmatatou algorÐjmou ekpaÐdeushc SPerry den tan statistik¸c shmantikˆ sugkrinìmena meta apotelèsmata tou algorÐjmou ekpaÐdeushc LN.

94 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc min max µ σ EpituqÐaOSS 16 965 117.9 (+) 107.1 92.0%GB1 16 999 98.5 (+) 95.9 94.6%GB2 16 991 111.6 (+) 109.1 93.1%LN 10 659 51.7 53.1 99.7%HS 11 657 58.2 (+) 52.8 99.8%SHS 11 659 58.1 (+) 52.9 99.8%FR 12 995 129.7 (+) 176.1 95.4%SFR 12 864 78.7 (+) 101.5 99.3%PR 13 379 63.9 (+) 49.9 100.0%SPR 13 674 59.7 (+) 47.5 100.0%Perry 13 965 99.8 (+) 98.9 97.7%SPerry 12 656 51.9 45.1 100.0%PÐnakac 4.4: Sugkritikˆ apotelèsmata gia to prìblhma tou 4 − 2 − 4 Kwdikopoiht/Apokwdikopoiht0.10.090.080.07CPU Time (sec)0.060.050.040.030.020.010OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.5: Mèsoc qrìnoc gia to prìblhma tou 4 − 2 − 4 Kwdikopoiht / Apokwdikopoiht

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 954−2−4 Encoder Problem10 1 Passes (FE+GE)SSE10 0HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 −10 50 100 150 200 250 300Sq ma 4.6: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma tou4 − 2 − 4 Kwdikopoiht / ApokwdikopoihtAlgìrijmoc min max µ σ EpituqÐaOSS 57 998 245.9 (+) 164.9 80.8%GB1 59 975 220.9 (+) 162.3 86.8%GB2 65 996 236.1 (+) 169.7 84.1%LN 27 973 130.6 (−) 127.5 99.7%HS 31 999 161.7 (+) 164.1 98.9%SHS 31 974 158.4 (+) 155.7 98.9%FR 24 995 226.2 (+) 205.9 77.1%SFR 24 976 179.7 (+) 169.8 96.1%PR 40 950 175.2 (+) 132.5 97.0%SPR 40 944 161.7 (+) 117.6 97.1%Perry 39 981 186.4 (+) 123.4 98.4%SPerry 29 977 130.4 128.2 99.2%PÐnakac 4.5: Sugkritikˆ apotelèsmata gia to prìblhma tou 8 − 3 − 8 Kwdikopoiht/Apokwdikopoiht

96 Neoi Algorijmoi Ekpaideushc TND0.160.140.12CPU Time (sec)0.10.080.060.040.020OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.7: Mèsoc qrìnoc gia to prìblhma tou 8 − 3 − 8 Kwdikopoiht / Apokwdikopoiht8−3−8 Encoder Problem10 2 Passes (FE+GE)SSE10 1HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 010 −10 50 100 150 200 250 300 350 400 450 500Sq ma 4.8: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma tou8 − 3 − 8 Kwdikopoiht / Apokwdikopoiht

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 97Algìrijmoc min max µ σ EpituqÐaOSS 114 997 383.1 (+) 170.1 57.7%GB1 130 999 363.3 (+) 177.1 65.4%GB2 117 995 389.4 (+) 187.6 59.6%LN 83 931 271.2 168.8 96.2%HS 89 998 412.9 (+) 244.1 89.6%SHS 89 999 403.6 (+) 237.1 88.7%FR 89 990 424.1 (+) 254.9 25.0%SFR 82 994 394.5 (+) 245.8 59.1%PR 128 997 359.1 (+) 183.9 84.6%SPR 132 998 340.5 (+) 169.6 85.6%Perry 91 997 313.8 (+) 163.3 90.5%SPerry 73 996 273.9 193.8 98.6%PÐnakac 4.6: Sugkritikˆ apotelèsmata gia to prìblhma tou 16 − 4 − 16 Kwdikopoiht/Apokwdikopoiht0.350.30.25CPU Time (sec)0.20.150.10.050OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.9: Mèsoc qrìnoc gia to prìblhma tou 16 − 4 − 16 Kwdikopoiht / Apokwdikopoiht

98 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc min max µ σ EpituqÐaOSS 179 988 504.9 (+) 201.4 11.6%GB1 197 999 524.7 (+) 192.8 10.5%GB2 207 997 516.7 (+) 204.8 11.6%LN 144 999 509.2 (+) 198.6 72.7%HS 256 998 595.3 (+) 209.5 63.8%SHS 225 999 591.7 (+) 206.8 62.9%FR 220 901 528.5 (+) 234.2 1.5%SFR 168 992 595.9 (+) 247.5 8.3%PR 319 999 621.2 (+) 172.1 56.0%SPR 327 995 617.6 (+) 164.9 59.3%Perry 170 997 503.5 (+) 190.9 54.3%SPerry 227 996 498.6 201.7 85.9%PÐnakac 4.7: Sugkritikˆ apotelèsmata gia to prìblhma tou 32 − 5 − 32 Kwdikopoiht/Apokwdikopoiht0.80.70.6CPU Time (sec)0.50.40.30.20.10OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.10: Mèsoc qrìnoc gia to prìblhma tou 32 − 5 − 32 Kwdikopoiht / Apokwdikopoiht

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 99Algìrijmoc min max µ σ EpituqÐaOSS 445 1915 1088.1 (+) 431.4 3.6%GB1 392 1977 1052.1 (+) 445.1 4.7%GB2 367 1983 1209.7 (+) 516.4 3.7%LN 344 1997 1144.3 (+) 397.1 43.9%HS 636 1999 1310.5 (+) 358.8 51.3%SHS 637 1999 1325.4 (+) 363.7 52.4%FR 1504 1504 1504 (+) 0 0.1%SFR 592 1999 1220.5 (+) 522.2 0.8%PR 820 1982 1415.5 (+) 266.2 41.9%SPR 861 1999 1476.7 (+) 287.1 32.3%Perry 346 1988 1008.2 372.9 23.8%SPerry 575 1996 1088.3 347.9 86.2%PÐnakac 4.8: Sugkritikˆ apotelèsmata gia to prìblhma tou 64 − 6 − 64 Kwdikopoiht/Apokwdikopoiht43.53CPU Time (sec)2.521.510.50OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.11: Mèsoc qrìnoc gia to prìblhma tou 64 − 6 − 64 Kwdikopoiht / Apokwdikopoiht

100 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc min max µ σ EpituqÐaOSS - - - - -GB1 - - - - -GB2 - - - - -LN 1698 9455 5619.5 (+) 2352.7 48%HS 1756 9873 4263.9 (+) 2359.1 77%SHS 1562 9544 4286.9 (+) 2251.1 80%FR - - - - -SFR - - - - -PR 2465 9872 4572.3 (+) 2068.1 59%SPR 1974 9804 6585.1 (+) 2661.2 31%Perry 1459 7760 3680.9 (+) 1935.1 30%SPerry 1211 9313 3117.1 1642.9 91%PÐnakac 4.9: Sugkritikˆ apotelèsmata gia to prìblhma tou 128 − 7 − 128 Kwdikopoiht/Apokwdikopoiht50454035CPU Time (sec)302520151050OSS LN HS SHS PR SPR Perry SPerryTraining AlgorithmsSq ma 4.12: Mèsoc qrìnoc gia to prìblhma tou 128 − 7 − 128 Kwdikopoiht / Apokwdikopoiht

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 101Algìrijmoc min max µ σ EpituqÐaOSS - - - - -GB1 - - - - -GB2 - - - - -LN 10377 46780 29146.2 (+) 13255.1 50%HS 10882 26464 17909.8 (+) 6432.4 60%SHS 6018 42443 23497 (+) 14148.1 60%FR - - - - -SFR - - - - -PR 49003 49003 49003 (+) 0 10%SPR 48826 48826 48826 (+) 0 10%Perry 9309 13094 11210.3 (+) 1892.6 30%SPerry 3975 9728 6015.9 2090.5 70%PÐnakac 4.10: Sugkritikˆ apotelèsmata gia to prìblhma tou 256 − 8 − 256 Kwdikopoiht/Apokwdikopoiht160014001200CPU Time (sec)10008006004002000LN HS SHS PR SPR Perry SPerryTraining AlgorithmsSq ma 4.13: Mèsoc qrìnoc gia to prìblhma tou 256 − 8 − 256 Kwdikopoiht / Apokwdikopoiht

102 Neoi Algorijmoi Ekpaideushc TND2.52CPU Time (sec)1.510.50OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.14: Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc kefalaÐwn grammˆtwn4.4.4 Anagn¸rish KefalaÐwn GrammˆtwnGia to prìblhma thc anagn¸rishc kefalaÐwn grammˆtwn thc Agglik c alfab tou (blèpeParˆrthma A') qrhsimopoi same èna 35 − 30 − 26 teqnhtì neurwnikì dÐktuo (1830bˆrh kai 56 merolhyÐec). To krufì epÐpedo tou neurwnikoÔ diktÔou eÐnai basismèno seneur¸nec me logistik sunˆrthsh energopoÐhshc, en¸ to epÐpedo exìdou eÐnai basismènose neur¸nec grammik c sunˆrthshc energopoÐhshc. H sunj kh termatismoÔ eÐnaiE ≤ 0.1 kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoceÐnai 2000. H arqikopoÐhsh twn bar¸n ègine me thn teqnik twn Nguyen kai Widrow,allˆ sth sunèqeia ta bˆrh metaxÔ tou krufoÔ epipèdou kai tou epipèdou thc exìdoupollaplasiˆsthkan me 0.01. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai stonPÐnaka 4.11 kai stic Eikìnec 4.14 kai 4.15.Se autì to prìblhma ìloi oi algìrijmoi apèdwsan polÔ kalˆ ìson aforˆ to posostìepituqÐac. Katˆferan na sugklÐnoun se ìlec tic exomoi¸seic pou pragmatopoijhkan, ektìc apì touc algorÐjmouc ekpaÐdeushc OSS, PR kai SPR. Wstìso, oialgìrijmoi ekpaÐdeushc pou den sunèklinan se ìlec tic exomoi¸seic parousÐasan ikanopoihtikˆposostˆ epituqÐac. Oi yeudì-Newton algìrijmoi ekpaÐdeushc qreiˆsthkanarketoÔc upologismoÔc thc sunˆrthshc sfˆlmatoc kai thc klÐsewc thc gia na sugklÐnoun,me exaÐresh ton algìrijmo ekpaÐdeushc LN o opoÐoc tan antagwnistikìc metouc algorÐjmouc suzug¸n klÐsewn. O kalÔteroc algìrijmoc apì touc klasikoÔc algorÐjmoucekpaÐdeushc suzug¸n klÐsewn tan o proteinìmenoc algìrijmoc Perry. Oi

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 103Alphabetic Font Learning Problem10 2 Passes (FE+GE)SSE10 1HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 010 −10 500 1000 1500 2000 2500 3000 3500Sq ma 4.15: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma thcanagn¸rishc kefalaÐwn grammˆtwnAlgìrijmoc min max µ σ EpituqÐaOSS 1033 1999 1517.3 (+) 241.1 90.0%GB1 888 1320 1034.7 (+) 72.7 100.0%GB2 899 1192 1020.1 (+) 65.5 100.0%LN 230 495 345.9 (+) 48.8 100.0%HS 195 533 355.7 (+) 65.9 100.0%SHS 234 561 359.5 (+) 66.9 100.0%FR 266 553 390.1 (+) 61.7 100.0%SFR 273 593 398.1 (+) 62.8 100.0%PR 293 1651 612.7 (+) 338.1 99.3%SPR 293 1952 557.3 (+) 371.9 93.5%Perry 219 534 344.3 (+) 59.2 100.0%SPerry 188 569 329.8 67.1 100.0%PÐnakac 4.11: Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwn

104 Neoi Algorijmoi Ekpaideushc TNDklimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn eÐqan thn Ðdia sumperiforˆ me taklasikˆ touc anˆloga. Oi algìrijmoi ekpaÐdeushc SHS kai SFR qreiˆsthkan elafr¸cperissìterouc sunarthsiakoÔc upologismoÔc kai upologismoÔc klÐsewn apì ta klasikˆtouc anˆloga, en¸ oi algìrijmoi ekpaÐdeushc SPerry kai SPR qreiˆsthkan ligìteroucsunarthsiakoÔc upologismoÔc kai upologismoÔc klÐsewn apì touc algìrijmoucekpaÐdeushc Perry kai PR, antÐstoiqa. O kalÔteroc algìrijmoc ekpaÐdeushc ìsonaforˆ to posostì epituqÐac kai sthn taqÔthta sÔgklishc tan o algìrijmoc SPerry.EpÐshc, ìpwc faÐnetai apì ton èlegqo Wilcoxon sign rank test, ta apotelèsmata toualgorÐjmou SPerry tan statistik¸c shmantikˆ se sqèsh me ta apotelèsmata twnalgorÐjmwn pou tan qeirìteroi apì autìn ìson forˆ thn taqÔthta sÔgklishc.4.4.5 Anagn¸rish Arijm¸nGia to prìblhma thc anagn¸rishc arijm¸n (blèpe Parˆrthma A') qrhsimopoi sameèna 64 − 6 − 10 teqnhtì neurwnikì dÐktuo (444 bˆrh kai 16 merolhyÐec). To krufìepÐpedo kai to epÐpedo exìdou tou neurwnikoÔ diktÔou eÐnai basismèna se neur¸nec melogistik sunˆrthsh energopoÐhshc. H sunj kh termatismoÔ eÐnai E ≤ 0.001 kai omègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 2000. Taarqikˆ bˆrh tou neurwnikoÔ diktÔou arqikopoi jhkan me tuqaÐouc arijmoÔc mèsa apìto kleistì diˆsthma [−1, 1]. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai stonPÐnaka 4.12 kai stic Eikìnec 4.16 kai 4.17.Se autì to prìblhma, o algìrijmoc ekpaÐdeushc SPerry parousÐase thn kalÔterhepÐdosh tìso se posostì epituqÐac ìso kai se taqÔthta sÔgklishc. Epiplèon, ta a-potelèsmata tou algorÐjmou ekpaÐdeushc SPerry ìson aforˆ thn taqÔthta sÔgklishctan statistikˆ shmantikˆ sugkrinìmena me touc upìloipouc algorÐjmouc ekpaÐdeushcektìc apì ton algìrijmo LN, ìpwc faÐnetai apì ton èlegqo Wilcoxon sign rank test.Sunèkline se ìlec tic exomoi¸seic pou pragmatopoi jhkan me ligìterouc upologismoÔctim¸n sunˆrthshc sfˆlmatoc kai upologismoÔc klÐsewn. Ki ˆlloi algìrijmoi ekpaÐdeushc,ìpwc oi HS, SHS kai SPR, katˆferan na sugklÐnoun se ìlec tic exomoi¸seic,allˆ tan shmantikˆ pio dapanhroÐ se upologismoÔc. Oi klimakwtoÐ algìrijmoi ekpaÐdeushcsuzug¸n klÐsewn parousiˆsthkan beltiwmènoi se sqèsh me ta klasikˆ toucanˆloga. Oi algìrijmoi ekpaÐdeushc SFR, SPR kai SPerry sunèklinan se perissìterecexomoi¸seic en¸ parousÐasan taqÔterh sÔgklish se sqèsh me touc algìrijmoucekpaÐdeushc FR, PR kai Perry. O algìrijmoc ekpaÐdeushc SHS sunèkline se ìlectic exomoi¸seic ìpwc kai o algìrijmoc HS, allˆ ìson aforˆ thn taqÔthta sÔgklishctan elafr¸c qeirìteroc. Oi algìrijmoi ekpaÐdeushc yeudì-Newton, me exaÐresh tonalgìrijmo LN pou tan antagwnistikìc me touc algìrijmouc ekpaÐdeushc suzug¸nklÐsewn, parousÐasan qamhlìtera posostˆ epituqÐac se autì to prìblhma kai apaitoÔsanperissìterouc sunarthsiakoÔc upologismoÔc kai upologismoÔc klÐsewn gia nasugklÐnoun.

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 10521.81.61.4CPU Time (sec)1.210.80.60.40.20OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.16: Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc arijm¸nNumeric Font Learning Problem10 2 Passes (FE+GE)SSE10 110 010 −1HSFRPRPerrySHSSFRSPRSPerryOSSGB1GB2LN10 −210 −30 200 400 600 800 1000 1200Sq ma 4.17: KampÔlec ekpaÐdeushc twn algorÐjmwn ekpaÐdeushc gia to prìblhma thcanagn¸rishc arijm¸n

106 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc min max µ σ EpituqÐaOSS 47 19422 1021.3 (+) 2455.8 81.5%GB1 45 19432 1080.4 (+) 2303.5 84.6%GB2 45 19900 1192.7 (+) 2760.3 83.5%LN 36 15732 245.5 (−) 644.3 98.7%HS 62 3287 244.9 (+) 247.1 100.0%SHS 62 3475 246.4 (+) 266.1 100.0%FR 50 17442 1382.5 (+) 2144.4 93.7%SFR 50 9425 376.6 (+) 613.4 98.9%PR 61 19025 428.1 (+) 940.5 99.5%SPR 61 11644 312.9 (+) 530.1 100.0%Perry 34 19901 2033.4 (+) 3766.9 90.8%SPerry 45 1754 182.2 162.6 100.0%PÐnakac 4.12: Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc arijm¸n4.4.6 Taxinìmhsh twn Fut¸n thc Oikogèneiac IrisGia to prìblhma thc taxinìmhshc twn fut¸n thc oikogèneiac Iris (blèpe ParˆrthmaA') qrhsimopoi same thn mèjodo thc diastaurwmènhc epikÔrwshc apì 10 mèrh. Tadedomèna qwrÐsthkan se 10 uposÔnola to kajèna apì ta opoÐa eÐqan 5 parousÐec apìkˆje kathgorÐa. Ta sÔnola ekpaÐdeushc perièqoun 135 prìtupa, 45 apì kˆje kathgorÐa.Ta upìloipa 15 prìtupa qrhsimopoi jhkan gia ton èlegqo thc genÐkeushc twnalgorÐjmwn ekpaÐdeushc. Kˆje forˆ èna apì ta 10 uposÔnola qrhsimopoioÔntan giathn exètash thc genÐkeushc en¸ ta upìloipa gia thn ekpaÐdeush tou diktÔou. Gia toprìblhma autì qrhsimopoi same èna 4−4−3 teqnhtì neurwnikì dÐktuo (28 bˆrh kai 7merolhyÐec). To krufì epÐpedo kai to epÐpedo exìdou tou neurwnikoÔ diktÔou eÐnai basismènase neur¸nec me logistik sunˆrthsh energopoÐhshc. H sunj kh termatismoÔeÐnai E ≤ 0.02 kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoceÐnai 1000. Ta arqikˆ bˆrh tou neurwnikoÔ diktÔou arqikopoi jhkan me tuqaÐoucarijmoÔc mèsa apì to kleistì diˆsthma [−1, 1]. Ta apotelèsmata twn exomoi¸sewnparousiˆzontai ston PÐnaka 4.13 kai thn Eikìna 4.18.'Oson aforˆ touc klasikoÔc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn, oi algìrijmoiPR kai Perry parousÐasan thn kalÔterh epÐdosh afoÔ sunèklinan se ìlec ticexomoi¸seic pou diex qjhkan, akoloujoÔmenoi apì touc algìrijmouc HS kai FR. 'Osonaforˆ thn taqÔthta sÔgklishc, o algìrijmoc FR sunèkline grhgorìtera se autìto prìblhma akoloujoÔmenoc apì touc algìrijmouc Perry, HS kai PR. Oi klimakwtoÐalgìrijmoi ekpaÐdeushc suzug¸n klÐsewn parousiˆsthkan beltiwmènoi se kˆpoiecpeript¸seic ènanti twn klasik¸n touc analìgwn. Pio sugkekrimèna, oi algìrijmoiekpaÐdeushc SPerry kai SHS apèdwsan parìmoia me ta klasikˆ touc anˆloga. ParousÐasanto Ðdio posostì epituqÐac, allˆ o algìrijmoc SPerry tan taqÔteroc sesqèsh me ton algìrijmo Perry, en¸ o algìrijmoc SHS tan oriakˆ pio argìc apì ton

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 1070.250.2CPU Time (sec)0.150.10.050OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.18: Mèsoc qrìnoc gia to prìblhma thc taxinìmhshc twn fut¸n thc oikogèneiacIrisAlgìrijmoc min max µ σ EpituqÐa GenÐkeushOSS 18 979 154.5 (+) 92.4 95.6% 96.8%GB1 14 910 51.8 (+) 37.6 99.2% 96.7%GB2 15 823 53.8 (+) 36.8 99.2% 96.8%LN 17 307 51.9 (+) 19.9 99.9% 96.7%HS 17 488 55.2 (+) 28.1 99.8% 96.8%SHS 17 505 55.3 (+) 29.6 99.8% 96.8%FR 18 504 48.9 (+) 31.4 99.5% 96.8%SFR 18 780 49.8 (+) 48.6 99.8% 96.7%PR 22 360 71.5 (+) 27.7 100.0% 96.9%SPR 22 590 77.9 (+) 41.9 99.6% 96.9%Perry 15 192 50.5 (+) 17.2 100.0% 96.9%SPerry 19 237 48.6 16.8 100.0% 96.9%PÐnakac 4.13: Sugkritikˆ apotelèsmata gia to prìblhma thc taxinìmhshc twn fut¸nthc oikogèneiac Iris

108 Neoi Algorijmoi Ekpaideushc TNDalgìrijmo HS. O algìrijmoc SFR parousiˆsthke kalÔteroc se posostì epituqÐac sesqèsh me ton algìrijmo FR, allˆ me elˆqista parapˆnw sunarthsiakoÔc upologismoÔckai upologismoÔc klÐsewn. O algìrijmoc ekpaÐdeushc SPR parousiˆsthke qeirìterocapì ton algìrijmo PR. Oi algìrijmoi ekpaÐdeushc yeudì-Newton apèdwsan epÐshckalˆ kai tan antagwnistikoÐ me touc algìrijmouc ekpaÐdeushc suzug¸n klÐsewn meexaÐresh ton algìrijmo OSS. 'Oson aforˆ thn genÐkeush twn algorÐjmwn ekpaÐdeushc,oi algìrijmoi PR, Perry, SPR kai SPerry parousÐasan thn kalÔterh epÐdosh. Oiupìloipoi algìrijmoi ekpaÐdeushc parousÐasan epÐshc kalì posostì genÐkeushc, allˆtan qamhlìtero apì to posostì twn proanaferjèntwn algorÐjmwn. SunoyÐzontacthn apìdosh twn algorÐjmwn ekpaÐdeushc gia to prìblhma thc taxinìmhshc twn fut¸nthc oikogèneiac Iris, o kalÔteroc algìrijmoc ekpaÐdeushc tan o SPerry afoÔparousÐase èna exairetikì posostì epituqÐac, me th grhgorìterh taqÔthta sÔgklishckai exairetikì posostì genÐkeushc. EpÐshc, o èlegqoc Wilcoxon sign rank test èdeixeìti ta apotelèsmata tou algorÐjmou ekpaÐdeushc SPerry eÐnai statistikˆ shmantikˆènanti twn upoloÐpwn algorÐjmwn ìson aforˆ thn taqÔthta sÔgklishc.4.4.7 Optik Anagn¸rish Qeirìgrafwn Arijm¸nGia to prìblhma thc optik c anagn¸rishc qeirìgrafwn arijm¸n (blèpe Parˆrthma A')qrhsimopoi same èna 64-32-10 teqnhtì neurwnikì dÐktuo (2368 bˆrh kai 42 merolhyÐec).To krufì epÐpedo kai to epÐpedo exìdou tou neurwnikoÔ diktÔou eÐnai basismènase neur¸nec me logistik sunˆrthsh energopoÐhshc. H ekpaÐdeush tou teqnhtoÔ neurwnikoÔdiktÔou oloklhr¸netai ìtan to dÐktuo èqei sfˆlma taxinìmhshc tou sunìlouekpaÐdeushc ligìtero apì 2%. O mègistoc epitreptìc arijmìc upologism¸n thc tim cthc sunˆrthshc sfˆlmatoc eÐnai 10000 kai pragmatopoi jhkan 100 exomoi¸seic. Taarqikˆ bˆrh tou neurwnikoÔ diktÔou arqikopoi jhkan me tuqaÐouc arijmoÔc mèsa apìto kleistì diˆsthma [−1, 1]. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai stonPÐnaka 4.14 kai thn Eikìna 4.19.Se autì to prìblhma, oi algìrijmoi ekpaÐdeushc SHS, SPR kai SPerry parousÐasanta Ðdia exairetikˆ posostˆ epituqÐac se sqèsh me ta klasikˆ touc anˆloga, allˆ eÐqantaqÔterh sÔgklish. Apì thn ˆllh meriˆ, o algìrijmoc SFR ìqi mìno parousÐase taqÔterhsÔgklish apì ton algìrijmo FR, allˆ eÐqe kai megalÔtero posostì epituqÐac.'Oson aforˆ touc algìrijmouc ekpaÐdeushc yeudì-Newton, oi algìrijmoi OSS, GB1kai GB2 eÐqan kak apìdosh se autì to prìblhma, afoÔ parousÐasan qamhlˆ posostˆepituqÐac kai arg sÔgklish. Apì thn ˆllh meriˆ, o algìrijmoc ekpaÐdeushc LN apèdwseexairetikˆ afoÔ sunèkline se ìlec tic exomoi¸seic pou pragmatopoi jhkan me thntaqÔterh sÔgklish. 'Oson aforˆ ta apotelèsmata genÐkeushc twn algorÐjmwn ekpaÐdeushc,oi algìrijmoi yeudì-Newton parousÐasan ta qamhlìtera posostˆ genÐkeushc.O algìrijmoc ekpaÐdeushc SHS parousÐase uyhlìtero mèso posostì genÐkeushc apìton algìrijmo HS. Oi upìloipoi klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewneÐqan thn Ðdia epÐdosh ìson aforˆ thn genÐkeush me ta klasikˆ touc anˆloga. Oi

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 109180160140120CPU Time (sec)100806040200OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.19: Mèsoc qrìnoc gia to prìblhma thc optik c anagn¸rishc qeirìgrafwnarijm¸nAlgìrijmoc min max µ σ EpituqÐa GenÐkeushOSS 357 2126 979.5 (+) 566.8 12% 94.7%GB1 239 7184 1599.4 (+) 2306.9 12% 94.6%GB2 213 1093 441.5 (+) 323.1 10% 94.8%LN 77 1259 161.4 174.2 100% 94.8%HS 65 1235 235.1 (+) 250.3 99% 95.2%SHS 65 1169 233.4 (+) 249.9 99% 95.3%FR 75 4001 435.4 (+) 818.8 97% 94.9%SFR 65 3157 349.3 (+) 518.1 99% 94.9%PR 102 9755 429.5 (+) 1135.6 100% 95.3%SPR 120 5554 417.5 (+) 790.2 100% 95.3%Perry 80 2251 244.4 (+) 299.8 100% 95.3%SPerry 66 1059 197.9 212.3 100% 95.3%PÐnakac 4.14: Sugkritikˆ apotelèsmata gia to prìblhma thc optik c anagn¸rishcqeirìgrafwn arijm¸n

110 Neoi Algorijmoi Ekpaideushc TNDAlgìrijmoc min max µ σ EpituqÐa GenÐkeushOSS 890 2334 1113.5 (+) 327.7 21% 55.4%GB1 674 1612 1037.5 (+) 323.1 8% 49.1%GB2 746 2708 1392.5 (+) 891.4 6% 49.2%LN 238 2730 511.8 (+) 380.5 99% 53.5%HS 288 1629 447.2 (+) 156.6 99% 55.3%SHS 312 1790 456.1 (+) 189.7 100% 55.3%FR 263 4166 955.2 (+) 759.9 83% 51.9%SFR 275 4039 899.7 (+) 635.2 98% 52.1%PR 476 4502 981.1 (+) 650.6 100% 55.5%SPR 460 3730 928.3 (+) 555.3 100% 55.6%Perry 279 981 444.6 (+) 110.1 98% 55.4%SPerry 298 665 404.9 78.4 100% 55.4%PÐnakac 4.15: Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc fwnhèntwnalgìrijmoi ekpaÐdeushc PR, SPR, Perry kai SPerry parousÐasan to uyhlìtero mèsoposostì genÐkeushc. Sunolikˆ. o algìrijmoc ekpaÐdeushc SPerry apèdwse kalÔteraapì touc upìloipouc algorÐjmouc se autì to prìblhma afoÔ sunèkline se ìlec ticexomoi¸seic me thn deÔterh kalÔterh taqÔthta sÔgklishc (kontˆ sthn epÐdosh tou algorÐjmouLN) kai to kalÔtero mèso posostì genÐkeushc. EpÐshc o èlegqoc Wilcoxonsign rank test èdeixe ìti ta apotelèsmata tou algorÐjmou ekpaÐdeushc SPerry eÐnaistatistikˆ shmantikˆ ènanti twn upoloÐpwn algorÐjmwn pou apèdwsan qeirìtera apìautìn ìson aforˆ thn taqÔthta sÔgklishc.4.4.8 Anagn¸rish FwnhèntwnGia to prìblhma thc anagn¸rishc fwnhèntwn (blèpe Parˆrthma A') qrhsimopoi sameèna 10-88-11 teqnhtì neurwnikì dÐktuo (1856 bˆrh kai 99 merolhyÐec). To krufì e-pÐpedo kai to epÐpedo exìdou tou neurwnikoÔ diktÔou eÐnai basismèna se neur¸nec melogistik sunˆrthsh energopoÐhshc. H ekpaÐdeush tou teqnhtoÔ neurwnikoÔ diktÔouoloklhr¸netai ìtan to dÐktuo èqei sfˆlma taxinìmhshc tou sunìlou ekpaÐdeushcligìtero apì 5%. O mègistoc epitreptìc arijmìc upologism¸n thc tim c thc sunˆrthshcsfˆlmatoc eÐnai 5000 kai pragmatopoi jhkan 100 exomoi¸seic. Ta arqikˆ bˆrhtou neurwnikoÔ diktÔou arqikopoi jhkan me tuqaÐouc arijmoÔc mèsa apì to kleistìdiˆsthma [−0.3, 0.3]. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka4.15 kai thn Eikìna 4.20.Se autì to prìblhma, oi klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn parousÐasanbeltiwmènh apìdosh se sqèsh me ta klasikˆ touc anˆloga. O algìrijmocekpaÐdeushc SHS tan elˆqista pio argìc se sqèsh me ton algìrijmo HS, allˆ eÐqemegalÔtero posostì epituqÐac. O algìrijmoc ekpaÐdeushc SPR parousÐase to Ðdio

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 11150454035CPU Time (sec)302520151050OSS GB1 GB2 LN HS SHS FR SFR PR SPR Perry SPerryTraining AlgorithmsSq ma 4.20: Mèsoc qrìnoc gia to prìblhma thc anagn¸rishc fwnhèntwnposostì epituqÐac me ton algìrijmo PR, allˆ sunèkline taqÔtera. Oi algìrijmoiekpaÐdeushc SFR kai SPerry tan kalÔteroi apì ta klasikˆ touc anˆloga tìso seposostˆ epituqÐac ìso kai se taqÔthta sÔgklishc. O algìrijmoc ekpaÐdeushc SPerryparousÐase thn taqÔterh sÔgklish, kai ìpwc fˆnhke apì ton èlegqo Wilcoxon signrank test, ta apotelèsmata tan statistikˆ shmantikˆ se sqèsh me touc upìloipoucalgorÐjmouc se autì to prìblhma. Apì touc algìrijmouc ekpaÐdeushc yeudì-Newton,mìno o algìrijmoc LN tan antagwnistikìc me touc algìrijmouc ekpaÐdeushc suzug¸nklÐsewn. 'Oson aforˆ thn genÐkeush twn algorÐjmwn ekpaÐdeushc, o algìrijmocSPR parousÐase ta kalÔtero mèso posostì genÐkeushc akoloujoÔmenoc apì ton algìrijmoPR. Oi algìrijmoi ekpaÐdeushc Perry kai SPerry parousÐasan epÐshc kalˆapotelèsmata genÐkeushc. O algìrijmoc ekpaÐdeushc OSS parousÐase thn Ðdia ikanìthtagenÐkeushc me touc algorÐjmouc Perry kai SPerry, allˆ sunèkline mìno se lÐgecexomoi¸seic me shmantikˆ qamhlìterh taqÔthta sÔgklishc. Oi algìrijmoi ekpaÐdeushcSHS kai SHS parousÐasan epark apìdosh genÐkeushc. O algìrijmoc ekpaÐdeushcSFR parousÐase kalÔtero posostì mèshc genÐkeushc apì to klasikì tou anˆlogo.Wstìso to posostì genÐkeushc touc tan shmantikˆ qamhlìtero apì autì twn proanaferjèntwnalgorÐjmwn ekpaÐdeushc kai eÐnai sugkrÐsimo mìno me autì tou algorÐjmouLN. 'Oson aforˆ touc algìrijmouc GB1 kai GB2 h sunolik touc apìdosh tan qeirìterhsugkrinìmenh me aut twn upoloÐpwn algorÐjmwn.

112 Neoi Algorijmoi Ekpaideushc TND4.5 SumperˆsmataSe autì to kefˆlaio parousiˆsthkan nèoi algìrijmoi ekpaÐdeushc suzug¸n klÐsewn.Ektìc apì touc upˆrqontec algìrijmouc ekpaÐdeushc HS, FR kai PR, protˆjhke hmèjodoc tou Perry [60] gia qr sh sthn ekpaÐdeush twn teqnht¸n neurwnik¸n diktÔwn.Sth sunèqeia, parˆqjhke mia nèa kathgorÐa algorÐjmwn ekpaÐdeushc suzug¸n klÐsewnpou onomˆzontai klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn kai basÐzontaisto fasmatikì b ma twn Barzilai kai Borwein [5]. Epiplèon enswmat¸jhke stoucalgìrijmouc ekpaÐdeushc suzug¸n klÐsewn mia apotelesmatik teqnik grammik c a-naz thshc pou basÐzetai stic sunj kec tou Wolfe kai se mia diasfalismènh kubikparembol [77]. Epiplèon, o arqikìc rujmìc ekpaÐdeushc, pou trofodoteÐ thn diadikasÐathc grammik c anaz thshc, prosarmìzetai autìmata se kˆje epanˆlhyh sÔmfwname èna kleistì tÔpo pou protˆjhke stic ergasÐec [77, 80]. Tèloc, efarmìsthke mia a-podotik diadikasÐa epanekkÐnhshc h opoÐa belti¸nei epiplèon thn apotelesmatikìthtatwn algorÐjmwn ekpaÐdeushc suzug¸n klÐsewn.Anafèrjhkan peiramatikˆ apotelèsmata gia 14 probl mata ekpaÐdeushc. Ta peirˆmatapou diex qjhsan èdeixan ìti oi proteinìmenoi algìrijmoi ekpaÐdeushc apèdwsankalˆ kai ìti tan antagwnistikoÐ me touc upˆrqontec algìrijmouc ekpaÐdeushc.Stic perissìterec peript¸seic parousÐasan kalÔtera posostˆ epituqÐac kai taqÔthtasÔgklishc. H ikanìthta genÐkeushc tou teqnhtoÔ neurwnikoÔ diktÔou eÐnai èna polÔdÔskolo jèma. Genikˆ, h genÐkeush tou diktÔou ephreˆzetai apì pollèc paramètrouc.Exartˆtai apì to mègejoc tou q¸rou twn bar¸n (arqitektonik tou diktÔou), to mègejoctou sunìlou dedomènwn kai o endeqìmenoc jìruboc pou mporeÐ na perièqoun,thn arqikopoÐhsh twn bar¸n kai apì ton algìrijmo ekpaÐdeushc pou qrhsimopoieÐtaikatˆ thn diˆrkeia thc ekpaÐdeushc. EÐnai gnwstì ìti h gr gorh ekpaÐdeush mporeÐna odhg sei se ftwqˆ apotelèsmata genÐkeushc. Sta peirˆmata pou diex qjhsan, harqitektonik tou diktÔou, ta dedomèna kai h mèjodoc arqikopoÐhshc twn bar¸n toudiktÔou tan stajerˆ gia ìlouc touc algìrijmouc ekpaÐdeushc. H mình diaforˆ tanoi algìrijmoi ekpaÐdeushc. ApodeÐqjhke mèsw twn peiramatik¸n apotelesmˆtwn ìtiparìlo pou oi proteinìmenoi algìrijmoi ekpaÐdeushc sugklÐnoun tic perissìterec forèctaqÔtera apì touc upìloipouc algìrijmouc ekpaÐdeushc, parousiˆzoun kai kalˆapotelèsmata genÐkeushc. Aut h sumperiforˆ mporeÐ na exhghjeÐ apì to gegonìcìti oi klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn kˆnoun kal qr sh thcdiajèsimhc plhroforÐac katˆ thn anaz thsh tou elaqÐstou sto q¸ro twn bar¸n. A-pofeÔgoun tic perissìterec forèc ta topikˆ elˆqista kai sugklÐnoun se {kalˆ} shmeÐalÔshc.O proteinìmenoc algìrijmoc ekpaÐdeushc SPerry tan o kalÔteroc algìrijmoc.ParousÐase ta kalÔtera apotelèsmata sqedìn se ìla ta probl mata. Epiplèon, ta a-potelèsmata tan isqurˆ statistik¸c shmantikˆ ìson aforˆ thn taqÔthta sÔgklishc,afoÔ h p-tim pou epèstrefe o èlegqoc Wilcoxon sign rank test tan kontˆ sto 0 sticperissìterec peript¸seic. EÐnai wstìso aparaÐthto na ereunhjeÐ peraitèrw h apìdosh

Nea Oikogeneia Algorijmwn Ekpaideushc Suzugwn Klisewn 113twn klimakwt¸n algorÐjmwn ekpaÐdeushc suzug¸n klÐsewn se ˆlla probl mata ekpaÐdeushcètsi ¸ste na diereunhjoÔn katˆ to dunatìn plhrèstera ta pleonekt matˆ touckai na apokalufjoÔn tuqìn periorismoÐ. Parìla autˆ, eÐnai shmantikì na anaferjeÐìti oi klimakwtoÐ algìrijmoi ekpaÐdeushc suzug¸n klÐsewn apoteloÔn beltÐwsh ènantitwn klasik¸n algorÐjmwn ekpaÐdeushc suzug¸n klÐsewn kai ìti eÐnai antagwnistikoÐme upˆrqontec algìrijmouc ekpaÐdeushc thc Ðdiac kathgorÐac.Upˆrqei mia megˆlh dunatìthta na beltiwjeÐ peraitèrw h apìdosh twn proteinìmenwnklimakwt¸n algorÐjmwn suzug¸n klÐsewn uiojet¸ntac nèec idèec apì thn perioqthc beltistopoÐhshc qwrÐc periorismoÔc. Pio sugkekrimèna, nèec teqnikèc grammik canaz thshc, pou basÐzontai sthn mh monìtonh strathgik [29], mporoÔn na qrhsimopoihjoÔnme touc proteinìmenouc algorÐjmouc. Aut h strathgik qrhsimopoi jhkesthn ergasÐa [64] gia thn beltÐwsh thc apìdoshc twn algorÐjmwn ekpaÐdeushc pou basÐzontaisthn kateÔjunsh thc pio apìtomhc kajìdou. Apì thn ˆllh meriˆ, o èlegqocthc gwnÐac pou qrhsimopoioÔn oi proteinìmenoi algìrijmoi ekpaÐdeushc wc sunj khepanekkÐnhshc mporeÐ na antikatastajeÐ me pio isqurèc diadikasÐec epanekkÐnhshc [66].EpÐshc, h parˆmetroc klimˆkwshc pou qrhsimopoieÐtai stouc proteinìmenouc algorÐjmoucmporeÐ na enisqujeÐ akìmh kai na antikatastajeÐ apì ˆllec epilogèc [81].Tèloc, ˆlloi algìrijmoi ekpaÐdeushc pou èqoun protajeÐ gia thn ekpaÐdeush teqnht¸nneurwnik¸n diktÔwn (p.q. [37]) mporoÔn na sumperilhfjoÔn mèsa sthn oikogèneiatwn klimakwt¸n algorÐjmwn ekpaÐdeushc suzug¸n klÐsewn, me endeqìmenh kalÔterhapìdosh.

114 Neoi Algorijmoi Ekpaideushc TND

Kefalaio 5Mh Monìtonoi AlgìrijmoiEkpaÐdeushc Suzug¸n KlÐsewnSto prohgoÔmeno kefˆlaio parousiˆsthke mia nèa kathgorÐa algorÐjmwn ekpaÐdeushcpou basÐzontai sthn klimakwt kateÔjunsh suzug¸n klÐsewn. Se autì to kefˆlaioapomon¸netai kai tropopoieÐtai o klimakwtìc algìrijmoc tou Perry. Pio sugkekrimèna,diathroÔntai merikˆ apì ta qarakthristikˆ tou algorÐjmou ekpaÐdeushc, ìpwc eÐnaih parˆmetroc klimˆkwshc pou basÐzetai sto fasmatikì b ma twn Barzilai kai Borwein[5], h sunj kh epanekkÐnhshc kai o arqikìc rujmìc ekpaÐdeushc, allˆ efarmìzetaimia diaforetik teqnik grammik c anaz thshc h opoÐa basÐzetai stic mh monìtonecsunj kec tou Wolfe. EpÐshc proteÐnetai ènac nèoc arqikìc rujmìc ekpaÐdeushc giaqr sh me ton klimakwtì algìrijmo ekpaÐdeushc suzug¸n klÐsewn o opoÐoc faÐnetaina eÐnai apodotikìteroc apì ton arqikì rujmì ekpaÐdeushc pou protˆjhke apì tonShanno [77] ìtan qrhsimopoieÐtai se sunduasmì me thn mh monìtonh teqnik grammik canaz thshc. Sthn sunèqeia parousiˆzontai ta peiramatikˆ apotelèsmata gia diˆforaprobl mata ekpaÐdeushc. Tèloc, ekpaideÔetai èna poluepÐpedo emprìsjia trofodotoÔmenoteqnhtì neurwnikì dÐktuo me ton proteinìmeno algìrijmo gia to prìblhma thctaxinìmhshc karkinik¸n kuttˆrwn tou egkefˆlou kai sugkrÐnetai h apìdos tou meaut enìc pijanotikoÔ teqnhtoÔ neurwnikoÔ diktÔou.5.1 O Algìrijmoc tou PerryTo prìblhma thc ekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn dÐnetai apì thn parakˆtwsqèshmin E(w), w ∈ R n (5.1)115

116 Neoi Algorijmoi Ekpaideushc TNDìpou h E eÐnai h sunˆrthsh sfˆlmatoc, pou dÐnetai apì thn exÐswsh (2.2). O algìrijmocthc klasik c mejìdou suzug¸n klÐsewn gia thn epÐlush tou probl matoc (5.1)eÐnai èna epanalhptikì sq ma thc morf cw k+1 = w k − α k d k (5.2)ìpou d k eÐnai h kateÔjunsh anaz thshc kai α k eÐnai o rujmìc ekpaÐdeushc. O rujmìcekpaÐdeushc α k mporeÐ na kajoristeÐ apì mia teqnik grammik c anaz thshc ètsi ¸steh E(w k + α k d k ) na elaqistopoieÐtai katˆ m koc thc kateÔjunshc d k , ìtan ta w k kaid k eÐnai stajerˆ.O klasikìc algìrijmoc ekpaÐdeushc suzug¸n klÐsewn xekinˆ thn diadikasÐa thcelaqistopoÐhshc me mia arqik ektÐmhsh tou w 0 kai mia arqik kateÔjunshd 0 = −∇E(w 0 ) = −g 0Kˆje kateÔjunsh d k+1 epilègetai ètsi ¸ste na eÐnai grammikìc sunduasmìc thc kateÔjunshcthc pio apìtomhc kajìdou −g k+1 kai thc prohgoÔmenhc kateÔjunshc d k .Epomènwc èqoumed k+1 = −g k+1 + β k d k (5.3)ìpou h bajmwt parˆmetroc β k eÐnai upì kajorismì apait¸ntac ìti h kateujÔnseic d kkai d k+1 prèpei na ikanopoioÔn thn sunj kh thc suzugÐac. Upˆrqoun polloÐ tÔpoi giathn parˆmetro β k . 'Enac apì autoÔc eÐnai o tÔpoc pou prìteine o Perry [60] kai dÐnetaiparakˆtw.β k = (y k − s k ) T g k+1s T k y k(5.4)ìpous k = w k+1 − w k kai y k = g k+1 − g k (5.5)5.2 Mh Monìtonoc Klimakwtìc Algìrijmoc EkpaÐdeushctou PerryLambˆnontac upìyin thn ergasÐa twn Birgin kai Martinez [9], upotÐjetai ìti mia piogenik morf thc kateÔjunshc anaz thshc suzug¸n klÐsewn dÐnetai apì thn exÐswshìpou g k = ∇E(w k ) kai d 0 = −g 0 .d k+1 = −ϑ k g k+1 + β k s k (5.6)

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 117Ac upojèsoume ìti E(w) ∈ C 2 kai ìti o Essianìc pÐnakac H ≡ ∇ 2 E(w) eÐnai jetikˆorismènoc. Autì upodhl¸nei ìti y k ≠ 0. 'Etsi to shmeÐo elaqÐstou w ∗ ikanopoieÐ thnsqèshìpouw ∗ = w k+1 + d ∗Hd ∗ = −g k+1Pollaplasiˆzontac thn prohgoÔmenh sqèsh me to diˆnusma s T k , èqoume ìtiEpomènwcs T k Hd ∗ = −s T k g k+1'Etsi to uperepÐpedoy T k d ∗ = −s T k g k+1H k ≡ {d ∈ R n |y T k d = −s T k g k+1 }perièqei thn bèltisth prosaÔxhsh d ∗ , h opoÐa dÐnei ìti w ∗ = w k+1 +d ∗ . ParathroÔme ìtih mhdenik kateÔjunsh d = 0 an kei sto uperepÐpedo H mìno an isqÔei ìti s T k g k+1 = 0to opoÐo eÐnai antÐjeto me thn arqik upìjes mac.Apì ta parapˆnw, eÐnai fusiologikì na epibˆlloume gia thn kateÔjunsh anaz thshcd k+1 ìtiTìte, apì thn sqèsh (5.6), èqoume ìtid k+1 ∈ H k (5.7)β k = (ϑ ky k − s k ) T g k+1s T k y k(5.8)Gia ϑ k = 1 o parapˆnw tÔpoc gia thn tim tou β k sumpÐptei me ton tÔpo (5.4) pouprìteine o Perry sthn ergasÐa [60].Stìqoc eÐnai na brejeÐ mia katˆllhlh epilog gia thn parˆmetro klimˆkwshc ϑ kètsi ¸ste na beltiwjeÐ h apìdosh kai h taqÔthta sÔgklishc tou klasikoÔ algìrijmouekpaÐdeushc tou Perry. Mia kal kai apodotik epilog gia thn parˆmetro klimˆkwshceÐnai to fasmatikì b ma pou protˆjhke apì touc Barzilai kai Borwein [5], dhladϑ k = sT k s ks T k y k(5.9)

118 Neoi Algorijmoi Ekpaideushc TNDOi idiìthtec kai oi lìgoi gia thn epilog tou b matoc twn Barzilai kai Borwein anafèrontaista prohgoÔmena kefˆlaia kai gi' autì ton lìgo den anafèrontai ed¸.'Opwc anafèrjhke prohgoumènwc, oi algìrijmoi suzug¸n klÐsewn gia na èqoun thnidiìthta thc olik c sÔgklishc prèpei h parˆmetroc tou rujmoÔ ekpaÐdeushc na kajorÐzetaiapì mia teqnik monodiˆstathc grammik c anaz thshc katˆ m koc thc kateÔjunshcsuzug¸n klÐsewn d k . Ston algìrijmo pou perigrˆfetai, o rujmìc ekpaÐdeushcpou lambˆnetai apì thn teqnik thc grammik c anaz thshc prèpei na ikanopoieÐ tic mhmonìtonec sunj kec tou Wolfe pou dÐnontai apì tic parakˆtw exis¸seicE(w k + α k d k ) − max0≤j≤M E(w k−j) ≤ c 1 α k ∇E(w k ) T d k (5.10)∇E(w k + α k d k ) T d k ≥ c 2 ∇E(w k ) T d k (5.11)ìpou 0 < c 1 ≤ c 2 < 1 kai M eÐnai ènac jetikìc akèraioc arijmìc, mh monìtonoc orÐzontacekpaÐdeushc. H pr¸th sunj kh (5.10) epitrèpei se opoiod pote shmeÐo na gÐneiapodektì an belti¸nei ikanopoihtikˆ thn sunarthsiak tim anaforikˆ me th megalÔterhapì tic M + 1 ( k an k ≤ M) pio prìsfatec sunarthsiakèc timèc. H deÔterhsunj kh (5.11) diasfalÐzei ìti o paronomast c thc fasmatik c klimakwt c paramètrouìti eÐnai pˆnta kalˆ orismènoc kai jetikìc, afoÔ upodhl¸nei ìti s T k y k > 0. Kaioi duo sunj kec epitrèpoun mia aÔxhsh sthn tim thc sunˆrthshc sfˆlmatoc qwrÐc naephreˆzei thn idiìthta thc olik c sÔgklishc ìpwc èqei apodeiqjeÐ sthn ergasÐa [30].Mia apl teqnik grammik c anaz thshc pou qrhsimopoieÐtai gia thn rÔjmish thcparamètrou tou rujmoÔ ekpaÐdeushc α k eÐnai h meÐwsh tou rujmoÔ ekpaÐdeushc katˆènan parˆgonta 1/q, ìpou q > 1, ètsi ¸ste na ikanopoioÔntai oi sunj kec (5.10) kai(5.11). H epilog tou q den eÐnai shmantik gia thn epituq ekpaÐdeush, allˆ èqeiˆmesh epÐdrash ston arijmì twn upologism¸n thc sunˆrthshc sfˆlmatoc kai thc klÐsewcthc. Mikrèc timèc thc paramètrou q mporeÐ na odhg soun se pollèc epanal yeicthc diadikasÐac thc grammik c anaz thshc en¸ megˆlec timèc mporeÐ na odhg soun seaÔxhsh twn epanal yewn tou algorÐjmou genikˆ. Sthn bibliografÐa proteÐnetai sunjwc h tim q = 2, h opoÐa leitourgeÐ tic perissìterec forèc apotelesmatikˆ kaiapodotikˆ. Aut h strathgik thc opÐsjiac iqnhlˆthshc diasfalÐzei ìti o rujmìc ekpaÐdeushcelatt¸netai ètsi ¸ste na ikanopoieÐtai h sunj kh (5.10). Pio sugkekrimèna,h elˆttwsh tou rujmoÔ ekpaÐdeushc dÐnetai apì ton tÔpo ˜α k = 2 −r α k , ìpou ˜α k eÐnaio rujmìc ekpaÐdeushc metˆ apì r upodiairèseic pou apaitoÔntai gia na ikanopoieÐtai hsunj kh (5.10). Epiplèon, gia na apofeuqjeÐ h ˆskoph elˆttwsh thc paramètrou tourujmoÔ ekpaÐdeushc, epibˆlletai ìti o rujmìc ekpaÐdeushc prèpei na ikanopoieÐ thnsunj kh (5.11).Mia dhmofil c epilog tou arqikoÔ rujmoÔ ekpaÐdeushc pou apodedeigmèna sunergˆzetaiepituq¸c me thc mejìdouc suzug¸n klÐsewn èqei protajeÐ apì ton Shannoston algìrijmo CONMIN [77]. O arqikìc rujmìc ekpaÐdeushc tou Shanno dÐnetai

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 119apì ton tÔpoα k ={1||g 0 ||, an k = 0;α k−1 ||d k ||||d k−1 ||, diaforetikˆ.(5.12)ìpou d k eÐnai h kateÔjunsh anaz thshc suzug¸n klÐsewn, d k−1 eÐnai h prohgoÔmenhkateÔjunsh kai α k−1 eÐnai o prohgoÔmenoc rujmìc ekpaÐdeushc. O arqikìc rujmìcmˆjhshc α 0 eÐnai 1/∥g 0 ∥ ìpou g 0 eÐnai h arqik kateÔjunsh thc pio apìtomhc kajìdou.Mia ˆllh apotelesmatik epilog tou rujmoÔ ekpaÐdeushc, pou protˆjhke sthnergasÐa [41], dÐnetai apì ton parakˆtw tÔpoα k ={ 1||g 0 ||, an k = 0;2 −(r+1) ∥d k−1 ∥, diaforetikˆ. (5.13)∥d k −d k−1 ∥ìpou r eÐnai o arijmìc twn upodiairèsewn pou qreiˆzetai o prohgoÔmenoc rujmìc ekpaÐdeushc¸ste na gÐnei apodektìc apì tic mh monìtonec sunj kec tou Wolfe (5.10)kai (5.11).H kateÔjunsh anaz thshc suzug¸n klÐsewn d k+1 pou dÐnetai apì ton tÔpo (5.6)merikèc forèc apotugqˆnei na eÐnai kateÔjunsh kajìdou. Gia na apofeuqjoÔn autèc oipajologikèc katastˆseic o algìrijmoc ekpaÐdeushc epanekkineÐtai. Pollèc teqnikècepanekkÐnhshc èqoun protajeÐ sthn bibliografÐa. Ston proteinìmeno algìrijmo ekpaÐdeushc,o algìrijmoc epanekkineÐtai me thn fasmatik kateÔjunsh thc pio apìtomhckajìdou pou dÐnetai apì ton parakˆtw tÔpod k+1 = −η k g k+1 (5.14)To krit rio pou qrhsimopoieÐtai gia na apofasisteÐ an h kateÔjunsh d k+1 eÐnai kajodik, dÐnetai apì ton akìloujo tÔpod T k+1g k+1 ≤ −10 −3 ∥d k+1 ∥∥g k+1 ∥ (5.15)Pio sugkekrimèna, an isqÔei h sqèsh (5.15), gÐnetai apodekt h kateÔjunsh anaz thshcpou dÐnetai apì thn sqèsh (5.6), diaforetikˆ h gwnÐa metaxÔ thc kateÔjunshc d k+1kai thc klÐshc g k+1 den eÐnai arketˆ oxeÐa, opìte epanekkineÐtai o algìrijmoc me thnkateÔjunsh pou dÐnetai apì thn sqèsh (5.14).Oi mh monìtonec sunj kec tou Wolfe (5.10) kai (5.11) mazÐ me thn sunj kh epanekkÐnhshc(5.15) eÐnai eparkeÐc gia na apodeÐxoun thn olik sÔgklish tou algorÐjmoukˆtw apì logikèc upojèseic. An h klÐsh thc sunˆrthshc sfˆlmatoc E, g k , eÐnai Lipschitzsuneq c kai h E eÐnai kˆtw fragmènh mporeÐ na apodeiqjeÐ ìtilim infk→∞ ∥g k∥ = 0

120 Neoi Algorijmoi Ekpaideushc TNDAutì upodhl¸nei ìti upˆrqei mia upakoloujÐa twn klÐsewn thc sunˆrthshc sfˆlmatocE pou sugklÐnei sto 0 [29, 56].Se autì to shmeÐo dÐnetai mia leptomer c diatÔpwsh tou mh monìtonou klimakwtoÔalgorÐjmou ekpaÐdeushc tou Perry. Shmei¸netai ìti gia ϑ k = 1, o algìrijmoc sumpÐpteime ton klasikì algìrijmo ekpaÐdeushc suzug¸n klÐsewn tou Perry. O algìrijmocekpaÐdeushc dèqetai wc eÐsodo to diˆnusma twn arqik¸n bar¸n w 0 , to krit rio termatismoÔϵ, ton mègisto arijmì epanal yewn µ, tic paramètrouc c 1 , c 2 kai M pouqreiˆzontai gia ton algìrijmo grammik c anaz thshc kai epistrèfei to shmeÐo elaqÐstouw ⋆ kai to elˆqisto thc sunˆrthshc sfˆlmatoc E(w ⋆ ).Algìrijmoc 6 O klimakwtìc algìrijmoc ekpaÐdeushc suzug¸n klÐsewn tou Perry.NMSP(w 0 , ϵ, µ, c 1 , c 2 , M, w ⋆ , E(w ⋆ ))1: Gia k := 0 èwc µ Kˆne2: E k = E(w k ), g k = ∇E(w k ).3: An E k ≤ ϵ Tìte4: Epèstreye w ⋆ = w k , E(w ⋆ ) = E(w k ).5: Tèloc6: An k := 0 Tìte7: d k = −g k .8: α k = 1/||g k ||.9: Alli¸c10: ϑ k = sT k s ks T k y k .11: Upolìgise to β k = (ϑ ky k −s k ) T g ks T k y .k12: d = −ϑ k g k + β k s k .13: An d T g k

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 1215.3 Peiramatikˆ ApotelèsmataSthn parˆgrafo aut axiologeÐtai h epÐdosh twn proteinìmenwn algorÐjmwn ekpaÐdeushcteqnht¸n neurwnik¸n diktÔwn NMSP1 [80] kai NMSP2 [41] kai sugkrÐnontai methn epÐdosh twn akìloujwn gnwst¸n algorÐjmwn ekpaÐdeushc:1. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me stajerì rujmìekpaÐdeushc (BP) [74].2. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me stajerì rujmìekpaÐdeushc kai orm (BPM) [74].3. Ton algìrijmo thc opisjodromik c diˆdoshc tou sfˆlmatoc me prosarmostikìrujmì ekpaÐdeushc kai orm (ABP) [84].Gia touc algìrijmouc ekpaÐdeushc NMSP1 kai NMSP2 oi parˆmetroi M, c 1 kai c 2èqoun tic timèc 10, 10 −4 kai 0.5, antÐstoiqa. Oi timèc twn parapˆnw paramètrwneÐnai oi Ðdiec gia ìla ta peirˆmata pou pragmatopoi same. Kai oi dÔo algìrijmoiqrhsimopoioÔn thn Ðdia mh monìtonh strathgik grammik c anaz thshc. Epiplèon, giathn arqikopoÐhsh twn bar¸n kai twn merolhyi¸n efarmìsthke h teqnik pou protˆjhkeapì touc Nguyen kai Widrow [55] gia ìlouc touc algorÐjmouc.Ta probl mata pou qrhsimopoi jhkan gia thn axiolìghsh thc taqÔthtac sÔgklishcwn proteinìmenwn algorÐjmwn ekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn NMSP1kai NMSP2 se sqèsh me touc proanaferjèntec algìrijmouc ekpaÐdeushc eÐnai ta akìlouja:1. To prìblhma tou ApokleistikoÔ-EITE.2. To prìblhma thc isotimÐac twn 3-bit.3. To prìblhma thc anagn¸rishc twn kefalaÐwn grammˆtwn thc Agglik c alfab -tou.4. To prìblhma thc prosèggishc thc suneqoÔc trigwnometrik c sunˆrthshc.Gia kˆje prìblhma, paratÐjetai ènac pÐnaka o opoÐoc sunoyÐzei thn apìdosh twnalgorÐjmwn gia prosomoi¸seic pou èftasan se lÔsh entìc enìc prokajorismènou o-rÐou upologism¸n thc sunˆrthshc sfˆlmatoc pou orÐzetai se kˆje prìblhma. Oianaferìmenec stouc pÐnakec parˆmetroi eÐnai: Min o elˆqistoc arijmìc epanal yewn,Max o mègistoc arijmìc epanal yewn, µ h mèsh tim twn epanal yewn, σ h tupikapìklish, kai EpituqÐa o arijmìc twn petuqhmènwn prosomoi¸sewn apì èna sÔnolo1000 dokim¸n. Se perÐptwsh pou ènac algìrijmoc apotÔqei na sugklÐnei mèsa sto

122 Neoi Algorijmoi Ekpaideushc TNDprokajorismèno ìrio twn upologism¸n thc sunˆrthshc sfˆlmatoc, jewreÐtai ìti apètuqena ekpaideÔsei to teqnhtì neurwnikì dÐktuo kai oi epanal yeic thc sugkekrimènhcprosomoÐwshc den perilambˆnontai sthn statistik anˆlush tou algorÐjmou.Se autì to shmeÐo, prèpei na epishmanjeÐ ìti genikˆ mia epanˆlhyh enìc algorÐjmouekpaÐdeushc shmaÐnei ìti ìla ta prìtupa ekpaÐdeushc èqoun parousiasteÐ sto teqnhtìneurwnikì dÐktuo. Stouc klasikoÔc algorÐjmouc, ìpwc h BP, kˆje epanˆlhyhantistoiqeÐ se èna sunarthsiakì upologismì thc sunˆrthshc sfˆlmatoc kai ènan upologismìthc klÐshc (parˆgwgoi pr¸thc tˆxewc). Stouc proteinìmenouc algorÐjmoucekpaÐdeushc suzug¸n klÐsewn lambˆnoun q¸ra perissìteroi sunarthsiakoÐ upologismoÐkai upologismoÐ klÐsewn anˆ epanˆlhyh, gegonìc pou ofeÐletai sthn parousÐathc teqnik c thc mh monìtonhc grammik c anaz thshc. ExaitÐac thc fÔsewc thc grammikc anazht sewc pou qrhsimopoieÐtai stouc algorÐjmouc autoÔc, apaiteÐtai o Ðdiocarijmìc sunarthsiak¸n upologism¸n kai upologism¸n klÐsewn anˆ epanˆlhyh. Dhladmetˆ thn ektèlesh thc diadikasÐac thc grammik c anaz thshc se mia epanˆlhyh toualgorÐjmou ekpaÐdeushc o Ðdioc arijmìc sunarthsiak¸n upologism¸n kai upologism¸nklÐsewn lambˆnei q¸ra.Epiplèon axiologeÐtai h genÐkeush tou algìrijmou ekpaÐdeushc NMSP2 qrhsimopoi¸ntacèna pragmatikì prìblhma taxinìmhshc karkinik¸n kuttˆrwn egkefˆlou apìdedomèna pou proèrqontai apì to Tm ma PajologÐac tou PanepisthmiakoÔ NosokomeÐouPatr¸n kai apì to Tm ma PajologÐac tou GenikoÔ AntikarkinikoÔ NosokomeÐouPeiraiˆ Metaxˆc. Ta apotelèsmata pou epitugqˆnontai me thn ekpaÐdeush tou poluepÐpedouemprìsjia trofodotoÔmenou teqnhtoÔ neurwnikoÔ diktÔou me ton proteinìmenoalgìrijmo sugkrÐnontai me autˆ pou epitugqˆnontai apì thn ekpaÐdeush enìc pijanotikoÔteqnhtoÔ neurwnikoÔ diktÔou [42].5.3.1 Apokleistikì-EITEGia to prìblhma tou apokleistikoÔ-EITE (blèpe Parˆrthma A') qrhsimopoi same èna2 − 2 − 1 teqnhtì neurwnikì dÐktuo (6 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn tim 0.1, antÐ thc proepilegmènhc tim c 0.01, ètsi ¸ste na epitaqunjeÐh sÔgklish touc, afoÔ me thn proepilegmènh tim sugklÐnoun polÔ argˆ se autì toprìblhma. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 5.1.O algìrijmoc BPM apèdwse kalÔtera ìson aforˆ to posostì epituqÐac se sqèshme ton algìrijmo BP, allˆ qreiˆsthke perissìterec epanal yeic. O algìrijmoc ABP,eÐqe to Ðdio posostì epituqÐac me ton algìrijmo BP allˆ eÐqe shmantikˆ mikrìtero

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 123Algìrijmoc min max µ σ EpituqÐaBP 28 992 94.5329 114.441 74.5%BPM 23 905 111.821 136.753 76.1%ABP 18 835 46.5517 73.1005 74.5%NMSP1 28 991 133.136 170.856 87.3%NMSP2 16 993 115.201 173.146 90.9%PÐnakac 5.1: Sugkritikˆ apotelèsmata gia to prìblhma tou apokleistikoÔ-EITEmèso arijmì epanal yewn. O algìrijmoc NMSP1 apèdwse arketˆ kalˆ èqontac polÔkalÔtero posostì epituqÐac se sqèsh me touc klasikoÔc algorÐjmouc ekpaÐdeushcallˆ tan dapanhrìteroc. O algìrijmoc ekpaÐdeushc NMSP2 parousiˆsthke pio beltiwmènocapì ton algìrijmo NMSP1 tìso se posostì epituqÐac ìso kai se taqÔthtasÔgklishc.5.3.2 IsotimÐa twn 3-bitGia to prìblhma thc isotimÐac twn 3-bit (blèpe Parˆrthma A') qrhsimopoi same èna3 − 2 − 1 teqnhtì neurwnikì dÐktuo (8 bˆrh kai 3 merolhyÐec). To krufì epÐpedotou neurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me sunˆrthsh energopoÐhshc thnuperbolik efaptomènh, en¸ to epÐpedo exìdou eÐnai basismèno se neur¸nec me sunˆrthshenergopoÐhshc thn grammik sunˆrthsh. H sunj kh termatismoÔ eÐnai E ≤ 0.01kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn tim 0.1, antÐ thc proepilegmènhc tim c 0.01, ètsi ¸ste na epitaqunjeÐh sÔgklish touc, afoÔ me thn proepilegmènh tim sugklÐnoun polÔ argˆ se autì toprìblhma. Ta apotelèsmata twn exomoi¸sewn parousiˆzontai ston PÐnaka 5.2.Algìrijmoc min max µ σ EpituqÐaBP - - - - 0.0%BPM 185 996 492.716 197.034 53.5%ABP 431 982 596.344 128.165 47.4%NMSP1 171 986 429.5 175.026 73.8%NMSP2 112 992 323.792 174.252 79.4%PÐnakac 5.2: Sugkritikˆ apotelèsmata gia to prìblhma thc isotimÐac twn 3-bitO algìrijmoc ekpaÐdeushc BP apètuqe na sugklÐnei mèsa sto ìrio twn upologism¸ntwn tim¸n thc sunˆrthshc sfˆlmatoc se ìlec tic prosomoi¸seic. AxÐzei naanaferjeÐ ìti dokimˆsame diaforetikèc timèc gia thn tim thc paramètrou tou rujmoÔekpaÐdeushc, allˆ to apotèlesma tan to Ðdio. O algìrijmoc BPM apèdwse arketˆkalˆ, en¸ tan kalÔteroc tìso se posostì epituqÐac ìso kai se upologismoÔc tim¸n

124 Neoi Algorijmoi Ekpaideushc TNDthc sunˆrthshc sfˆlmatoc (kai upologismoÔc klÐsewc), anaforikˆ me ton algìrijmoekpaÐdeushc ABP. O algìrijmoc NMSP1 apèdwse exairetikˆ èqontac polÔ kalÔteroposostì epituqÐac se sqèsh me touc klasikoÔc algorÐjmouc ekpaÐdeushc en¸ tantaqÔteroc apì ton algìrijmo BPM pou parousÐase thn taqÔterh sÔgklish apì toucklasikoÔc algorÐjmouc. O algìrijmoc ekpaÐdeushc NMSP2 parousiˆsthke pio beltiwmènocapì ton algìrijmo NMSP1 tìso se posostì epituqÐac ìso kai se taqÔthtasÔgklishc mei¸nontac drastikˆ ton upologismì twn tim¸n thc sunˆrthshc sfˆlmatockai thc klÐshc thc.5.3.3 Anagn¸rish KefalaÐwn GrammˆtwnGia to prìblhma thc anagn¸rishc kefalaÐwn grammˆtwn thc Agglik c alfab tou (blèpeParˆrthma A') qrhsimopoi same èna 35 − 30 − 26 teqnhtì neurwnikì dÐktuo (1830bˆrh kai 56 merolhyÐec). To krufì epÐpedo tou neurwnikoÔ diktÔou eÐnai basismèno seneur¸nec me logistik sunˆrthsh energopoÐhshc, en¸ to epÐpedo exìdou eÐnai basismènose neur¸nec grammik c sunˆrthshc energopoÐhshc. H sunj kh termatismoÔ eÐnaiE ≤ 0.1 kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoceÐnai 2000. Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìcekpaÐdeushc orÐsthke sthn proepilegmènh tim 0.01. H arqikopoÐhsh twn bar¸n ègineme thn teqnik twn Nguyen kai Widrow, allˆ sth sunèqeia ta bˆrh metaxÔ tou krufoÔepipèdou kai tou epipèdou thc exìdou pollaplasiˆsthkan me 0.01. Ta apotelèsmatatwn exomoi¸sewn parousiˆzontai ston PÐnaka 5.3.Algìrijmoc min max µ σ EpituqÐaBP 1100 1999 1548.22 197.517 75.6%BPM 1246 1995 1579.5 175.82 4.8%ABP 1234 1999 1785.87 156.965 36.1%NMSP1 406 1210 707.732 131.425 100.0%NMSP2 288 801 486.909 84.5928 100.0%PÐnakac 5.3: Sugkritikˆ apotelèsmata gia to prìblhma thc anagn¸rishc kefalaÐwngrammˆtwnApì touc klasikoÔc algìrijmouc ekpaÐdeushc, o BP parousiˆzei thn kalÔterh e-pÐdosh tìso se posostì epituqÐac ìso kai se upologismoÔc thc tim c thc sunˆrthshcsfˆlmatoc. Eidikˆ h BPM parousiˆzei polÔ mikrì posostì epituqÐac. O algìrijmocNMSP1 parousiˆzei exairetikì posostì epituqÐac, afoÔ sugklÐnei se ìlec tic prosomoi¸seic.Epiplèon h apìdosh tou se sunarthsiakoÔc upologismoÔc kai upologismoÔcklÐsewc eÐnai exairetik . O algìrijmoc NMSP2, epÐshc sugklÐnei se ìlec tic prosomoi¸seic,allˆ parousiˆzei kalÔterh epÐdosh ìson aforˆ thn taqÔthta sÔgklishc.AxÐzei na shmeiwjeÐ ìti o mègistoc arijmìc sunarthsiak¸n upologism¸n pou qreiˆsthkeo algìrijmoc ekpaÐdeushc NMSP2 eÐnai mikrìteroc apì 1000 to opoÐo shmaÐneiìti gia autìn ton algìrijmo eÐnai dunatì na mei¸soume to ìrio twn sunarthsiak¸n

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 125upologism¸n qwrÐc na ephreasteÐ to posostì epituqÐac. Se aut thn perÐptwsh ìmwc,kanènac apì touc klasikoÔc algìrijmouc den ja sunèkline, afoÔ o elˆqistoc arijmìcupologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai megalÔteroc apì 1000 en¸ oalgìrijmoc NMSP1 ja eÐqe mikrìtero posostì epituqÐac.5.3.4 Prosèggish SuneqoÔc Trigwnometrik c SunˆrthshcGia to prìblhma thc prosèggishc miac suneqoÔc trigwnometrik c sunˆrthshc (blèpeParˆrthma A') qrhsimopoi same èna 1 − 10 − 1 teqnhtì neurwnikì dÐktuo (20 bˆrh kai11 merolhyÐec). To krufì epÐpedo tou neurwnikoÔ diktÔou eÐnai basismèno se neur¸necme logistik sunˆrthsh energopoÐhshc, en¸ to epÐpedo exìdou eÐnai basismèno seneur¸nec grammik c sunˆrthshc energopoÐhshc. H sunj kh termatismoÔ eÐnai E ≤ 0.1kai o mègistoc arijmìc upologism¸n thc tim c thc sunˆrthshc sfˆlmatoc eÐnai 1000.Gia tic mejìdouc stajeroÔ rujmoÔ ekpaÐdeushc (BP kai BPM) o rujmìc ekpaÐdeushcorÐsthke sthn proepilegmènh tim 0.01. Ta apotelèsmata twn exomoi¸sewn parousiˆzontaiston PÐnaka 5.4.Algìrijmoc min max µ σ EpituqÐaBP 398 994 767.359 154.092 7.8%BPM 389 995 760.506 157.334 7.7%ABP 116 999 610.51 217.033 29.0%NMSP1 112 998 486.212 179.492 70.4%NMSP2 84 983 343.841 168.9 76.9%PÐnakac 5.4: Sugkritikˆ apotelèsmata gia to prìblhma thc prosèggishc suneqoÔctrigwnometrik c sunˆrthshc.Se autì to prìblhma, o algìrijmoc ekpaÐdeushc ABP eÐqe thn kalÔterh epÐdoshmetaxÔ twn klasik¸n algorÐjmwn ekpaÐdeushc se posostì epituqÐac kai se mèso ìroupologism¸n twn tim¸n thc sunˆrthshc sfˆlmatoc. Wstìso to posostì epituqÐac toutan qamhlì se sqèsh me to posostì epituqÐac twn algorÐjmwn NMSP1 kai NMSP2.O algìrijmoc NMSP1 apèdwse polÔ kalˆ ìson aforˆ to posostì epituqÐac afoÔ hdiaforˆ tou se posostì epituqÐac apì ton algìrijmo ABP tan pˆnw apì 40%. Epiplèon,eÐqe polÔ mikrìtero kìstoc se sunarthsiakoÔc upologismoÔc kai upologismoÔcklÐsewc. O algìrijmoc ekpaÐdeushc NMSP2 parousÐase to kalÔtero posostì epituqÐacsto parˆdeigma autì. EpÐshc, parousÐase mikrìtero kìstoc se sunarthsiakoÔcupologismoÔc katˆ mèso ìro apì ton NMSP1 en¸ o mèsoc ìroc twn upologism¸nklÐsewc tan arketˆ mikrìteroc.

126 Neoi Algorijmoi Ekpaideushc TND5.3.5 Taxinìmhsh Karkinik¸n Kuttˆrwn EgkefˆlouH taxinìmhsh twn ìgkwn egkefˆlou eÐnai èna krÐsimo b ma gia ton sqediasmì thcagwg c kai thc antimet¸pish thc asjèneiac. Ta astrokut¸mata tou egkefˆlou jewreÐtaiapì tic pio epijetikèc kai dÔskola iˆsimec morfèc karkÐnou. Gia na apofasisteÐo bajmìc thc karkinik c anwmalÐac, pajolìgoi exetˆzoun tic bioyÐec me mikroskìpio.SÔmfwna me to sÔsthma tou POU (Pagkìsmio Organismì UgeÐac), klinikoÐ iatroÐtaxinomoÔn ta astrokut¸mata se duo kathgorÐec, qamhloÔ kai uyhloÔ bajmoÔ neoplˆsmata.Wstìso, h diadikasÐa exètashc èqei apodeiqjeÐ ìti eÐnai upokeimenik kai ìtiexartˆtai se megˆlo bajmì apì thn empeirÐa kai tic ikanìthtec tou eidikoÔ. Prokeimènouna dieukolunjoÔn oi pajolìgoi ¸ste na mporoÔn na pˆroun pio antikeimenikècapofˆseic, eis qjhsan upologistikˆ diagnwstikˆ sust mata, ta opoÐa ìmwc sunepikouroÔntouc eidikoÔc kai den touc antikajistoÔn.Pollˆ upologistikˆ sust mata taxinìmhshc ta opoÐa basÐzontai sta dèntra apofˆsewn,ta emprìsjia trofodotoÔmena neurwnikˆ dÐktua, thn diakritik anˆlush, thnasaf logik , klp èqoun protajeÐ sth bibliografÐa. Wstìso, ta sust mata autˆ eÐnaipolÔ dÔskolo na enswmatwjoÔn sthn kajhmerin klinik diˆgnwsh eˆn den qrhsimopoioÔnkˆpoia kajierwmèna prìtupa ìpwc to sq ma taxinìmhshc tou POU kai thndiadikasÐa qr¸shc AimatoxulÐnhc - EosÐnhc (Hematoxylin-Eosin H&E) [67].Sto pragmatikì autì prìblhma ekpaÐdeushc, proetoimˆsthkan 140 bioyÐec akolouj¸ntacto prwtìkollo thc qr¸shc AimatoxulÐnhc - EosÐnhc. Oi bioyÐec proèrqontai a-pì to Tm ma PajologÐac tou PanepisthmiakoÔ NosokomeÐou Patr¸n kai apì to Tm maPajologÐac tou GenikoÔ AntikarkinikoÔ NosokomeÐou Peiraiˆ Metaxˆc. H taxinìmhshpragmatopoi jhke apì dÔo pajolìgouc, pou taxinìmhsan touc karkinikoÔc ìgkoucwc qamhloÔ kai uyhloÔ bajmoÔ sÔmfwna me ton sÔsthma tou POU. 61 peript¸seicqarakthrÐsthkan wc qamhloÔ bajmoÔ kai 79 wc uyhloÔ bajmoÔ. Qrhsimopoi¸ntacèna eidikì fwtografikì mikroskìpio, pou perigrˆfetai sthn ergasÐa [28], pˆrjhkanfwtografÐec apì perioqèc prokajorismènec apì eidikoÔc (Eikìna 5.1). Sth sunèqeiaefarmìsthke ènac algìrijmoc tmhmatopoÐhshc o opoÐoc diaqwrÐzei ton pur na apì tonperibˆllwn istì ètsi ¸ste na kwdikopoihjeÐ o kako jhc ìgkoc se èna sÔnolo apì 40posotikˆ qarakthristikˆ tou pur na. Ta qarakthristikˆ tou pur na èqei apodeiqjeÐìti eÐnai isquroÐ perigrafeÐc tou bajmoÔ thc karkinik c anwmalÐac [19].Ta qarakthristikˆ pou ex qjhsan mporoÔn na kathgoriopoihjoÔn se èna sÔnolomorfologik¸n qarakthristik¸n pou perigrˆfoun to mègejoc kai to sq ma tou purna kai se èna sÔnolo apì qarakthristikˆ uf c pou kwdikopoioÔn thn katanom kaiorgˆnwsh thc qrwmatÐnhc mèsa ston pur na. Ta morfologikˆ qarakthristikˆ perilambˆnounmetr seic tou embadoÔ, thc stroggulìthtac kai thc koilìthtac. Gia kˆje ènaapì autˆ ta qarakthristikˆ upologÐsthke h mèsh tim , h tupik apìklish, to eÔroc, hloxìthta, h kÔrtwsh kai h mègisth tim . Ta qarakthristikˆ uf c upologÐsthkan apìto istìgramma tou DNA, thn sunÔparxh kai touc pÐnakec diˆrkeiac ektèleshc. Miapio leptomer c perigraf tou upologismoÔ aut¸n twn qarakthristik¸n dÐnetai stic

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 127Sq ma 5.1: ParadeÐgmata qamhloÔ (epˆnw) kai uyhloÔ bajmoÔ (kˆtw) egkefalik¸nastrokutwmˆtwn mazÐ me thn tmhmatopoihmènh eikìna.ergasÐec [82, 83].Sthn perÐptws mac h diˆstash twn dianusmˆtwn twn qarakthristik¸n eÐnai polÔmegˆlh (40 qarakthristikˆ). Wstìso, oi sunist¸sec tou dianÔsmatoc eÐnai uyhlˆ susqetismènec,afoÔ kˆpoia qarakthristikˆ ìpwc to embadìn tou pur na kai h perÐmetroctou eÐnai isqurˆ susqetismèna. Epomènwc eÐnai qr simo na elatt¸soume th diˆstashtou dianÔsmatoc qarakthristik¸n afoÔ ètsi ja meiwjeÐ shmantikˆ kai o upologistikìcqrìnoc. Mia apotelesmatik diadikasÐa pou pragmatopoieÐ aut n th diergasÐa eÐnai hAnˆlush KÔriwn Sunistws¸n (PCA). Aut h diadikasÐa èqei treic kÔriec epipt¸seic:1. OrjogwnopoieÐ tic sunist¸sec twn dianusmˆtwn qarakthristik¸n ètsi ¸ste namh susqetÐzontai metaxÔ touc.2. Diatˆssei tic orjogwnopoihmènec sunist¸sec, dhlad thc kÔriec sunist¸sec,ètsi ¸ste autèc me thn megalÔterh diaforopoÐhsh na èrqontai pr¸tec.3. ExaleÐfei autèc tic sunist¸sec pou suneisfèroun elˆqista sthn diaforopoÐhshtwn dedomènwn.Sthn pragmatikìthta, o algìrijmoc upologÐzei katarq n th mèsh tim twn dianusmˆtwnqarakthristik¸n kai sth sunèqeia afaireÐ apì autˆ thn mèsh tim . Katìpin,upologÐzetai o pÐnakac sundiakÔmanshc kai entopÐzontai oi idiotimèc kai ta idiodianÔsmatˆtou. Ta idiodianÔsmata pou antistoiqoÔn stic M megalÔterec idiotimèc diathroÔntaikai ta dianÔsmata qarakthristik¸n probˆllontai pˆnw sta idiodianÔsmata autˆ gia naupologÐsoun tic sunist¸sec twn metasqhmatismènwn dianusmˆtwn tou M-diˆstatouq¸rou. Gia to prìblhma autì, qrhsimopoi same thn Anˆlush KÔriwn Sunistws¸n giana elatt¸soume thn diˆstash tou q¸rou twn qarakthristik¸n apì 40 se 11. Kajènaapì autˆ ta nèa 11 qarakthristikˆ upologÐsthke wc to ˆjroisma twn kurÐwn sunistws¸npou proèkuyan apì kˆje prwtìtupo qarakthristikì (PÐnakac 1 sthn ergasÐa

128 Neoi Algorijmoi Ekpaideushc TND[42]). En suneqeÐa, ta dianÔsmata eisìdou metatrèpontai apì ta arqikˆ 40 qarakthristikˆsta 11 pou proèrqontai apì thn Anˆlush KÔriwn Sunistws¸n. Ta dianÔsmataautˆ eÐnai t¸ra mh susqetismèna.Gia thn ekpaÐdeush twn teqnht¸n neurwnik¸n diktÔwn (emprìsjia trofodotoÔmenwnkai pijanotik¸n) qrhsimopoi jhkan 91 prìtupa (41 qamhloÔ kai 50 uyhloÔbajmoÔ). Gia thn exètash thc genÐkeushc qrhsimopoi jhkan ta upìloipa prìtupa (20qamhloÔ kai 29 uyhloÔ bajmoÔ).Gia to prìblhma autì qrhsimopoi same dÔo teqnhtˆ neurwnikˆ dÐktua. To pr¸totan èna emprìsjia trofodotoÔmeno neurwnikì dÐktuo to opoÐo ekpaideÔthke me thnmèjodo NMSP2. Dokimˆsthkan pollèc arqitektonikèc diktÔwn afoÔ den tan gnwstìek twn protèrwn poia arqitektonik ja odhgoÔse sta kalÔtera apotelèsmata.H arqitektonik pou qrhsimopoi jhke tan tri¸n epipèdwn kai apoteloÔntan apì toepÐpedo eisìdou (11 eÐsodoi), to krufì epÐpedo me N neur¸nec ìpou N = 5, . . . , 21kai to epÐpedo exìdou me 2 neur¸nec. To krufì epÐpedo kai to epÐpedo exìdou touneurwnikoÔ diktÔou eÐnai basismèno se neur¸nec me logistik sunˆrthsh energopoÐhshc.H ekpaÐdeush tou teqnhtoÔ neurwnikoÔ diktÔou oloklhr¸netai ìtan to dÐktuoèqei sfˆlma taxinìmhshc tou sunìlou ekpaÐdeushc ligìtero apì 1%.O deÔteroc taxinomht c tan èna pijanotikì neurwnikì dÐktuo. To dÐktuo apoteleÐtaiapì tèssera epÐpeda. To pr¸to epÐpedo eÐnai to epÐpedo eisìdou to opoÐo èqeièna kìmbo gia kˆje sunist¸sa tou dianÔsmatoc eisìdou. To deÔtero epÐpedo eÐnai toepÐpedo protÔpwn to opoÐo perièqei èna kìmbo gia kˆje prìtupo ekpaÐdeushc. Kˆjekìmboc protÔpou sqhmatÐzei to ginìmeno tou dianÔsmatoc twn bar¸n me to dojènparˆdeigma proc taxinìmhsh, ìpou ta bˆrh pou eisèrqontai ston kìmbo eÐnai apì ènasugkekrimèno parˆdeigma. Sth sunèqeia to ginìmeno trofodoteÐtai se mia sunˆrthshenergopoÐhshc pou akoloujeÐ thn kanonik katanomk(x) =1(2π) d 2 σ d exp(−∥x∥2 2σ 2 )kai eisèrqetai sto epÐpedo ˆjroishc to opoÐo lambˆnei tic exìdouc apì to epÐpedoprotÔpwn pou sqetÐzontai me mia tˆxh. To epÐpedo exìdou èqei tìsouc kìmbouc ìseceÐnai kai oi tˆxeic.H kalÔterh apìdosh epiteÔqjhke qrhsimopoi¸ntac to emprìsjia trofodotoÔmenoneurwnikì dÐktuo pou ekpaideÔthke me thn mèjodo NMSP2 me 7 neur¸nec sto krufìepÐpedo, to opoÐo dièkrine qamhloÔ bajmoÔ apì uyhloÔ bajmoÔ ìgkouc me akrÐbeia92% (PÐnakac 5.5). H euaisjhsÐa kai h exeidÐkeush kumˆnjhke sto 93.1% kai 90.5%,antÐstoiqa. To pijanotikì neurwnikì dÐktuo eÐqe qamhlìtera posostˆ taxinìmhshc(83.8% exeidÐkeush, 91.5% euaisjhsÐa kai 89.9% sunolik akrÐbeia). Autì mporeÐ naexhghjeÐ apì thn polÔplokh pijanotik katanom pou akoloujoÔn ta dedomèna poume to pijanotikì neurwnikì dÐktuo proseggÐzetai qrhsimopoi¸ntac èna pur na pouakoloujeÐ thn kanonik katanom . Wstìso, axÐzei na anaferjeÐ ìti to pijanotikì

Mh Monotonoi Algorijmoi Ekpaideushc Suzugwn Klisewn 129neurwnikì dÐktuo ekpaideÔthke taqÔtera apì to emprìsjia trofodotoÔmeno neurwnikìdÐktuo, en¸ den qreiˆsthkan dokimèc gia thn topologÐa tou.Algìrijmoc ExeidÐkeush EuaisjhsÐa Sunolik AkrÐbeiaMLP 90.5% 93.1% 92.0%PNN 83.3% 91.5% 89.9%PÐnakac 5.5: Sugkritikˆ apotelèsmata gia to prìblhma thc Taxinìmhshc Karkinik¸nKuttˆrwn Egkefˆlou metaxÔ tou emprìsjia trofodotoÔmenou neurwnikoÔ diktÔou me7 neur¸nec (MLP) sto krufì epÐpedo kai tou pijanotikoÔ neurwnikoÔ diktÔou (PNN).Sto shmeÐo autì, axÐzei na anaferjeÐ ìti to kalÔtero apotèlesma taxinìmhshc deneÐnai autì to opoÐo parousiˆzei thn bèltisth sunolik akrÐbeia. To diagnwstikì lˆjoceÐnai pio shmantikì ìtan taxinomeÐte lanjasmèna ènac karkinikìc ìgkoc uyhloÔbajmoÔ wc qamhloÔ bajmoÔ, afoÔ tètoiou eÐdouc lˆjh odhgoÔn se lanjasmènh agwgme sobarèc sunèpeiec sthn ugeÐa tou asjen . Apì thn ˆllh meriˆ h lanjasmènh taxinìmhshtwn karkinik¸n ìgkwn qamhloÔ bajmoÔ den eÐnai tìso shmantik . Epomènwc,bèltisto apotèlesma taxinìmhshc jewreÐtai h megistopoÐhsh thc akrÐbeiac anagn¸rishctwn karkinik¸n ìgkwn uyhloÔ bajmoÔ me to qamhlìtero dunatì sfˆlma stouckarkinikoÔc ìgkouc qamhloÔ bajmoÔ [39]. SÔmfwna me aut th logik jewr jhke toemprìsjia trofodotoÔmeno neurwnikì dÐktuo me 7 neur¸nec sto krufì epÐpedo wc hbèltisth arqitektonik afoÔ eÐqe ton bèltisto sunduasmì exeidÐkeushc kai euaisjhsÐac(PÐnakac 5.5). H kalÔterh euaisjhsÐa epiteÔqjhke me to emprìsjia trofodotoÔmenoneurwnikì dÐktuo me 8 neur¸nec sto krufì epÐpedo (96.5%), allˆ aut h arqitektonikden epilèqjhke wc bèltisth afoÔ to sqetikì kìstoc sthn taxinìmhsh twn karkinik¸nìgkwn qamhloÔ bajmoÔ tan shmantikì (85.7%). H Eikìna 5.2 parousiˆzei thn apìdoshtou emprìsjia trofodotoÔmenou neurwnikoÔ diktÔou gia diˆforec arqitektonikèc.H Ðdia strathgik akolouj jhke kai gia to pijanotikì neurwnikì dÐktuo. H kalÔterhapìdosh (PÐnakac 5.5) epiteÔqjhke gia σ = 0.05.5.4 SumperˆsmataSe autì to kefˆlaio parousiˆsthkan dÔo nèoi algìrijmoi ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn pou basÐzontai sthn mèjodo suzug¸n klÐsewn tou Perry. Pio sugkekrimèna,sunduˆsthke o klimakwtìc algìrijmoc suzug¸n klÐsewn tou Perry me parˆmetroklimˆkwshc to b ma twn Barzilai kai Borwein me mia mh monìtonh strathgikgrammik c anaz thshc. EpÐshc oi algìrijmoi ekpaÐdeushc qrhsimopoioÔn mia teqnikepanekkÐnhshc h opoÐa exasfalÐzei kai thn idiìthta olik c sÔgklishc. H diaforˆ twndÔo proteinìmenwn algorÐjmwn eÐnai h parˆmetroc tou arqikoÔ rujmoÔ ekpaÐdeushcpou trofodoteÐtai sthn diadikasÐa thc grammik c anaz thshc. Ston pr¸to algìrijmoo arqikìc rujmìc ekpaÐdeushc prosarmìzetai autìmata se kˆje epanˆlhyh sÔmfwna

130 Neoi Algorijmoi Ekpaideushc TNDSq ma 5.2: KalÔterec epidìseic gia to emprìsjia trofodotoÔmeno neurwnikì dÐktuogia diaforetikì arijmì neur¸nwn sto krufì epÐpedo.me èna kleistì tÔpo pou protˆjhke stic ergasÐec [77], [80] en¸ ston deÔtero algìrijmoqrhsimopoieÐtai o rujmìc ekpaÐdeushc pou protˆjhke sthn ergasÐa [41].Ta peiramatikˆ apotelèsmata èdeixan ìti oi proteinìmenoi algìrijmoi eÐnai apodotikìteroiapì touc klasikoÔc algorÐjmouc ekpaÐdeushc, en¸ h tropopoÐhsh tou arqikoÔrujmoÔ ekpaÐdeushc pou protˆjhke sthn ergasÐa [41] beltÐwse peraitèrw thn apìdoshtou mh monìtonou klimakwtoÔ algorÐjmou ekpaÐdeushc suzug¸n klÐsewn touPerry. Wstìso, h mh monìtonh teqnik grammik c anaz thshc pou qrhsimopoi jhkestic ergasÐec autèc eÐnai sqetikˆ apl . PisteÔoume ìti oi sugkekrimènoi algìrijmoimporeÐ na beltiwjoÔn peraitèrw qrhsimopoi¸ntac pio exeligmènec mh monìtonec teqnikècgrammik c anaz thshc ìpou o rujmìc ekpaÐdeushc den ja epitugqˆnetai apì thnapl upodiaÐresh tou arqikoÔ rujmoÔ ekpaÐdeushc.

Mèroc IIIPararthmata Bibliografia131

Pararthma AþProbl mata EkpaÐdeushcNeurwnik¸n DiktÔwnSe autì to parˆrthma parousiˆzontai ta probl mata ekpaÐdeushc teqnht¸n neurwnik¸ndiktÔwn pou qrhsimopoi jhkan se aut th diatrib gia thn axiolìghsh twn nèwnalgorÐjmwn ekpaÐdeushc wc proc thn taqÔthta, thn axiopistÐa kai thn genÐkeush.Aþ.1Apokleistikì-EITE (XOR)To Apokleistikì-EITE eÐnai èna polÔ dÔskolo prìblhma pou qrhsimopoieÐtai eurèwcgia thn axiolìghsh algorÐjmwn ekpaÐdeushc teqnht¸n neurwnik¸n diktÔwn. H sunˆrthshtou ApokleistikoÔ-EITE apeikonÐzei dÔo duadikèc eisìdouc se mia duadik èxodo.Sto Sq ma Aþ.1 apeikonÐzontai sto epÐpedo ta prìtupa ekpaÐdeushc tou ApokleistikoÔ-EITE. 'Opwc faÐnetai apì to Sq ma Aþ.1, h sunˆrthsh tou ApokleistikoÔ-EITE deneÐnai grammikˆ diaqwrÐsimh. Gi' autì to lìgo apaiteÐtai toulˆqiston èna krufì epÐpedoneur¸nwn gia thn epÐlush tou probl matoc. To prìblhma autì eÐnai polÔ euaÐsjhtosthn epilog twn arqik¸n bar¸n, kai h epifˆneia sfˆlmatoc parousiˆzei pollˆ topikˆelˆqista me sqetikˆ megˆlec sunarthsiakèc timèc.133

134 Pararthmata10(0,1) (1,1)Είσοδο̋(0, 0)(0, 1)(1, 0)(1, 1)Έξοδο̋0110(0,0) (1,0)Sq ma Aþ.1: To prìblhma tou ApokleistikoÔ-EITE sto epÐpedo.kajarˆ ìti ta prìtupa ekpaÐdeushc den eÐnai grammikˆ diaqwrÐsima.Apì to sq ma faÐnetaiAþ.2IsotimÐa twn 3-bitTo prìblhma thc IsotimÐac twn 3-bit mporeÐ na jewrhjeÐ wc mia genÐkeush tou problmatoc tou apokleistikoÔ-EITE, to opoÐo ìmwc eÐnai polÔ pio dÔskolo. H duadiksunˆrthsh tou probl matoc apeikonÐzei treic duadikèc eisìdouc se mia duadik èxodo.H tim thc exìdou eÐnai 1 ìtan o arijmìc twn bits me tim 1 sthn eÐsodo eÐnai perittìc,kai 0 se kˆje ˆllh perÐptwsh (peritt isotimÐa). H epifˆneia sfˆlmatoc tou probl -matoc autoÔ parousiˆzei pollˆ topikˆ elˆqista ìpou eÐnai eÔkolo na pagideutoÔn oialgìrijmoi ekpaÐdeushc. Sto PÐnaka Aþ.1 apeikonÐzontai oi 8 eisìdoi kai oi antÐstoiqecexìdoi tou probl matoc.

Problhmata Ekpaideushc Neurwnikwn Diktuwn 135EÐsodoc 'Exodoc(0, 0, 0) 0(0, 0, 1) 1(0, 1, 0) 1(0, 1, 1) 0(1, 0, 0) 1(1, 0, 1) 0(1, 1, 0) 0(1, 1, 1) 1PÐnakac Aþ.1: Ta prìtupa ekpaÐdeushc thc IsotimÐac twn 3-bit.Aþ.3Anagn¸rish KefalaÐwn GrammˆtwnTo prìblhma thc anagn¸rishc twn kefalaÐwn grammˆtwn thc Agglik c alfab touapoteleÐtai apì 26 pÐnakec diastˆsewc 7 × 5. Oi pÐnakec autoÐ èqoun kwdikopoihjeÐme duadikèc timèc kai apoteloÔn thn eÐsodo tou diktÔou. Ta grˆmmata A, B, C kaiD kai oi duadikèc touc kwdikopoi seic faÐnontai sto Sq ma Aþ.2. Oi èxodoi èqounkwdikopoihjeÐ me dianÔsmata 26 duadik¸n stoiqeÐwn. Ta 25 stoiqeÐa tou ekˆstotedianÔsmatoc exìdou èqoun thn duadik tim 0 kai mìno èna èqei thn duadik timh 1 kaibrÐsketai sthn antÐstoiqh jèsh pou èqei to grˆmma sthn Agglik alfˆbhto.100 1 0 01 1 101 1 1 011010101110001110101011000011 1 1111 1 11110 0 0011100 011100 001110001111000111100011110111011 1 111 1 1 011 1 111 1 1 0111000111100011110001111000111 1 111 1 101110 0 0111100 0111100 0111100011110001111111110111111110Sq ma Aþ.2: H kwdikopoÐhsh twn grammˆtwn A, B, C kai D kai oi antÐstoiqec duadikèckwdikopoi seic.

f136 PararthmataAþ.4Prosèggish SuneqoÔc Trigwnometrik c SunˆrthshcTo prìblhma thc prosèggishc miac sunˆrthshc eÐnai èna apì ta basikˆ probl matapou kaloÔntai na lÔsoun ta teqnhtˆ neurwnikˆ dÐktua. Sthn perÐptwsh aut to teqnhtìneurwnikì dÐktuo kaleÐtai na proseggÐsei thn suneq trigwnometrik sunˆrthshf(x) = sin(x) cos(2x). Ta prìtupa eisìdou eÐnai 20 isapèqousec timèc sto diˆsthma[0, 2π]. H èxodoc apoteleÐtai apì thn tim f(x) gia thn ekˆstote eÐsodo x. Sto Sq maAþ.3 faÐnetai h grafik parˆstash thc sunˆrthshc f kai ta prìtupa ekpaÐdeushc.10.80.60.40.20−0.2−0.4−0.6−0.8−10 pi/2 pi 3pi/2 2pixSq ma Aþ.3: H grafik parˆstash thc f(x) = sin(x) cos(2x) sto [0, 2π]. Ta 20 prìtupaekpaÐdeushc apeikonÐzontai pˆnw sth grafik parˆstash me kìkkinec koukÐdec.Aþ.5N−M−N Kwdikopoiht c/Apokwdikopoiht cSe autì to prìblhma to teqnhtì neurwnikì dÐktuo dèqetai M diaforetikˆ prìtupaeisìdou m kouc M bit ìpou to kajèna èqei èna mìno bit me thn tim 1 kai ìla taupìloipa me thn tim 0. H èxodoc tou diktÔou eÐnai akrib¸c h Ðdia me thn eÐsodo.AfoÔ oi plhroforÐec thc eisìdou pernoÔn apì to krufì epÐpedo twn M neur¸nwn(ìpou M = log 2 N), to dÐktuo prèpei na kataskeuˆsei mia monadik kwdikopoÐhshgia kˆje èna apì ta N prìtupa stouc M krufoÔc neur¸nec kai bˆrh tètoia ¸stena eÐnai dunat h kwdikopoÐhsh kai h apokwdikopoÐhsh. To prìblhma autì, an kai

Problhmata Ekpaideushc Neurwnikwn Diktuwn 137faÐnetai aplì, prosomoiˆzei pragmatikˆ probl mata taxinìmhshc ìpou mikrèc allagècstic eisìdouc èqoun wc apotèlesma mikrèc allagèc stic exìdouc. DÔo paradeÐgmataN − M − N Kwdikopoiht /Apokwdikopoiht gia N = 4 kai N = 8 dÐnontai stoucPÐnakec Aþ.2 kai Aþ.3, antÐstoiqa.EÐsodoc 'Exodoc(1, 0, 0, 0) (1, 0, 0, 0)(0, 1, 0, 0) (0, 1, 0, 0)(0, 0, 1, 0) (0, 0, 1, 0)(0, 0, 0, 1) (0, 0, 0, 1)PÐnakac Aþ.2: Ta prìtupa ekpaÐdeushc 4 − 2 − 4 Kwdikopoiht /Apokwdikopoiht .EÐsodoc'Exodoc(1, 0, 0, 0, 0, 0, 0, 0) (1, 0, 0, 0, 0, 0, 0, 0)(0, 1, 0, 0, 0, 0, 0, 0) (0, 1, 0, 0, 0, 0, 0, 0)(0, 0, 1, 0, 0, 0, 0, 0) (0, 0, 1, 0, 0, 0, 0, 0)(0, 0, 0, 1, 0, 0, 0, 0) (0, 0, 0, 1, 0, 0, 0, 0)(0, 0, 0, 0, 1, 0, 0, 0) (0, 0, 0, 0, 1, 0, 0, 0)(0, 0, 0, 0, 0, 1, 0, 0) (0, 0, 0, 0, 0, 1, 0, 0)(0, 0, 0, 0, 0, 0, 1, 0) (0, 0, 0, 0, 0, 0, 1, 0)(0, 0, 0, 0, 0, 0, 0, 1) (0, 0, 0, 0, 0, 0, 0, 1)PÐnakac Aþ.3: Ta prìtupa ekpaÐdeushc 8 − 3 − 8 Kwdikopoiht /Apokwdikopoiht .Aþ.6Anagn¸rish Arijm¸nTo prìblhma thc anagn¸rishc twn arijm¸n apoteleÐtai apì 10 pÐnakec diastˆsewc8 × 8. Oi pÐnakec autoÐ èqoun kwdikopoihjeÐ me duadikèc timèc kai apoteloÔn thneÐsodo tou diktÔou. Oi arijmoÐ 0 kai 1 kai oi duadikèc touc kwdikopoi seic faÐnontaisto Sq ma Aþ.4. Oi èxodoi èqoun kwdikopoihjeÐ me dianÔsmata 10 duadik¸n stoiqeÐwn.Ta 10 stoiqeÐa tou ekˆstote dianÔsmatoc exìdou èqoun thn duadik tim 0 kai mìnoèna èqei thn duadik timh 1 kai brÐsketai sthn antÐstoiqh jèsh pou èqei o arijmìc.

138 Pararthmata01111111 1 1 100000 0 0000000 000000000000 1 1 1 11 1 000000 000010001111111000000000 0 0 100100 0 0000010000001111110 1 1 1 10 0 000000 0000100010000001Sq ma Aþ.4: H kwdikopoÐhsh twn arijm¸n 0 kai 1 kai oi antÐstoiqec duadikèc kwdikopoi seic.Aþ.7 Taxinìmhsh twn Fut¸n thc Oikogèneiac I-risTo prìblhma thc taxinìmhshc twn fut¸n thc oikogèneiac Iris [24] apoteleÐtai apì 150deÐgmata, to kajèna apì ta opoÐa èqei tèssera qarakthristikˆ. To sÔnolo perièqeitreic kathgorÐec, dÔo apì tic opoÐec eÐnai grammikˆ mh diaqwrÐsimec. Kˆje kathgorÐaapoteleÐtai apì 50 peript¸seic kai parapèmpoun sta futˆ thc oikogèneiac Iris, Setosa,Versicolor kai Virginica. Stic Eikìnec Aþ.5, Aþ.6 kai Aþ.7 apeikonÐzontai ta tria futˆthc oikogèneiac Iris pou qrhsimopoioÔntai se autì to prìblhma.Sq ma Aþ.5: Iris Setosa.

Problhmata Ekpaideushc Neurwnikwn Diktuwn 139Sq ma Aþ.6: Iris Versicolor.Sq ma Aþ.7: Iris Virginica.Aþ.8Optik Anagn¸rish Qeirìgrafwn Arijm¸nTo prìblhma thc optik c anagn¸rishc qeirìgrafwn arijm¸n eÐnai èna pragmatikì prìblhmataxinìmhshc. Se autì to peÐrama, dhmiourg jhke mia yhfiak bˆsh dedomènwnsullègontac deÐgmata apì 43 anexˆrthtouc suggrafeÐc. Ta deÐgmata pou sullèqjhkanapo 30 suggrafeÐc qrhsimopoi jhkan gia thn ekpaÐdeush en¸ ta upìloipa gia tonèlegqo thc genÐkeushc. To sÔnolo ekpaÐdeushc apoteleÐtai apo 3823 prìtupa, en¸to sÔnolo elègqou thc genÐkeushc apì 1797 prìtupa. Kˆje yhfÐo èqei kwdikopoihjeÐse duadik morf kai èqoun topojethjeÐ se pÐnakec diastˆsewc 32 × 32. Gia naelatt¸soume thn diˆstash tou dianÔsmatoc eisìdou oi pÐnakec diastˆsewc 32 × 32.diairèjhkan se mh epikaluptìmena tm mata diastˆsewc 4 × 4 kai metr jhke oarijmìctwn monˆdwn se kˆje tm ma. 'Etsi dhmiourg jhke ènac pÐnakac diˆstasewc 8 × 8.Wstìso, sth sunèqeia ta dianÔsmata eisìdou metatrˆphkan ètsi ¸ste oi sunist¸sectouc na èqoun timèc pou na brÐskontai mèsa sto diˆsthma [0, 1].

140 PararthmataAþ.8.1Anagn¸rish FwnhèntwnO stìqoc autoÔ tou pragmatikoÔ probl matoc taxinìmhshc eÐnai h anagn¸rish enìcfwnhèntoc apì ènan tuqaÐo omilht . Pio sugkekrimèna dhmiourg jhke mia bˆsh dedomènwnsullègontac deÐgmata apì 15 diaforetikoÔc omilhtèc (8 ˆntrec kai 7 gunaÐkec)pou èlegan ta 11 fwn enta 6 forèc (990 deÐgmata). Ta dedomèna qwrÐsthkan se ènasÔnolo ekpaÐdeushc kai se èna sÔnolo elègqou thc genÐkeushc. To sÔnolo ekpaÐdeushcapoteleÐtai apì tèsseric ˆntrec kai tèsseric gunaÐkec omilhtèc (11 × 6 × 8 = 528deÐgmata). To sÔnolo elègqou thc genÐkeushc apoteleÐtai apì tèsseric ˆntrec kaitreic gunaÐkec omilhtèc (11 × 6 × 7 = 462 deÐgmata). Kˆje deÐgma èqei dèka qarakthristikˆme suneqeÐc timèc pou proèrqontai apì ta fasmatikˆ dedomèna (gia perissìterecplhroforÐec gia to prìblhma autì blèpe tic ergasÐec [61] kai [53]).

Pararthma BþDhmosieÔseic - AnaforècDhmosieÔseic[1]. D. G. Sotiropoulos, A. E. Kostopoulos, and T. N. Grapsa. A spectral versionof Perry’s conjugate gradient method for neural network training. In D. T.Tsahalis, editor, 4th GRACM Congress on Computational Mechanics, volume 1,pages 172–179, 2002.[2]. D. G. Sotiropoulos, A. E. Kostopoulos, and T. N. Grapsa. Training neuralnetworks using two-point stepsize gradient methods. In T. E. Simos and Ch.Tsitouras, editors, Proceedings of International Conference of Numerical Analysisand Applied Mathematics 2004 (ICNAAM-2004), pages 356–359, 2004.[3]. A. E. Kostopoulos, D. G. Sotiropoulos, and T. N. Grapsa A new efficientvariable learning rate for Perry’s Spectral conjugate gradient training method.In D. T. Tsahalis, editor, Proceedings of 1st International Conference ”FromScientific Computing to Computational Engineering”, volume 1, pages 66–73,2004[4]. A. E. Kostopoulos, D. Glotsos, P. Spyridonos, G. Nikiforidis, D. G. Sotiropoulos,and T. N. Grapsa. Comparative evaluation of feedforward and probabilisticneural networks for the automatic classification of brain tumours. In D. T.Tsahalis, editor, Proceedings of 1st International Conference ”From ScientificComputing to Computational Engineering”, volume 1, pages 309–315, 2004.[5]. A. E. Kostopoulos, A. P. Happonen, and U. Ruotsalainen. Three-dimensionalalignment of objects in dynamic PET scans using filtered sinusoidal trajectoriesof sinogram. Nuclear Instruments and Methods in Physics Research Section A,569(2):434-439, 2006.141

142 Pararthmata[6]. T. N. Grapsa, G. E. Antonelou, and A.E. Kostopoulos. Perturbed Newtonmethod for Unconstrained Optimization. In E. Gallopoulos, E. Houstis, I.S.Kotsireas, D. Noutsos and M.N. Vrahatis, editors, Proceedings of Conferencein Numerical Analysis, Recent Approaches to Numerical Analysis, Theory, Methodsand Applications (NumAn2007), pages 76–79, 2007.[7]. A. E. Kostopoulos, and T. N. Grapsa. Self-scaled conjugate gradient trainingalgorithms. Neurocomputing, 72:3000–3019, 2009.[8]. A.E. Kostopoulos, G.S. Androulakis, and T.N. Grapsa. A new nonmonotoneNewton’s modification for unconstrained optimization, Technical report 09-05,Department of Mathematics, University of Patras, Patras, Greece, 2009.AnaforècSthn ergasÐa [1][1]. X. Mu, and Y. Zhang. A Modified Gradient-based Backpropagation TrainingMethod for Neural Networks. In The 2009 IEEE International Conferenceon Granular Computing, GrC 2009, GrC 2009, Lushan Mountain, Nanchang,China, 17-19 August 2009, GrC, IEEE, 2009.[2]. X. Mu, and Y. Zhang. Advances in Neural Networks - ISNN 2010, chapter ANew Two-Step Gradient-Based Backpropagation Training Method for NeuralNetworks. Springer Berlin / Heidelberg, Volume 6063, pages 95-101, 2010.[3]. Nazri Mohd Nawi, Meghana R. Ransing, and Rajesh S. Ransing. An ImprovedLearning Algorithm Based on The Broyden-Fletcher-Goldfarb-Shanno (BFGS)Method For Back Propagation Neural Networks. In Proceedings of the SixthInternational Conference on Intelligent Systems Design and Applications (ISDA06), pages 152–157, 2006.[4]. Nazri Mohd Nawi, Rajesh S. Ransing, and Meghana R. Ransing. A NewMethod to Improve the Gradient Based Search Direction to Enhance the ComputationalEfficiency of Back Propagation Based Neural Network Algorithms.In Proceedings of the Second Asia International Conference on Modelling &Simulation, Asia Modelling Symposium (AMS 2008), pages 546-552, 2008.[5]. G. Lappas, R. J. Frank, A. A. Albrecht. A computational study on circuitsize versus circuit depth. International Journal on Artificial Intelligence Tools,15(2):143-161, 2006.

Dhmosieuseic - Anaforec 143[6]. C-C Peng, and G. D. Magoulas. Adaptive Nonmonotone Conjugate GradientTraining Algorithm for Recurrent Neural Networks. In Proceedings of 19th IEE-E International Conference on Tools with Artificial Intelligence (ICTAI 2007),29-31 Oct. 2007, volume 2, pages 374-381, 2007.[7]. C-C Peng, and G. D. Magoulas. Advanced Adaptive Nonmonotone ConjugateGradient Training Algorithm for Recurrent Neural Networks. InternationalJournal on Artificial Intelligence Tools, 17(5):963-984, 2008.[8]. I.E. Livieris, and P. Pintelas. A survey on algorithms for training artificial neuralnetworks, Technical report 08-01, Department of Mathematics, Universityof Patras, Patras, Greece, 2008.[9]. I.E. Livieris, and P. Pintelas. Evaluation of neural network training algorithmsusing biomedical data, Technical report 08-02, Department of Mathematics,University of Patras, Patras, Greece, 2009.[10]. I.E. Livieris, and P. Pintelas. Performance Evaluation of Descent CG Methodsfor Neural Network Training. In Proceedings of 9th Hellinic European Researchon Computer and its Application Conference (HERCMA’09), Athens, Greece,September, 2009.Sthn ergasÐa [3][1]. M. Terzyiska, Y. Todorov and M. Petrov. NMPC with Adaptive LearningRate Scheduling of an Internal Fuzzy-Neural Model. In Proceedings of theInternational Conference ISAC 2006: Modern Trends in Control, InternationalStudy in Automatic Control, Košice, Slovak Republic, 15–30 June 2006, pages289-298, 2006.Sthn ergasÐa [4][1]. E. I. Papageorgiou, P. P. Spyridonos, D. T. Glotsos, C. D. Stylios, P. Ravazoula,G. N. Nikiforidis, P. P. Groumpos. Brain tumor characterization using thesoft computing technique of fuzzy cognitive maps. Applied Soft Computing,8(1):820-828, 2008.Sthn ergasÐa [5][1]. A. P. Happonen, and M. O. Koskinen. Assessment of automated scintigram a-lignment in the stackgram domain for cardiac stress-rest SPECT studies. ComputerizedMedical Imaging and Graphics, 31:595-606, 2007.

144 PararthmataSthn ergasÐa [7][1]. S. Chabaa, A. Zeroual, and J. Antari. Identification and Prediction of InternetTraffic Using Artificial Neural Networks. Journal of Intelligent LearningSystems and Applications, 2:147-155, 2010.[2]. M. Kabsch-Korbutowicz, and M. Kutylowska. Artificial Neural Networks - AViable Option for Predicting Changes in Water Turbidity after Treatment byCoagulation/Ultrafiltration. OCHRONA SRODOWISKA, 32(2):5-20, 2010.[3]. Gao Donglian, and Shan Rui. TR-BFGS Algorithm for Multi-layer Feed-Forward Neural Networks . In Cryptography and Network Security, Data Miningand Knowledge Discovery, E-Commerce & Its Applications and EmbeddedSystems (CDEE), 2010 First ACIS International, 23-24 October 2010, pages143-146, 2010.[4]. Y. Zhang, Z. Dong, L. Wu, and S. Wang. A hybrid method for MRI brain imageclassification. Expert Systems with Applications, 38(8):10049-10053 , 2011.[5]. J. C. F. Wong, and J. L. Xie. Inverse determination of a heat source fromnatural convection in a porous cavity. Computers & Fluids, 52:1-14, 2011.[6]. Jeff C. F. Wong and Bartosz Protas. Application of scaled nonlinear conjugategradientalgorithms to the inverse natural convection problem. OptimizationMethods and Software, iFirst:1-27, 2011.

BibliografÐa[1] H. Akaike On succesive transformation of probability distribution and its applicationto the analysis of the optimum gradient method. Ann. Inst. Statist.Math. Tokyo, 11:1–17, 1959.[2] M. Al Baali Improved Hessian approximations for the limited memory BFGSmethod. Numerical Algorithms, 22:99–112, 1999.[3] A. D. Anastasiadis, G. D. Magoulas and M. N. Vrahatis. New globally convergenttraining scheme based on the resilient propagation algorithm. Neurocomputing,64:253–270, 2005.[4] A. D. Anastasiadis, G. D. Magoulas and M. N. Vrahatis. Sign-based learningschemes for pattern classification. Pattern Recognition Letters, 26:1926–1936,2005.[5] J. Barzilai and J. M. Borwein. Two point step size gradient methods. IMA J.Numer. Anal., 8:141–148, 1988.[6] R. Battiti First- and second-order methods for learning: Between steepest descentand newton’s method. Neural Computation, 4:141–166, 1992.[7] E. M. L. Beale Numerical Methods for Nonlinear Optimization, chapter A derivationof conjugate gradients, pages 39–43. Academic Press, London, 1972.[8] S. Becker and Y.Le Cun. Improving the convergence of the back-propagationlearning with second order methods. In D. S. Touretzky, G. E. Hinton, T. J.Sejnowski, editor, 1988 Connectionist Models Summer School.[9] E. G. Birgin and J. M. Martinez. A spectral conjugate gradient method forunconstrained optimization. Applied Mathematics and Optimization, 43:117–128, 1999.[10] C. M. Bishop em Neural networks for pattern recognition. Oxford UniversityPress Inc, New York, 1995.145

146 Bibliografia[11] A. Bortoletti, C. Di Fiore, S. Fanelli and P. Zellini. A new class of quasi-Newtonian methods for optimal learning in MLP-networks. IEEE Trans. NeuralNetworks, 14:263–273, 2003.[12] R. P.Brent. Algorithms for minimization without derivatives. Englewood Cliffs,NJ: Prentice Hall, 1973.[13] A. E. Jr. Bryson and Y. C. Ho. Applied Optimal Control. Blaisdell, 1969.[14] A. Cauchy. Méthode générale pour la résolution des systèms d’équations simultanées.Comp. Rend. Sci. Paris, 25:536–538, 1847.[15] L. W. Chan and F. Fallside. An adaptive training algorithm for back-propagationnetworks. Computers Speech and Language, 2:205–218, 1987.[16] C. Charalambous. Conjugate gradient algorithm for efficient training of artificialneural networks. IEEE Proceedings, 139:301–310, 1992.[17] Y. H. Dai, L. Z. Liao and D. Li. On restart procedures for the conjugate gradientmethod. Numerical Algorithms, 35:249–260, 2004.[18] W. C. Davidon. Variable metric method for minimization. Technical report,Technical Report ANL-5990, Argonne National Laboratory, 1959.[19] C. Decaestecker, I. Salmon, O. Dewitte, I. Camby, P. Van Ham, J. Pasteels, J.Brotchi and R. Kiss. Nearest-neighor classification for identification of aggressiveversus nonaggressive astrocytic tumurs by means of image cytometry-generatedvariables. Journal of Neurosurgery, 86:532–537, 1997.[20] C. Di Fiore, S. Fanelli, F. Lepore and P. Zellini. Matrix algebras in quasi-Newtonmethods for unconstrained minimization. Numerische Mathematik, 94:4979–5000, 2003.[21] C. Di Fiore, S. Fanelli and P. Zellini. An efficient generalization of Battiti-Shanno’s quasi-Newton algorithm for optimal learning in MLP-networks. In N.R. Pal et al., editors, ICONIC 2004 LNCS 3316, volume 1.[22] L. C. W. Dixon, P. G. Ducksbury and P. Singth. A new three term conjugategradient method. Technical report, Technical Report No. 130, Numerical O-ptimization Centre, The Hatfield Polytechnic, Hatfield, Hertforshire, England,1985.[23] S. E. Fahlman. Faster-learning variations on back-propagation: An empiricalstudy. In D. S. Touretzky, G. E. Hinton, T. J. Sejnowski, editors, 1988 ConnectionistModels Summer School.[24] R. A. Fisher. The use of multiple measurements in taxonomic problems. AnnualEugenics, 7:179–188, 1936.

Bibliografia 147[25] R. Fletcher. Practical Methods of Optimization. New York: Wiley, 1987.[26] R. Fletcher and C. M. Reeves. Function minimization by conjugate gradients.Computer Journal, 7:149–154, 1964.[27] J. D. Gibbons. Nonparametric Statistical Inference. M. Dekker, 2nd Ed., 1985.[28] D. Glotsos, P. Spyridonos, P. Petalas, D. Cavouras, I. Ravazoula, S. Dadioti, E.Lekka and G. Nikiforidis. Supporting the regular diagnostic procedure followedby the experts in astrocytomas malignancy grading by means of an automaticclassification methodology using Support Vector Machines. Analytical andQuantitative Cytology and Histology, 26:77–83, 2004.[29] L. Grippo, F. Lampariello and S. Lucidi. A nonmonotone line search techniquefor Newton’s method. SIAM Journal of Numerical Analysis, 23:707–716, 1986.[30] J. Han, J. Sun and W. Sun. Global Convergence of Non-Monotone DescentMethods for Unconstrained Optimization Problems. Journal of Computationaland Applied Mathematics, 2002.[31] S. Haykin. Neural Networks: A Comprehensive Foundation. Prentice–Hall Inc,1999.[32] D. O. Hebb. The Organization of Behavior: A Neurophyciological. New York:Wiley, 1949.[33] M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linearsystems. Journal of Research of the National Bureau of Standards, 49:409–436,1952.[34] H. C. Hsin, C. C. Li, M. Sun and R. J. Sclabassi. An adaptive training algorithmfor back-propagation neural networks. IEEE Transactions on System, Man, andCybernetics, 25:521–514, 1995.[35] C. Igel and M. Hüsken. Empirical evaluation of the improved RPROP learningalgorithms. Neurocomputing, 50:105–123, 2003.[36] R. A. Jacobs. Increased rates of convergence through learning rate adaption.Neural Networks, 1:295–307, 1988.[37] M. Jiang, G. Gielen, B. Zhang and Z. Luo. Fast learning algorithms for feedforwardneural networks. Applied Intelligence, 18:37–54, 2003.[38] E. M. Johansson, F. U. Dowla and D. M. Goodman. Backpropagation learnningfor multi-layer feed-forward neural networks using the conjugate gradient trainingmethod. Technical report, Technical Report UCRL-JC-104850, LawrenceLivermore National Laboratory, 1990.

148 Bibliografia[39] A. Kaye and D. Walker. Low-grade astrocytomas: controversies in management.Journal of Clinical Neuroscience, 7:475–483, 2000.[40] K. M. Khoda and C. Storey. Efficient implementation of a generalized polak–ribière algorithm for nonlinear optimization. International Journal of ComputerMathematics, 48:53–61, 1993.[41] A. E. Kostopoulos, D. G. Sotiropoulos and T. N. Grapsa. A new efficient variablelearning rate for Perry’s Spectral conjugate gradient training method. In D. T.Tsahalis, editor, Proceedings of 1st International Conference ”From ScientificComputing to Computational Engineering”, volume 1. pages 66–73, 2004[42] A. E. Kostopoulos, D. Glotsos, P. Spyridonos, G. Nikiforidis, D. G. Sotiropoulosand T. N. Grapsa. Comparative evaluation of feedforward and probabilisticneural networks for the automatic classification of brain tumours. In D. T.Tsahalis, editor, Proceedings of 1st International Conference ”From ScientificComputing to Computational Engineering”, volume 1, pages 309–315, 2004.[43] A. E. Kostopoulos and T. N. Grapsa. Self-scaled conjugate gradient trainingalgorithms. Neurocomputing, 72:3000–3019, 2009.[44] Y. LeCun. Une procedure d’apprentissage pour reseau a seuil assymetrique.Cognitiva, 85:599–604, 1985.[45] Y. LeCun. Efficient Learning and Second–order Methods, A Tutorial at NIPS93. Denver, 1993.[46] D. C. Liu and J. Nocedal. On the L-BFGS method for large scale optimization.Math. Programming, 45:503–528, 1989.[47] D. Luenberger. Introduction to Linear and Nonlinear Progamming. AddisonWesley, 1984.[48] G. D. Magoulas, M. N. Vrahatis and G. S. Androulakis. Effective backpropagationwith variable stepsize. Neural Networks, 10:69–82, 1997.[49] G. D. Magoulas, M. N. Vrahatis and G. S. Androulakis. Improving the convergenceof the back-propagation algorithm using learning rate adaptation techniques.Neural Computation, 11:1769–1796, 1999.[50] W. S. McCulloch and W. Pitts. A logical calculus of the ideas immanent innervous activity. Bulletin of Mathematical Biophysics, 5:115–133, 1943.[51] M. L. Minsky and S. A. Papert. Perceptrons. Cambridge, MA: MIT Press, 1969.[52] M. F. Møller. A scaled conjugate gradient algorithm, for fast supervised learning.Neural Networks, 6:525–533, 1993.

Bibliografia 149[53] P. M. Murphy and D. W. Aha. UCI repository of machine learning databases.Technical report, University of California, Department of Informatics andComputer Science, Irvine, CA, 1994.[54] L. Nazareth. A conjugate direction algorithm without line searches. Journal ofOptimization Theory and Applications, 23:373–387, 1977.[55] D. Nguyen and B. Widrow. Improving the learning speed of 2-layer neuralnetwork by choosing initial values of the adaptive weights. Biological Cybernetics,59:257–263, 1990.[56] J. Nocedal. Thoery of algorithms for unconstrained optimization. Acta Numerica,pages 199–242, 1992.[57] J. Nocedal and J. S. Wright. Numerical Optimization. Springer-Verlag, 1999.[58] D. B. Parker. Learning –logic: Casting the cortex of the human brain in silicon.Technical Report TR–47, Center for Computational Research in Economics andManagement Sciense, Cambridge, MA: MIT Press, 1985.[59] D. B. Parker. Optimal algorithms for adaptive networks: Second order backpropagation,second order direct propagation, and second order Hebbian learning.In IEEE 1st International Conference on Neural Networks, volume 2, pages 593–600, 1987.[60] A. Perry. A modified conjugate gradient algorithm. Operations Research, 26:26–43, 1978.[61] M. Pfister and R. Rojas. Hybrid learning algorithms for neural networks - Theadaptive inclusion of second order information. Zeitschrift für Angewandte Mathematikund Mechanik, 79:215–218, 1996.[62] M. Pfister and R. Rojas. Speeding up back-propagation-acomparison of orthogonaltechniques. In Joint Conference on Neural Networks, pages 517–523, 1993.[63] V. P. Plagianakos, D. G. Sotiropoulos and M. N. Vrahatis. Recent Advances inCircuits and Systems, chapter Automatic adaptation of learning rate for backpropagationneural networks, pages 337–341. World Scientific Publishing Co.Pte. Ltd., 1998.[64] V. P. Plagianakos, G. D. Magoulas and M. N. Vrahatis. Deterministic nonmonotonestrategies for effective training of multilayer perceptrons. IEEE Trans.Neural Networks, 13:1268–1284, 2002.[65] E. Polak and G. Ribière. Note sur la convergence de méthodes de directionsconjuguées. Revue Française d’Informatique et de Recherche Opérationnelle,16:35–43, 1969.

150 Bibliografia[66] M. J. D. Powell. Restart procedures for the conjugate gradient method. MathematicalProgramming, 12:241–254, 1977.[67] R. A. Prayson, D. P. Agamanolis, M. L. Cohen and M. L. Estes. Interobserverreproducibility among neuropathologists and surgical pathologists in fibrillaryastrocytoma grading. Neurological sciences, 175:475–483, 2000.[68] S. Ramón y Cajál. Histologie du Systéms Nerveux de l’homme et des vertébrés.Paris: Maloine, 1911.[69] M. Raydan. The Barzilai and Borwein Gradient Method for the Large ScaleUnconstrained Minimization Problem. SIAM Journal of Optimization, 13:321–326, 1993.[70] M. Riedmiller and H. Braun. A direct adaptive method for faster backpropagationlearning: The rprop algorithm. In IEEE International Conferenceon Neural Networks, pages 586–591, 1993.[71] F. Rosenblatt. The Perceptron: A probabilistic model for information storageand organization in the brain. Psychological Review, 65:386–408, 1958.[72] F. Rosenblatt. On the convergence of reinforcement procedures in simple perceptrons.Technical Report VG–1196–G–4, Cornel Aeronautical Laboratory,1960.[73] D. E. Rumelhart, G. E. Hinton and R. J. Williams. Learning representations ofbackpropagation errors. Nature (London), 323:533–536, 1986.[74] D. E. Rumelhart and J. L. McClelland. Parallel Distributed Processing: Explorationsin the Microstructures of Cognition. Cambridge, MA: MIT Press, 1986.[75] L. E. Scales. Introduction to non-linear optimization. New York: Springer-Verlag,1985.[76] D. F. Shanno. Globally convergent conjugate gradient methods. MathematicalProgramming, 33:61–67, 1985.[77] D. F. Shanno and K. H. Phua. Minimization of unconstrained multivariatefunctions. ACM Transactions on Mathematical Software, 2:87–94, 1976.[78] D. F. Shanno and P. C. Kettler. Optimal conditioning of quasi-Newton methods.Mathematical Computation, 24:657–664, 1970.[79] F. Silva and Almeida. Advanced Neural Computers, chapter Speeding up Backpropagation,pages 151–158. Elsevier Science Publishers B.V. (North Holland,Amsterdam), 1990.

Bibliografia 151[80] D. G. Sotiropoulos, A. E. Kostopoulos and T. N. Grapsa. A spectral versionof Perry’s conjugate gradient method for neural network training. In D. T.Tsahalis, editor, 4th GRACM Congress on Computational Mechanics, volume 1,pages 172–179, 2002.[81] D. G. Sotiropoulos, A. E. Kostopoulos and T. N. Grapsa. Training neural networksusing two-point stepsize gradient methods. In T. E. Simos and Ch. Tsitouras,editor, Proceedings of International Conference of Numerical Analysis andApplied Mathematics 2004 (ICNAAM-2004), pages 356–359, 2004.[82] P. Spyridonos, V. Zolota, D. Cavouras, G. Zenebissis, D. Glotsos and G. Nikiforidis.A computer-based image analysis system for classification of astrocytomasaccording the WHO grading system. In Proceedings of the IASTED InternationalConference on Signal Processing Pattern Recognition and Applications,pages 371–374, 2002.[83] S. Theodoridis and K. Koutroubas. Pattern Regognition, chapter Feature GenerationII, pages 233–270. Academic Press, 1999.[84] T. P.Vogl, J. K. Mangis, J. K. Rigler, W. T. Zink and D. L. Alkon. Acceleratingthe convergence of the back-propagation method. Biological Cybernetics, 59:257–263, 1988.[85] M. N. Vrahatis, G. D. Magoulas, V. P. Plagianakos. Globally convergent modificationof the Quickprop method. Neural Processing Letters, 12:159–170, 2000.[86] P. J. Werbos. Beyond regression: New tools for prediction and analysis in thebehavioral sciences. PhD thesis, Harvard University, Cambridge, 1974.[87] B. Widrow. Self–Organizing Systems, chapter Generalization and informationstorage in networks of adeline ”neurons”, pages 435–461. Washington, DC:Spartan Books, 1962.[88] B. Widrow and M. E. Hoff, Jr. Adaptive switching circuits. IRE WESCONConvention record, pages 96–104, 1960.[89] F. Wilcoxon. Individual comparison by ranking methods. Biometrics Bull.,1:80–83, 1945.[90] J. Z. Zhang, N. Y. Deng and L. H. Chen. New quasi–Newton equation and relatedmethods for unconstrained optimization. Journal of Optimization Theory andApplications, 102:147–167, 1999.